Oct 30 21:27:25.012678 ip-10-0-129-195 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 21:27:25.380247 ip-10-0-129-195 kubenswrapper[2573]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:27:25.380247 ip-10-0-129-195 kubenswrapper[2573]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 21:27:25.380247 ip-10-0-129-195 kubenswrapper[2573]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:27:25.380247 ip-10-0-129-195 kubenswrapper[2573]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:27:25.383875 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.383788    2573 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:27:25.383875 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.383874    2573 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:27:25.383875 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.383879    2573 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:27:25.383875 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.383881    2573 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:27:25.383990 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.383885    2573 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:27:25.383990 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.383888    2573 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:27:25.383990 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.383891    2573 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:27:25.383990 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.383894    2573 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:27:25.383990 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.383897    2573 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:27:25.383990 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.383899    2573 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:27:25.383990 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.383902    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:27:25.383990 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.383905    2573 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:27:25.383990 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.383908    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:27:25.383990 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.383910    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:27:25.383990 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.383913    2573 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:27:25.383990 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.383916    2573 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:27:25.383990 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.383927    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:27:25.383990 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.383930    2573 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:27:25.383990 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.383933    2573 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:27:25.383990 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.383936    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:27:25.383990 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.383940    2573 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:27:25.383990 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.383945    2573 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:27:25.383990 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.383949    2573 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:27:25.384485 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.383952    2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:27:25.384485 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.383955    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:27:25.384485 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.383957    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:27:25.384485 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.383960    2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:27:25.384485 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.383963    2573 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:27:25.384485 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.383966    2573 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:27:25.384485 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.383969    2573 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:27:25.384485 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.383972    2573 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:27:25.384485 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.383974    2573 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:27:25.384485 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.383977    2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:27:25.384485 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.383980    2573 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:27:25.384485 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.383982    2573 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:27:25.384485 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.383985    2573 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:27:25.384485 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.383987    2573 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:27:25.384485 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.383990    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:27:25.384485 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.383993    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:27:25.384485 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.383996    2573 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:27:25.384485 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.383999    2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:27:25.384485 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.384001    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:27:25.384930 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.384004    2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:27:25.384930 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.384007    2573 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:27:25.384930 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.384010    2573 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:27:25.384930 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.384013    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:27:25.384930 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.384015    2573 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:27:25.384930 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.384018    2573 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:27:25.384930 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.384020    2573 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:27:25.384930 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.384024    2573 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:27:25.384930 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.384026    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:27:25.384930 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.384029    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:27:25.384930 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.384031    2573 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:27:25.384930 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.384034    2573 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:27:25.384930 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.384036    2573 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:27:25.384930 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.384039    2573 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:27:25.384930 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.384042    2573 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:27:25.384930 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.384046    2573 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:27:25.384930 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.384049    2573 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:27:25.384930 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.384053    2573 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:27:25.384930 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.384055    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:27:25.385422 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.384058    2573 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:27:25.385422 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.384060    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:27:25.385422 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.384063    2573 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:27:25.385422 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.384066    2573 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:27:25.385422 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.384068    2573 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:27:25.385422 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.384071    2573 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:27:25.385422 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.384073    2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:27:25.385422 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.384076    2573 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:27:25.385422 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.384079    2573 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:27:25.385422 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.384081    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:27:25.385422 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.384084    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:27:25.385422 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.384087    2573 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:27:25.385422 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.384090    2573 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:27:25.385422 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.384093    2573 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:27:25.385422 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.384095    2573 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:27:25.385422 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.384098    2573 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:27:25.385422 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.384100    2573 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:27:25.385422 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.384103    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:27:25.385422 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.384105    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:27:25.385422 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.384108    2573 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:27:25.385891 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.384110    2573 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:27:25.385891 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.384114    2573 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:27:25.385891 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.384116    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:27:25.385891 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.384119    2573 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:27:25.385891 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.384121    2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:27:25.385891 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.384124    2573 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:27:25.385891 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.384126    2573 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:27:25.385891 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.384891    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:27:25.385891 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.384897    2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:27:25.385891 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.384900    2573 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:27:25.385891 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.384904    2573 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:27:25.385891 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.384907    2573 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:27:25.385891 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.384911    2573 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:27:25.385891 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.384914    2573 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:27:25.385891 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.384917    2573 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:27:25.385891 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.384920    2573 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:27:25.385891 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.384922    2573 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:27:25.385891 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.384925    2573 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:27:25.385891 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.384928    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:27:25.385891 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.384930    2573 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:27:25.386371 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.384933    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:27:25.386371 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.384936    2573 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:27:25.386371 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.384939    2573 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:27:25.386371 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.384942    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:27:25.386371 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.384946    2573 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:27:25.386371 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385767    2573 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:27:25.386371 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385776    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:27:25.386371 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385779    2573 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:27:25.386371 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385782    2573 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:27:25.386371 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385784    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:27:25.386371 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385787    2573 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:27:25.386371 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385790    2573 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:27:25.386371 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385793    2573 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:27:25.386371 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385796    2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:27:25.386371 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385799    2573 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:27:25.386371 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385802    2573 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:27:25.386371 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385805    2573 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:27:25.386371 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385807    2573 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:27:25.386371 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385810    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:27:25.386371 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385813    2573 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:27:25.386850 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385815    2573 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:27:25.386850 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385818    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:27:25.386850 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385821    2573 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:27:25.386850 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385824    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:27:25.386850 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385826    2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:27:25.386850 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385829    2573 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:27:25.386850 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385831    2573 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:27:25.386850 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385834    2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:27:25.386850 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385838    2573 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:27:25.386850 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385842    2573 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:27:25.386850 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385846    2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:27:25.386850 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385848    2573 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:27:25.386850 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385852    2573 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:27:25.386850 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385855    2573 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:27:25.386850 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385858    2573 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:27:25.386850 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385860    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:27:25.386850 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385863    2573 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:27:25.386850 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385865    2573 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:27:25.386850 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385868    2573 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:27:25.387302 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385871    2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:27:25.387302 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385873    2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:27:25.387302 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385876    2573 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:27:25.387302 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385878    2573 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:27:25.387302 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385883    2573 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:27:25.387302 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385886    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:27:25.387302 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385889    2573 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:27:25.387302 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385892    2573 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:27:25.387302 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385895    2573 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:27:25.387302 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385898    2573 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:27:25.387302 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385901    2573 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:27:25.387302 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385904    2573 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:27:25.387302 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385906    2573 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:27:25.387302 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385909    2573 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:27:25.387302 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385911    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:27:25.387302 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385914    2573 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:27:25.387302 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385917    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:27:25.387302 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385919    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:27:25.387302 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385922    2573 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:27:25.387773 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385925    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:27:25.387773 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385927    2573 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:27:25.387773 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385930    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:27:25.387773 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385933    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:27:25.387773 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385936    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:27:25.387773 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385938    2573 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:27:25.387773 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385941    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:27:25.387773 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385944    2573 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:27:25.387773 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385947    2573 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:27:25.387773 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385949    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:27:25.387773 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385952    2573 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:27:25.387773 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385954    2573 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:27:25.387773 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385957    2573 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:27:25.387773 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385959    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:27:25.387773 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385961    2573 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:27:25.387773 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385967    2573 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:27:25.387773 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385969    2573 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:27:25.387773 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385972    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:27:25.387773 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385974    2573 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:27:25.388255 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385977    2573 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:27:25.388255 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385979    2573 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:27:25.388255 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385982    2573 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:27:25.388255 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385985    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:27:25.388255 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385987    2573 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:27:25.388255 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385991    2573 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:27:25.388255 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385994    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:27:25.388255 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.385997    2573 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:27:25.388255 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386000    2573 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:27:25.388255 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386003    2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:27:25.388255 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386006    2573 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:27:25.388255 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386009    2573 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:27:25.388255 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386011    2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:27:25.388255 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386014    2573 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:27:25.388255 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386016    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:27:25.388255 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386019    2573 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:27:25.388255 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386021    2573 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:27:25.388255 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386024    2573 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:27:25.388255 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386027    2573 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:27:25.388255 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386029    2573 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:27:25.388745 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386032    2573 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:27:25.388745 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386034    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:27:25.388745 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386037    2573 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:27:25.388745 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386039    2573 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:27:25.388745 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386042    2573 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:27:25.388745 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386044    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:27:25.388745 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386047    2573 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:27:25.388745 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386049    2573 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:27:25.388745 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386053    2573 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:27:25.388745 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386055    2573 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:27:25.388745 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386058    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:27:25.388745 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386060    2573 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:27:25.388745 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386063    2573 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:27:25.388745 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386067    2573 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:27:25.388745 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386584    2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:27:25.388745 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386590    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:27:25.388745 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386593    2573 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:27:25.388745 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386596    2573 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:27:25.388745 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386599    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:27:25.389203 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386602    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:27:25.389203 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386605    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:27:25.389203 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386607    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:27:25.389203 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386610    2573 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:27:25.389203 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386613    2573 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:27:25.389203 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386616    2573 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:27:25.389203 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386618    2573 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:27:25.389203 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386621    2573 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:27:25.389203 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386624    2573 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:27:25.389203 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386626    2573 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:27:25.389203 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386629    2573 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:27:25.389203 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386632    2573 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:27:25.389203 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386634    2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:27:25.389203 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386637    2573 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:27:25.389203 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386640    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:27:25.389203 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386642    2573 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:27:25.389203 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386645    2573 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:27:25.389203 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386648    2573 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:27:25.389203 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386650    2573 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:27:25.389203 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386653    2573 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:27:25.389682 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386656    2573 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:27:25.389682 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386659    2573 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:27:25.389682 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386661    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:27:25.389682 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386664    2573 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:27:25.389682 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386667    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:27:25.389682 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386669    2573 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:27:25.389682 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386672    2573 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:27:25.389682 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386674    2573 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:27:25.389682 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386676    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:27:25.389682 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386679    2573 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:27:25.389682 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386682    2573 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:27:25.389682 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386685    2573 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:27:25.389682 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386687    2573 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:27:25.389682 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386690    2573 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:27:25.389682 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386692    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:27:25.389682 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386695    2573 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:27:25.389682 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386697    2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:27:25.389682 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386700    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:27:25.389682 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386702    2573 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:27:25.389682 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386704    2573 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:27:25.390142 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386707    2573 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:27:25.390142 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386710    2573 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:27:25.390142 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386712    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:27:25.390142 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386714    2573 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:27:25.390142 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386717    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:27:25.390142 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386719    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:27:25.390142 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386722    2573 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:27:25.390142 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386725    2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:27:25.390142 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386727    2573 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:27:25.390142 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386729    2573 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:27:25.390142 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386732    2573 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:27:25.390142 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386735    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:27:25.390142 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386737    2573 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:27:25.390142 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386740    2573 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:27:25.390142 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386742    2573 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:27:25.390142 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386745    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:27:25.390142 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386747    2573 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:27:25.390142 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386750    2573 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:27:25.390142 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386752    2573 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:27:25.390142 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386754    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:27:25.390677 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386757    2573 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:27:25.390677 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386759    2573 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:27:25.390677 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386763    2573 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:27:25.390677 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386766    2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:27:25.390677 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386770    2573 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:27:25.390677 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386772    2573 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:27:25.390677 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386775    2573 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:27:25.390677 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386777    2573 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:27:25.390677 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386780    2573 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:27:25.390677 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386783    2573 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:27:25.390677 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386785    2573 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:27:25.390677 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386787    2573 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:27:25.390677 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386791    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:27:25.390677 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386793    2573 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:27:25.390677 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386796    2573 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:27:25.390677 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386799    2573 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:27:25.390677 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386801    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:27:25.390677 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386804    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:27:25.390677 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386807    2573 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:27:25.391418 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386811    2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:27:25.391418 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386815    2573 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:27:25.391418 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386818    2573 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:27:25.391418 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386820    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:27:25.391418 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386823    2573 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:27:25.391418 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386825    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:27:25.391418 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386828    2573 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:27:25.391418 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386830    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:27:25.391418 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386835    2573 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:27:25.391418 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386837    2573 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:27:25.391418 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386840    2573 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:27:25.391418 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386844    2573 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:27:25.391418 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386849    2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:27:25.391418 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386853    2573 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:27:25.391418 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386856    2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:27:25.391418 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386859    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:27:25.391418 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386863    2573 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:27:25.391418 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386866    2573 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:27:25.391418 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386868    2573 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:27:25.392085 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386870    2573 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:27:25.392085 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386873    2573 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:27:25.392085 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.386875    2573 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:27:25.392085 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388342    2573 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 21:27:25.392085 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388353    2573 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 21:27:25.392085 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388374    2573 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 21:27:25.392085 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388379    2573 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 21:27:25.392085 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388387    2573 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 21:27:25.392085 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388390    2573 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 21:27:25.392085 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388394    2573 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 21:27:25.392085 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388398    2573 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 21:27:25.392085 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388403    2573 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 21:27:25.392085 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388406    2573 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 21:27:25.392085 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388410    2573 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 21:27:25.392085 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388413    2573 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 21:27:25.392085 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388416    2573 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 21:27:25.392085 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388419    2573 flags.go:64] FLAG: --cgroup-root=""
Oct 30 21:27:25.392085 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388422    2573 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 21:27:25.392085 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388425    2573 flags.go:64] FLAG: --client-ca-file=""
Oct 30 21:27:25.392085 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388428    2573 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 21:27:25.392085 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388431    2573 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 21:27:25.392085 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388436    2573 flags.go:64] FLAG: --cluster-domain=""
Oct 30 21:27:25.392085 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388439    2573 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 21:27:25.392085 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388442    2573 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 21:27:25.392673 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388445    2573 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 21:27:25.392673 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388448    2573 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 21:27:25.392673 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388452    2573 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 21:27:25.392673 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388455    2573 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 21:27:25.392673 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388460    2573 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 21:27:25.392673 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388464    2573 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 21:27:25.392673 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388467    2573 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 21:27:25.392673 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388470    2573 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 21:27:25.392673 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388473    2573 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 21:27:25.392673 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388477    2573 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 21:27:25.392673 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388480    2573 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 21:27:25.392673 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388484    2573 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 21:27:25.392673 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388487    2573 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 21:27:25.392673 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388490    2573 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 21:27:25.392673 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388493    2573 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 21:27:25.392673 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388496    2573 flags.go:64] FLAG: --enable-server="true"
Oct 30 21:27:25.392673 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388499    2573 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 21:27:25.392673 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388503    2573 flags.go:64] FLAG: --event-burst="100"
Oct 30 21:27:25.392673 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388507    2573 flags.go:64] FLAG: --event-qps="50"
Oct 30 21:27:25.392673 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388510    2573 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 21:27:25.392673 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388513    2573 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 21:27:25.392673 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388516    2573 flags.go:64] FLAG: --eviction-hard=""
Oct 30 21:27:25.392673 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388520    2573 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 21:27:25.392673 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388523    2573 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 21:27:25.393301 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388526    2573 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 21:27:25.393301 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388529    2573 flags.go:64] FLAG: --eviction-soft=""
Oct 30 21:27:25.393301 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388532    2573 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 21:27:25.393301 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388535    2573 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 21:27:25.393301 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388538    2573 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 21:27:25.393301 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388541    2573 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 21:27:25.393301 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388544    2573 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 21:27:25.393301 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388547    2573 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 21:27:25.393301 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388550    2573 flags.go:64] FLAG: --feature-gates=""
Oct 30 21:27:25.393301 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388554    2573 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 21:27:25.393301 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388558    2573 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 21:27:25.393301 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388561    2573 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 21:27:25.393301 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388565    2573 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 21:27:25.393301 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388568    2573 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 21:27:25.393301 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388571    2573 flags.go:64] FLAG: --help="false"
Oct 30 21:27:25.393301 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388575    2573 flags.go:64] FLAG: --hostname-override="ip-10-0-129-195.us-east-2.compute.internal"
Oct 30 21:27:25.393301 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388578    2573 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 21:27:25.393301 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388581    2573 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 21:27:25.393301 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388584    2573 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 21:27:25.393301 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388588    2573 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 21:27:25.393301 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388591    2573 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 21:27:25.393301 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388594    2573 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 21:27:25.393301 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388597    2573 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 21:27:25.393913 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388599    2573 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 21:27:25.393913 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388602    2573 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 21:27:25.393913 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388605    2573 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 21:27:25.393913 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388608    2573 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 21:27:25.393913 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388611    2573 flags.go:64] FLAG: --kube-reserved=""
Oct 30 21:27:25.393913 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388614    2573 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 21:27:25.393913 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388617    2573 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 21:27:25.393913 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388620    2573 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 21:27:25.393913 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388623    2573 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 21:27:25.393913 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388626    2573 flags.go:64] FLAG: --lock-file=""
Oct 30 21:27:25.393913 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388629    2573 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 21:27:25.393913 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388632    2573 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 21:27:25.393913 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388635    2573 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 21:27:25.393913 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388641    2573 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 21:27:25.393913 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388643    2573 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 21:27:25.393913 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388646    2573 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 21:27:25.393913 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388649    2573 flags.go:64] FLAG: --logging-format="text"
Oct 30 21:27:25.393913 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388652    2573 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 21:27:25.393913 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388656    2573 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 21:27:25.393913 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388659    2573 flags.go:64] FLAG: --manifest-url=""
Oct 30 21:27:25.393913 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388662    2573 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 21:27:25.393913 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388667    2573 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 21:27:25.393913 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388670    2573 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 21:27:25.393913 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388674    2573 flags.go:64] FLAG: --max-pods="110"
Oct 30 21:27:25.393913 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388678    2573 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 21:27:25.394523 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388681    2573 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 21:27:25.394523 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388684    2573 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 21:27:25.394523 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388687    2573 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 21:27:25.394523 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388690    2573 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 21:27:25.394523 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388693    2573 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 21:27:25.394523 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388695    2573 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 21:27:25.394523 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388703    2573 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 21:27:25.394523 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388706    2573 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 21:27:25.394523 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388709    2573 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 21:27:25.394523 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388712    2573 flags.go:64] FLAG: --pod-cidr=""
Oct 30 21:27:25.394523 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388715    2573 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 21:27:25.394523 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388717    2573 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 21:27:25.394523 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388720    2573 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 21:27:25.394523 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388723    2573 flags.go:64] FLAG: --port="10250"
Oct 30 21:27:25.394523 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388726    2573 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 21:27:25.394523 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388729    2573 flags.go:64] FLAG: --provider-id="aws:///us-east-2a/i-07ceafd550d2fa8d9"
Oct 30 21:27:25.394523 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388732    2573 flags.go:64] FLAG: --qos-reserved=""
Oct 30 21:27:25.394523 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388736    2573 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 21:27:25.394523 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388739    2573 flags.go:64] FLAG: --register-node="true"
Oct 30 21:27:25.394523 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388742    2573 flags.go:64] FLAG: --register-with-taints=""
Oct 30 21:27:25.394523 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388746    2573 flags.go:64] FLAG: --registry-burst="10"
Oct 30 21:27:25.394523 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388749    2573 flags.go:64] FLAG: --registry-qps="5"
Oct 30 21:27:25.394523 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388751    2573 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 21:27:25.394523 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388754    2573 flags.go:64] FLAG: --reserved-memory=""
Oct 30 21:27:25.394523 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388758    2573 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 21:27:25.394523 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388762    2573 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 21:27:25.395128 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388765    2573 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 21:27:25.395128 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388768    2573 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 21:27:25.395128 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388772    2573 flags.go:64] FLAG: --runonce="false"
Oct 30 21:27:25.395128 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388775    2573 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 21:27:25.395128 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388778    2573 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 21:27:25.395128 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388781    2573 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 21:27:25.395128 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388784    2573 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 21:27:25.395128 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388787    2573 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 21:27:25.395128 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388790    2573 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 21:27:25.395128 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388794    2573 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 21:27:25.395128 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388797    2573 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 21:27:25.395128 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388800    2573 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 21:27:25.395128 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388803    2573 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 21:27:25.395128 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388806    2573 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 21:27:25.395128 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388809    2573 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 21:27:25.395128 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388812    2573 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 21:27:25.395128 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388815    2573 flags.go:64] FLAG: --system-cgroups=""
Oct 30 21:27:25.395128 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388817    2573 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 21:27:25.395128 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388823    2573 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 21:27:25.395128 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388826    2573 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 21:27:25.395128 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388829    2573 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 21:27:25.395128 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388832    2573 flags.go:64] FLAG: --tls-min-version=""
Oct 30 21:27:25.395128 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388835    2573 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 21:27:25.395128 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388839    2573 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 21:27:25.395128 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388842    2573 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 21:27:25.395929 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388845    2573 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 21:27:25.395929 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388848    2573 flags.go:64] FLAG: --v="2"
Oct 30 21:27:25.395929 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388853    2573 flags.go:64] FLAG: --version="false"
Oct 30 21:27:25.395929 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388858    2573 flags.go:64] FLAG: --vmodule=""
Oct 30 21:27:25.395929 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388861    2573 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 21:27:25.395929 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.388865    2573 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 21:27:25.395929 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389001    2573 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:27:25.395929 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389005    2573 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:27:25.395929 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389007    2573 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:27:25.395929 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389012    2573 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:27:25.395929 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389015    2573 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:27:25.395929 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389018    2573 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:27:25.395929 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389021    2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:27:25.395929 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389024    2573 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:27:25.395929 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389027    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:27:25.395929 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389030    2573 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:27:25.395929 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389033    2573 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:27:25.395929 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389036    2573 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:27:25.395929 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389039    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:27:25.395929 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389042    2573 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:27:25.395929 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389044    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:27:25.396478 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389047    2573 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:27:25.396478 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389050    2573 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:27:25.396478 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389052    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:27:25.396478 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389055    2573 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:27:25.396478 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389057    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:27:25.396478 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389060    2573 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:27:25.396478 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389062    2573 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:27:25.396478 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389065    2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:27:25.396478 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389067    2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:27:25.396478 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389071    2573 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:27:25.396478 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389074    2573 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:27:25.396478 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389077    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:27:25.396478 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389079    2573 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:27:25.396478 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389082    2573 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:27:25.396478 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389085    2573 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:27:25.396478 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389087    2573 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:27:25.396478 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389090    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:27:25.396478 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389092    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:27:25.396478 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389095    2573 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:27:25.396478 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389097    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:27:25.396953 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389101    2573 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:27:25.396953 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389104    2573 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:27:25.396953 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389106    2573 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:27:25.396953 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389109    2573 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:27:25.396953 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389111    2573 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:27:25.396953 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389114    2573 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:27:25.396953 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389116    2573 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:27:25.396953 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389119    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:27:25.396953 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389122    2573 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:27:25.396953 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389124    2573 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:27:25.396953 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389127    2573 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:27:25.396953 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389131    2573 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:27:25.396953 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389135    2573 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:27:25.396953 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389137    2573 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:27:25.396953 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389140    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:27:25.396953 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389143    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:27:25.396953 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389145    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:27:25.396953 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389149    2573 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:27:25.396953 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389152    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:27:25.397417 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389155    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:27:25.397417 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389158    2573 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:27:25.397417 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389162    2573 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:27:25.397417 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389165    2573 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:27:25.397417 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389167    2573 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:27:25.397417 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389170    2573 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:27:25.397417 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389173    2573 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:27:25.397417 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389175    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:27:25.397417 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389178    2573 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:27:25.397417 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389180    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:27:25.397417 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389183    2573 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:27:25.397417 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389185    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:27:25.397417 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389188    2573 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:27:25.397417 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389192    2573 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:27:25.397417 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389195    2573 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:27:25.397417 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389197    2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:27:25.397417 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389200    2573 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:27:25.397417 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389202    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:27:25.397417 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389205    2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:27:25.397417 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389208    2573 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:27:25.397920 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389211    2573 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:27:25.397920 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389214    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:27:25.397920 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389216    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:27:25.397920 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389219    2573 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:27:25.397920 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389222    2573 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:27:25.397920 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389224    2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:27:25.397920 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389226    2573 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:27:25.397920 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389229    2573 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:27:25.397920 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389232    2573 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:27:25.397920 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389234    2573 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:27:25.397920 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389237    2573 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:27:25.397920 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389239    2573 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:27:25.397920 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389241    2573 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:27:25.397920 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389244    2573 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:27:25.397920 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389248    2573 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:27:25.397920 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389250    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:27:25.397920 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389253    2573 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:27:25.397920 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389255    2573 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:27:25.397920 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389258    2573 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:27:25.397920 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389261    2573 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:27:25.398399 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389263    2573 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:27:25.398399 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389266    2573 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:27:25.398399 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389268    2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:27:25.398399 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389271    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:27:25.398399 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389273    2573 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:27:25.398399 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389277    2573 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:27:25.398399 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389279    2573 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:27:25.398399 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389282    2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:27:25.398399 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389284    2573 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:27:25.398399 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389287    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:27:25.398399 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389289    2573 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:27:25.398399 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389292    2573 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:27:25.398399 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.389317    2573 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:27:25.398399 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389356    2573 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:27:25.398399 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389377    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:27:25.398756 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389382    2573 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:27:25.398756 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389386    2573 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:27:25.398756 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389390    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:27:25.398756 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389393    2573 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:27:25.398756 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389395    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:27:25.398756 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389398    2573 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:27:25.398756 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389401    2573 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:27:25.398756 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389403    2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:27:25.398756 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389406    2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:27:25.398756 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389409    2573 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:27:25.398756 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389411    2573 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:27:25.398756 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389414    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:27:25.398756 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389419    2573 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:27:25.398756 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389421    2573 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:27:25.398756 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389424    2573 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:27:25.398756 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389427    2573 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:27:25.398756 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389429    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:27:25.398756 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389432    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:27:25.398756 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389435    2573 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:27:25.398756 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389437    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:27:25.399218 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389440    2573 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:27:25.399218 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389442    2573 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:27:25.399218 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389446    2573 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:27:25.399218 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389449    2573 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:27:25.399218 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389451    2573 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:27:25.399218 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389454    2573 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:27:25.399218 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389456    2573 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:27:25.399218 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389459    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:27:25.399218 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389461    2573 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:27:25.399218 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389464    2573 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:27:25.399218 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389466    2573 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:27:25.399218 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389470    2573 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:27:25.399218 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389475    2573 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:27:25.399218 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389477    2573 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:27:25.399218 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389480    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:27:25.399218 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389483    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:27:25.399218 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389485    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:27:25.399218 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389490    2573 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:27:25.399218 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389494    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:27:25.399727 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389497    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:27:25.399727 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389500    2573 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:27:25.399727 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389502    2573 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:27:25.399727 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389505    2573 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:27:25.399727 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389507    2573 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:27:25.399727 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389511    2573 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:27:25.399727 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389513    2573 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:27:25.399727 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389516    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:27:25.399727 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389519    2573 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:27:25.399727 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389521    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:27:25.399727 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389524    2573 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:27:25.399727 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389526    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:27:25.399727 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389529    2573 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:27:25.399727 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389546    2573 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:27:25.399727 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389550    2573 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:27:25.399727 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389555    2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:27:25.399727 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389558    2573 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:27:25.399727 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389560    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:27:25.399727 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389563    2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:27:25.399727 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389566    2573 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:27:25.400211 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389568    2573 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:27:25.400211 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389571    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:27:25.400211 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389573    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:27:25.400211 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389576    2573 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:27:25.400211 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389578    2573 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:27:25.400211 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389581    2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:27:25.400211 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389583    2573 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:27:25.400211 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389586    2573 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:27:25.400211 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389589    2573 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:27:25.400211 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389591    2573 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:27:25.400211 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389594    2573 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:27:25.400211 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389596    2573 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:27:25.400211 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389599    2573 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:27:25.400211 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389601    2573 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:27:25.400211 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389604    2573 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:27:25.400211 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389606    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:27:25.400211 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389609    2573 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:27:25.400211 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389613    2573 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:27:25.400211 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389615    2573 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:27:25.400211 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389618    2573 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:27:25.400693 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389621    2573 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:27:25.400693 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389623    2573 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:27:25.400693 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389626    2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:27:25.400693 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389628    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:27:25.400693 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389631    2573 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:27:25.400693 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389633    2573 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:27:25.400693 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389636    2573 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:27:25.400693 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389639    2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:27:25.400693 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389642    2573 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:27:25.400693 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389645    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:27:25.400693 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389647    2573 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:27:25.400693 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389650    2573 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:27:25.400693 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389652    2573 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:27:25.400693 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389654    2573 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:27:25.400693 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389657    2573 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:27:25.400693 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389660    2573 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:27:25.400693 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389662    2573 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:27:25.400693 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389665    2573 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:27:25.400693 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389667    2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:27:25.401150 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389670    2573 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:27:25.401150 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389673    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:27:25.401150 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389675    2573 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:27:25.401150 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389678    2573 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:27:25.401150 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389680    2573 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:27:25.401150 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.389683    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:27:25.401150 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.397851    2573 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 21:27:25.401150 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.397871    2573 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 21:27:25.401150 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.397887    2573 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 21:27:25.401150 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.397892    2573 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 21:27:25.401150 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.397955    2573 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:27:25.401150 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.397960    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:27:25.401150 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.397964    2573 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:27:25.401150 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.397968    2573 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:27:25.401150 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.397970    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:27:25.401150 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.397973    2573 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:27:25.401150 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.397976    2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:27:25.401150 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.397978    2573 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:27:25.401150 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.397983    2573 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:27:25.401629 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.397987    2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:27:25.401629 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.397990    2573 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:27:25.401629 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.397993    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:27:25.401629 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.397996    2573 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:27:25.401629 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.397999    2573 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:27:25.401629 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398002    2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:27:25.401629 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398004    2573 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:27:25.401629 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398007    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:27:25.401629 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398010    2573 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:27:25.401629 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398013    2573 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:27:25.401629 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398017    2573 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:27:25.401629 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398020    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:27:25.401629 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398022    2573 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:27:25.401629 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398025    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:27:25.401629 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398028    2573 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:27:25.401629 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398030    2573 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:27:25.401629 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398033    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:27:25.401629 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398035    2573 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:27:25.401629 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398038    2573 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:27:25.401629 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398040    2573 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:27:25.402097 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398044    2573 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:27:25.402097 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398047    2573 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:27:25.402097 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398050    2573 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:27:25.402097 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398053    2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:27:25.402097 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398055    2573 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:27:25.402097 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398058    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:27:25.402097 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398060    2573 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:27:25.402097 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398063    2573 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:27:25.402097 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398065    2573 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:27:25.402097 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398068    2573 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:27:25.402097 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398071    2573 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:27:25.402097 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398073    2573 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:27:25.402097 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398076    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:27:25.402097 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398078    2573 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:27:25.402097 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398083    2573 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:27:25.402097 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398087    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:27:25.402097 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398090    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:27:25.402097 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398093    2573 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:27:25.402097 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398096    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:27:25.402559 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398099    2573 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:27:25.402559 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398102    2573 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:27:25.402559 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398104    2573 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:27:25.402559 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398107    2573 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:27:25.402559 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398110    2573 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:27:25.402559 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398113    2573 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:27:25.402559 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398115    2573 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:27:25.402559 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398118    2573 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:27:25.402559 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398121    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:27:25.402559 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398123    2573 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:27:25.402559 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398126    2573 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:27:25.402559 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398128    2573 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:27:25.402559 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398131    2573 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:27:25.402559 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398134    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:27:25.402559 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398138    2573 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:27:25.402559 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398141    2573 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:27:25.402559 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398143    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:27:25.402559 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398146    2573 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:27:25.402559 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398148    2573 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:27:25.402559 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398151    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:27:25.403054 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398154    2573 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:27:25.403054 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398156    2573 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:27:25.403054 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398159    2573 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:27:25.403054 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398161    2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:27:25.403054 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398164    2573 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:27:25.403054 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398166    2573 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:27:25.403054 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398169    2573 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:27:25.403054 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398171    2573 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:27:25.403054 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398174    2573 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:27:25.403054 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398177    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:27:25.403054 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398179    2573 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:27:25.403054 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398182    2573 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:27:25.403054 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398185    2573 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:27:25.403054 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398187    2573 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:27:25.403054 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398190    2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:27:25.403054 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398192    2573 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:27:25.403054 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398195    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:27:25.403054 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398197    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:27:25.403054 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398200    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:27:25.403560 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398203    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:27:25.403560 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398205    2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:27:25.403560 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398208    2573 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:27:25.403560 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398210    2573 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:27:25.403560 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398213    2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:27:25.403560 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398215    2573 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:27:25.403560 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398219    2573 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:27:25.403560 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398222    2573 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:27:25.403560 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398224    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:27:25.403560 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398227    2573 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:27:25.403560 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398229    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:27:25.403560 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398232    2573 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:27:25.403560 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398234    2573 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:27:25.403560 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398237    2573 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:27:25.403560 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398239    2573 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:27:25.403560 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398242    2573 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:27:25.403560 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398244    2573 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:27:25.403560 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398247    2573 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:27:25.403560 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398250    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:27:25.404035 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.398278    2573 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:27:25.404035 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398436    2573 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:27:25.404035 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398441    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:27:25.404035 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398444    2573 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:27:25.404035 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398447    2573 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:27:25.404035 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398449    2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:27:25.404035 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398453    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:27:25.404035 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398455    2573 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:27:25.404035 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398458    2573 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:27:25.404035 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398462    2573 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:27:25.404035 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398466    2573 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:27:25.404035 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398470    2573 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:27:25.404035 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398473    2573 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:27:25.404035 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398476    2573 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:27:25.404035 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398479    2573 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:27:25.404035 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398482    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:27:25.404461 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398485    2573 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:27:25.404461 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398488    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:27:25.404461 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398491    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:27:25.404461 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398494    2573 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:27:25.404461 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398496    2573 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:27:25.404461 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398499    2573 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:27:25.404461 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398502    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:27:25.404461 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398504    2573 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:27:25.404461 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398508    2573 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:27:25.404461 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398512    2573 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:27:25.404461 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398515    2573 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:27:25.404461 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398518    2573 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:27:25.404461 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398520    2573 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:27:25.404461 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398523    2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:27:25.404461 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398525    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:27:25.404461 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398528    2573 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:27:25.404461 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398531    2573 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:27:25.404461 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398533    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:27:25.404461 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398536    2573 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:27:25.404904 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398538    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:27:25.404904 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398541    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:27:25.404904 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398543    2573 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:27:25.404904 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398546    2573 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:27:25.404904 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398549    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:27:25.404904 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398552    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:27:25.404904 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398554    2573 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:27:25.404904 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398557    2573 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:27:25.404904 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398560    2573 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:27:25.404904 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398562    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:27:25.404904 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398565    2573 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:27:25.404904 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398567    2573 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:27:25.404904 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398570    2573 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:27:25.404904 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398573    2573 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:27:25.404904 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398575    2573 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:27:25.404904 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398578    2573 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:27:25.404904 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398581    2573 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:27:25.404904 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398584    2573 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:27:25.404904 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398587    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:27:25.405397 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398589    2573 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:27:25.405397 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398592    2573 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:27:25.405397 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398594    2573 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:27:25.405397 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398597    2573 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:27:25.405397 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398599    2573 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:27:25.405397 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398602    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:27:25.405397 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398604    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:27:25.405397 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398607    2573 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:27:25.405397 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398609    2573 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:27:25.405397 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398612    2573 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:27:25.405397 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398614    2573 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:27:25.405397 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398617    2573 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:27:25.405397 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398619    2573 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:27:25.405397 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398622    2573 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:27:25.405397 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398625    2573 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:27:25.405397 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398628    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:27:25.405397 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398630    2573 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:27:25.405397 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398633    2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:27:25.405397 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398640    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:27:25.405397 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398643    2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:27:25.405935 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398646    2573 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:27:25.405935 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398649    2573 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:27:25.405935 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398652    2573 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:27:25.405935 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398654    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:27:25.405935 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398657    2573 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:27:25.405935 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398659    2573 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:27:25.405935 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398662    2573 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:27:25.405935 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398665    2573 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:27:25.405935 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398667    2573 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:27:25.405935 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398670    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:27:25.405935 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398673    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:27:25.405935 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398676    2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:27:25.405935 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398678    2573 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:27:25.405935 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398681    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:27:25.405935 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398683    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:27:25.405935 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398686    2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:27:25.405935 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398689    2573 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:27:25.405935 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398691    2573 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:27:25.405935 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398694    2573 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:27:25.406533 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398697    2573 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:27:25.406533 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398699    2573 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:27:25.406533 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398702    2573 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:27:25.406533 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398704    2573 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:27:25.406533 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398707    2573 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:27:25.406533 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398709    2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:27:25.406533 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398712    2573 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:27:25.406533 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398715    2573 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:27:25.406533 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398718    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:27:25.406533 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398720    2573 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:27:25.406533 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398723    2573 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:27:25.406533 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398725    2573 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:27:25.406533 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398728    2573 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:27:25.406533 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398731    2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:27:25.406533 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.398755    2573 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:27:25.406533 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398792    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:27:25.406920 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398796    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:27:25.406920 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398800    2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:27:25.406920 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398803    2573 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:27:25.406920 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398805    2573 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:27:25.406920 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398808    2573 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:27:25.406920 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398811    2573 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:27:25.406920 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398814    2573 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:27:25.406920 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398816    2573 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:27:25.406920 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398819    2573 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:27:25.406920 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398821    2573 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:27:25.406920 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398824    2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:27:25.406920 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398827    2573 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:27:25.406920 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398829    2573 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:27:25.406920 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398832    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:27:25.406920 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398834    2573 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:27:25.406920 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398837    2573 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:27:25.406920 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398839    2573 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:27:25.406920 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398842    2573 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:27:25.406920 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398844    2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:27:25.407493 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398850    2573 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:27:25.407493 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398852    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:27:25.407493 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398855    2573 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:27:25.407493 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398858    2573 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:27:25.407493 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398860    2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:27:25.407493 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398863    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:27:25.407493 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398866    2573 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:27:25.407493 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398868    2573 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:27:25.407493 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398872    2573 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:27:25.407493 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398876    2573 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:27:25.407493 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398886    2573 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:27:25.407493 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398889    2573 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:27:25.407493 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398892    2573 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:27:25.407493 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398894    2573 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:27:25.407493 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398897    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:27:25.407493 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398900    2573 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:27:25.407493 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398902    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:27:25.407493 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398905    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:27:25.407493 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398908    2573 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:27:25.407932 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398910    2573 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:27:25.407932 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398913    2573 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:27:25.407932 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398915    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:27:25.407932 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398918    2573 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:27:25.407932 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398922    2573 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:27:25.407932 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398926    2573 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:27:25.407932 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398928    2573 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:27:25.407932 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398931    2573 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:27:25.407932 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398933    2573 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:27:25.407932 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398936    2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:27:25.407932 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398938    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:27:25.407932 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398941    2573 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:27:25.407932 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398944    2573 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:27:25.407932 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398948    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:27:25.407932 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398950    2573 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:27:25.407932 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398953    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:27:25.407932 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398956    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:27:25.407932 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398958    2573 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:27:25.407932 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398961    2573 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:27:25.408405 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398964    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:27:25.408405 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398966    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:27:25.408405 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398969    2573 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:27:25.408405 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398971    2573 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:27:25.408405 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398974    2573 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:27:25.408405 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398977    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:27:25.408405 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398979    2573 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:27:25.408405 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398982    2573 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:27:25.408405 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398985    2573 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:27:25.408405 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398988    2573 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:27:25.408405 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398991    2573 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:27:25.408405 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398993    2573 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:27:25.408405 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398996    2573 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:27:25.408405 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.398999    2573 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:27:25.408405 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.399001    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:27:25.408405 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.399004    2573 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:27:25.408405 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.399006    2573 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:27:25.408405 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.399009    2573 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:27:25.408405 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.399012    2573 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:27:25.408874 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.399014    2573 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:27:25.408874 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.399017    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:27:25.408874 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.399019    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:27:25.408874 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.399022    2573 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:27:25.408874 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.399024    2573 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:27:25.408874 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.399027    2573 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:27:25.408874 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.399029    2573 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:27:25.408874 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.399035    2573 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:27:25.408874 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.399037    2573 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:27:25.408874 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.399040    2573 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:27:25.408874 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.399043    2573 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:27:25.408874 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.399046    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:27:25.408874 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.399049    2573 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:27:25.408874 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.399051    2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:27:25.408874 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.399054    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:27:25.408874 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.399075    2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:27:25.408874 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.399079    2573 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:27:25.408874 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.399083    2573 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:27:25.408874 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.399086    2573 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:27:25.408874 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.399089    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:27:25.409344 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.399092    2573 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:27:25.409344 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.399094    2573 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:27:25.409344 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.399097    2573 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:27:25.409344 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.399100    2573 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:27:25.409344 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.399102    2573 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:27:25.409344 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.399105    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:27:25.409344 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.399108    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:27:25.409344 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.399110    2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:27:25.409344 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:25.399114    2573 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:27:25.409344 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.399182    2573 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 21:27:25.409344 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.401562    2573 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Oct 30 21:27:25.409344 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.402426    2573 server.go:1014] "Starting client certificate rotation"
Oct 30 21:27:25.409344 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.402525    2573 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 21:27:25.409344 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.402561    2573 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 21:27:25.409344 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.404336    2573 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 21:27:25.409344 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.405574    2573 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 21:27:25.417169 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.417145    2573 log.go:25] "Validated CRI v1 runtime API"
Oct 30 21:27:25.423537 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.423516    2573 log.go:25] "Validated CRI v1 image API"
Oct 30 21:27:25.424411 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.424383    2573 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 21:27:25.426750 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.426706    2573 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 e60aa7d7-0091-4b1c-9a9d-101906f592a1:/dev/nvme0n1p4 e9e2424f-0447-4325-ae80-d80a1da18197:/dev/nvme0n1p3]
Oct 30 21:27:25.426842 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.426739    2573 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Oct 30 21:27:25.433640 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.433505    2573 manager.go:217] Machine: {Timestamp:2026-10-30 21:27:25.431577798 +0000 UTC m=+0.312640743 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100918 MemoryCapacity:33164337152 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2a4a41525858d008977e6d6a0fcd46 SystemUUID:ec2a4a41-5258-58d0-0897-7e6d6a0fcd46 BootID:fc49fec5-ea94-4d22-8da1-172f96334599 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:e8:3a:ae:1d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:e8:3a:ae:1d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ca:ce:47:b9:ca:37 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164337152 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 21:27:25.434221 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.434207    2573 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 21:27:25.434322 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.434310    2573 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 21:27:25.437210 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.437175    2573 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 21:27:25.437349 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.437210    2573 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-195.us-east-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 21:27:25.437431 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.437352    2573 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 21:27:25.437431 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.437390    2573 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 21:27:25.437431 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.437409    2573 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 21:27:25.438066 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.438055    2573 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 21:27:25.438662 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.438640    2573 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 21:27:25.439220 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.439209    2573 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 21:27:25.443412 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.443396    2573 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 21:27:25.443534 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.443524    2573 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 21:27:25.443566 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.443559    2573 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 21:27:25.443602 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.443570    2573 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 21:27:25.443602 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.443586    2573 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 21:27:25.443602 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.443597    2573 kubelet.go:405] "Adding apiserver pod source"
Oct 30 21:27:25.443708 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.443606    2573 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 21:27:25.443997 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.443983    2573 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 21:27:25.446051 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.446033    2573 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 21:27:25.452723 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.452704    2573 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 21:27:25.453396 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.453380    2573 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 21:27:25.453431 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.453416    2573 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 21:27:25.453588 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.453577    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 21:27:25.453616 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.453593    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 21:27:25.453616 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.453600    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 21:27:25.453616 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.453606    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 21:27:25.453616 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.453611    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 21:27:25.453616 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.453617    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 21:27:25.454169 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.454160    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 21:27:25.454197 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.454171    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 21:27:25.454197 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.454179    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 21:27:25.454197 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.454185    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 21:27:25.454342 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.454201    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 21:27:25.454342 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.454211    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 21:27:25.454895 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.454884    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 21:27:25.454895 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.454894    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 21:27:25.458766 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.458749    2573 server.go:1263] "Started kubelet"
Oct 30 21:27:25.458914 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.458885    2573 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 21:27:25.458969 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.458873    2573 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 21:27:25.458969 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.458945    2573 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 21:27:25.459547 ip-10-0-129-195 systemd[1]: Started Kubernetes Kubelet.
Oct 30 21:27:25.460804 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.460785    2573 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 21:27:25.462719 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.462237    2573 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-195.us-east-2.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 30 21:27:25.463163 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.463145    2573 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 21:27:25.465185 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:25.465167    2573 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Oct 30 21:27:25.467655 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.467633    2573 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 21:27:25.467738 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.467668    2573 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 21:27:25.468048 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:25.468029    2573 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-195.us-east-2.compute.internal\" not found"
Oct 30 21:27:25.468414 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.468387    2573 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 21:27:25.468414 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.468390    2573 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 21:27:25.468414 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.468419    2573 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 21:27:25.468620 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.468481    2573 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 21:27:25.468620 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.468490    2573 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 21:27:25.469167 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.469147    2573 factory.go:55] Registering systemd factory
Oct 30 21:27:25.469251 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.469195    2573 factory.go:223] Registration of the systemd container factory successfully
Oct 30 21:27:25.469457 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.469443    2573 factory.go:153] Registering CRI-O factory
Oct 30 21:27:25.469542 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.469530    2573 factory.go:223] Registration of the crio container factory successfully
Oct 30 21:27:25.469604 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.469591    2573 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 21:27:25.469710 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.469628    2573 factory.go:103] Registering Raw factory
Oct 30 21:27:25.469710 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.469644    2573 manager.go:1199] Started watching for new ooms in manager
Oct 30 21:27:25.470671 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.470653    2573 manager.go:319] Starting recovery of all containers
Oct 30 21:27:25.470935 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:25.470912    2573 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service"
Oct 30 21:27:25.471070 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:25.471050    2573 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-195.us-east-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node"
Oct 30 21:27:25.471190 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:25.470457    2573 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-195.us-east-2.compute.internal.18e36bd6754c896d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-195.us-east-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-195.us-east-2.compute.internal,},FirstTimestamp:2026-10-30 21:27:25.458712941 +0000 UTC m=+0.339775885,LastTimestamp:2026-10-30 21:27:25.458712941 +0000 UTC m=+0.339775885,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-195.us-east-2.compute.internal,}"
Oct 30 21:27:25.471190 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:25.471171    2573 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-195.us-east-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Oct 30 21:27:25.478309 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:25.478269    2573 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver"
Oct 30 21:27:25.481296 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.481276    2573 manager.go:324] Recovery completed
Oct 30 21:27:25.485859 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.485721    2573 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:27:25.488134 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.488114    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-195.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:27:25.488211 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.488164    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-195.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:27:25.488211 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.488180    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-195.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:27:25.488784 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.488770    2573 cpu_manager.go:226] "Starting" policy="none"
Oct 30 21:27:25.488784 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.488781    2573 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 21:27:25.488885 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.488793    2573 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:27:25.488885 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.488797    2573 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 21:27:25.489474 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.489459    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-195.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:27:25.489513 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.489485    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-195.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:27:25.489513 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.489497    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-195.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:27:25.490392 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:25.490283    2573 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-195.us-east-2.compute.internal.18e36bd6770d7fb3  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-195.us-east-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-195.us-east-2.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-195.us-east-2.compute.internal,},FirstTimestamp:2026-10-30 21:27:25.488136115 +0000 UTC m=+0.369199064,LastTimestamp:2026-10-30 21:27:25.488136115 +0000 UTC m=+0.369199064,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-195.us-east-2.compute.internal,}"
Oct 30 21:27:25.492900 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.492869    2573 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-h2npp"
Oct 30 21:27:25.503963 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.503937    2573 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-h2npp"
Oct 30 21:27:25.506036 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:25.505960    2573 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-195.us-east-2.compute.internal.18e36bd6770e0b11  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-195.us-east-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-129-195.us-east-2.compute.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-129-195.us-east-2.compute.internal,},FirstTimestamp:2026-10-30 21:27:25.488171793 +0000 UTC m=+0.369234742,LastTimestamp:2026-10-30 21:27:25.488171793 +0000 UTC m=+0.369234742,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-195.us-east-2.compute.internal,}"
Oct 30 21:27:25.569149 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:25.569114    2573 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-195.us-east-2.compute.internal\" not found"
Oct 30 21:27:25.631042 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.630969    2573 policy_none.go:50] "Start"
Oct 30 21:27:25.631042 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.630999    2573 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 21:27:25.631042 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.631017    2573 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 21:27:25.635933 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.635917    2573 policy_none.go:44] "Start"
Oct 30 21:27:25.658827 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.655503    2573 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 21:27:25.658827 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.656866    2573 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 21:27:25.658827 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.656887    2573 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 21:27:25.658827 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.656906    2573 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 21:27:25.658827 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:25.656937    2573 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Oct 30 21:27:25.661055 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.661038    2573 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:27:25.668131 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.668100    2573 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 21:27:25.669427 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:25.669406    2573 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-195.us-east-2.compute.internal\" not found"
Oct 30 21:27:25.672341 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.672325    2573 manager.go:341] "Starting Device Plugin manager"
Oct 30 21:27:25.672436 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:25.672356    2573 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 21:27:25.672436 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.672380    2573 server.go:87] "Starting device plugin registration server"
Oct 30 21:27:25.672675 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.672663    2573 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 21:27:25.672709 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.672680    2573 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 21:27:25.672782 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.672766    2573 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 21:27:25.672847 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.672839    2573 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 21:27:25.672847 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.672846    2573 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 21:27:25.673381 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:25.673343    2573 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 21:27:25.673471 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:25.673410    2573 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-195.us-east-2.compute.internal\" not found"
Oct 30 21:27:25.677705 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:25.677687    2573 nodelease.go:50] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-129-195.us-east-2.compute.internal\" not found" node="ip-10-0-129-195.us-east-2.compute.internal"
Oct 30 21:27:25.757378 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.757304    2573 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-129-195.us-east-2.compute.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-195.us-east-2.compute.internal"]
Oct 30 21:27:25.757548 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.757413    2573 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:27:25.758461 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.758445    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-195.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:27:25.758533 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.758475    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-195.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:27:25.758533 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.758489    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-195.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:27:25.760665 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.760653    2573 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:27:25.760809 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.760793    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-195.us-east-2.compute.internal"
Oct 30 21:27:25.760844 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.760828    2573 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:27:25.761458 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.761441    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-195.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:27:25.761526 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.761470    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-195.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:27:25.761526 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.761480    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-195.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:27:25.761526 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.761442    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-195.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:27:25.761626 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.761540    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-195.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:27:25.761626 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.761549    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-195.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:27:25.763528 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.763511    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-195.us-east-2.compute.internal"
Oct 30 21:27:25.763632 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.763535    2573 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:27:25.764266 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.764251    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-195.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:27:25.764349 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.764275    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-195.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:27:25.764349 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.764285    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-195.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:27:25.769661 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.769643    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7efe602e8ba1524abe391aabbb698457-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-195.us-east-2.compute.internal\" (UID: \"7efe602e8ba1524abe391aabbb698457\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-195.us-east-2.compute.internal"
Oct 30 21:27:25.769726 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.769671    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7efe602e8ba1524abe391aabbb698457-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-195.us-east-2.compute.internal\" (UID: \"7efe602e8ba1524abe391aabbb698457\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-195.us-east-2.compute.internal"
Oct 30 21:27:25.769726 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.769690    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7c1602df5a81ac24aa1efba1059dbbe8-config\") pod \"kube-apiserver-proxy-ip-10-0-129-195.us-east-2.compute.internal\" (UID: \"7c1602df5a81ac24aa1efba1059dbbe8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-195.us-east-2.compute.internal"
Oct 30 21:27:25.773133 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.773113    2573 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:27:25.773874 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.773860    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-195.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:27:25.773946 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.773885    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-195.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:27:25.773946 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.773894    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-195.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:27:25.773946 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.773922    2573 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-129-195.us-east-2.compute.internal"
Oct 30 21:27:25.783517 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.783497    2573 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-129-195.us-east-2.compute.internal"
Oct 30 21:27:25.783623 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:25.783523    2573 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-195.us-east-2.compute.internal\": node \"ip-10-0-129-195.us-east-2.compute.internal\" not found"
Oct 30 21:27:25.787991 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:25.787971    2573 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-195.us-east-2.compute.internal\" not found" node="ip-10-0-129-195.us-east-2.compute.internal"
Oct 30 21:27:25.794788 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:25.794763    2573 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-195.us-east-2.compute.internal\" not found" node="ip-10-0-129-195.us-east-2.compute.internal"
Oct 30 21:27:25.821998 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:25.821971    2573 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-195.us-east-2.compute.internal\" not found"
Oct 30 21:27:25.870399 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.870349    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7efe602e8ba1524abe391aabbb698457-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-195.us-east-2.compute.internal\" (UID: \"7efe602e8ba1524abe391aabbb698457\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-195.us-east-2.compute.internal"
Oct 30 21:27:25.870399 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.870402    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7efe602e8ba1524abe391aabbb698457-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-195.us-east-2.compute.internal\" (UID: \"7efe602e8ba1524abe391aabbb698457\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-195.us-east-2.compute.internal"
Oct 30 21:27:25.870604 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.870428    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7c1602df5a81ac24aa1efba1059dbbe8-config\") pod \"kube-apiserver-proxy-ip-10-0-129-195.us-east-2.compute.internal\" (UID: \"7c1602df5a81ac24aa1efba1059dbbe8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-195.us-east-2.compute.internal"
Oct 30 21:27:25.870604 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.870467    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7efe602e8ba1524abe391aabbb698457-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-195.us-east-2.compute.internal\" (UID: \"7efe602e8ba1524abe391aabbb698457\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-195.us-east-2.compute.internal"
Oct 30 21:27:25.870604 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.870477    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7c1602df5a81ac24aa1efba1059dbbe8-config\") pod \"kube-apiserver-proxy-ip-10-0-129-195.us-east-2.compute.internal\" (UID: \"7c1602df5a81ac24aa1efba1059dbbe8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-195.us-east-2.compute.internal"
Oct 30 21:27:25.870604 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:25.870513    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7efe602e8ba1524abe391aabbb698457-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-195.us-east-2.compute.internal\" (UID: \"7efe602e8ba1524abe391aabbb698457\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-195.us-east-2.compute.internal"
Oct 30 21:27:25.922327 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:25.922262    2573 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-195.us-east-2.compute.internal\" not found"
Oct 30 21:27:26.022964 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:26.022929    2573 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-195.us-east-2.compute.internal\" not found"
Oct 30 21:27:26.092647 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:26.092611    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-195.us-east-2.compute.internal"
Oct 30 21:27:26.099314 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:26.099295    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-195.us-east-2.compute.internal"
Oct 30 21:27:26.123734 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:26.123695    2573 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-195.us-east-2.compute.internal\" not found"
Oct 30 21:27:26.227403 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:26.227297    2573 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-195.us-east-2.compute.internal\" not found"
Oct 30 21:27:26.331690 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:26.331658    2573 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-195.us-east-2.compute.internal\" not found"
Oct 30 21:27:26.359651 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:26.359623    2573 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:27:26.403293 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:26.403263    2573 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Oct 30 21:27:26.403816 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:26.403432    2573 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 21:27:26.403816 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:26.403460    2573 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 21:27:26.432786 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:26.432746    2573 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-195.us-east-2.compute.internal\" not found"
Oct 30 21:27:26.510087 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:26.509951    2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 21:22:25 +0000 UTC" deadline="2028-08-12 14:50:02.29378733 +0000 UTC"
Oct 30 21:27:26.510087 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:26.510026    2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15641h22m35.783764123s"
Oct 30 21:27:26.533480 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:26.533436    2573 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-195.us-east-2.compute.internal\" not found"
Oct 30 21:27:26.634349 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:26.634298    2573 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-195.us-east-2.compute.internal\" not found"
Oct 30 21:27:26.680991 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:26.680957    2573 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:27:26.682354 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:26.682332    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-195.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:27:26.682460 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:26.682373    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-195.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:27:26.682460 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:26.682436    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-195.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:27:26.735548 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:26.735505    2573 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-195.us-east-2.compute.internal\" not found"
Oct 30 21:27:26.763290 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:26.763138    2573 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:27:26.768188 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:26.768153    2573 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-195.us-east-2.compute.internal"
Oct 30 21:27:26.780700 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:26.780670    2573 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:27:26.781723 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:26.781701    2573 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-195.us-east-2.compute.internal"
Oct 30 21:27:26.795692 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:26.795670    2573 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:27:26.866618 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:26.866575    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c1602df5a81ac24aa1efba1059dbbe8.slice/crio-674d20fe97c246ae471b27f055ab576608b0a4cad5fabb0dca5b5ee7a31bfc48 WatchSource:0}: Error finding container 674d20fe97c246ae471b27f055ab576608b0a4cad5fabb0dca5b5ee7a31bfc48: Status 404 returned error can't find the container with id 674d20fe97c246ae471b27f055ab576608b0a4cad5fabb0dca5b5ee7a31bfc48
Oct 30 21:27:26.868425 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:26.868401    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7efe602e8ba1524abe391aabbb698457.slice/crio-b9c929b2079c2178d1e0dbb3a0cd5c7965226441feeb7576f7b91f47724b22ff WatchSource:0}: Error finding container b9c929b2079c2178d1e0dbb3a0cd5c7965226441feeb7576f7b91f47724b22ff: Status 404 returned error can't find the container with id b9c929b2079c2178d1e0dbb3a0cd5c7965226441feeb7576f7b91f47724b22ff
Oct 30 21:27:26.871656 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:26.871637    2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 21:27:27.021387 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.021278    2573 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:27:27.445695 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.445608    2573 apiserver.go:52] "Watching apiserver"
Oct 30 21:27:27.455305 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.454482    2573 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 21:27:27.455305 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.454882    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-csbxh","kube-system/konnectivity-agent-l8g24","openshift-multus/multus-additional-cni-plugins-l67jk","openshift-multus/multus-d9b8n","openshift-network-diagnostics/network-check-target-4cvlt","openshift-network-operator/iptables-alerter-bn9ng","kube-system/kube-apiserver-proxy-ip-10-0-129-195.us-east-2.compute.internal","openshift-cluster-node-tuning-operator/tuned-7ntsz","openshift-image-registry/node-ca-jgns8","openshift-multus/network-metrics-daemon-vwxrd","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qcj76","openshift-dns/node-resolver-97tsr","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-195.us-east-2.compute.internal","openshift-ovn-kubernetes/ovnkube-node-gl47s"]
Oct 30 21:27:27.457709 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.457684    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vwxrd"
Oct 30 21:27:27.457862 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:27.457794    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vwxrd" podUID="352b5b4c-c640-4300-abf3-aa70b08cbb4b"
Oct 30 21:27:27.460182 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.460157    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qcj76"
Oct 30 21:27:27.465525 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.465498    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 21:27:27.465660 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.465591    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-l67jk"
Oct 30 21:27:27.465724 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.465668    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 21:27:27.465782 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.465753    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 21:27:27.465944 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.465928    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-g2c5m\""
Oct 30 21:27:27.466444 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.466418    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-d9b8n"
Oct 30 21:27:27.467920 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.467897    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4cvlt"
Oct 30 21:27:27.468015 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:27.467974    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4cvlt" podUID="3913f7d4-86cb-4e86-af5a-debd8b859252"
Oct 30 21:27:27.468960 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.468942    2573 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Oct 30 21:27:27.469572 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.469465    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 21:27:27.469572 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.469548    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 21:27:27.469727 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.469597    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 21:27:27.470558 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.470529    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bn9ng"
Oct 30 21:27:27.471991 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.471970    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 21:27:27.472089 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.472032    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-z8x7v\""
Oct 30 21:27:27.472089 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.472057    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 21:27:27.472227 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.472199    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 21:27:27.472585 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.472569    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-flwfj\""
Oct 30 21:27:27.474757 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.474708    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-jgns8"
Oct 30 21:27:27.477246 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.477224    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:27:27.479708 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.479684    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-l8g24"
Oct 30 21:27:27.480149 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.480121    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/84a732a8-045d-4a8b-bb54-859c8d46f08d-host-slash\") pod \"iptables-alerter-bn9ng\" (UID: \"84a732a8-045d-4a8b-bb54-859c8d46f08d\") " pod="openshift-network-operator/iptables-alerter-bn9ng"
Oct 30 21:27:27.480249 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.480166    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cf90e1fc-bac9-4c33-8dc4-dbca18974399-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qcj76\" (UID: \"cf90e1fc-bac9-4c33-8dc4-dbca18974399\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qcj76"
Oct 30 21:27:27.480249 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.480198    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f6750ff3-d2db-42b7-9c85-7896c9ba67c9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-l67jk\" (UID: \"f6750ff3-d2db-42b7-9c85-7896c9ba67c9\") " pod="openshift-multus/multus-additional-cni-plugins-l67jk"
Oct 30 21:27:27.480249 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.480226    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5e93253e-a238-4426-a584-b9de8e931cf7-cni-binary-copy\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:27:27.480471 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.480295    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-host-var-lib-cni-bin\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:27:27.480471 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.480360    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ms8rv\" (UniqueName: \"kubernetes.io/projected/3913f7d4-86cb-4e86-af5a-debd8b859252-kube-api-access-ms8rv\") pod \"network-check-target-4cvlt\" (UID: \"3913f7d4-86cb-4e86-af5a-debd8b859252\") " pod="openshift-network-diagnostics/network-check-target-4cvlt"
Oct 30 21:27:27.480471 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.480444    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-system-cni-dir\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:27:27.480654 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.480478    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-multus-cni-dir\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:27:27.480654 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.480508    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-etc-kubernetes\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:27:27.480654 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.480547    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f6750ff3-d2db-42b7-9c85-7896c9ba67c9-cni-binary-copy\") pod \"multus-additional-cni-plugins-l67jk\" (UID: \"f6750ff3-d2db-42b7-9c85-7896c9ba67c9\") " pod="openshift-multus/multus-additional-cni-plugins-l67jk"
Oct 30 21:27:27.480654 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.480587    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f6750ff3-d2db-42b7-9c85-7896c9ba67c9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-l67jk\" (UID: \"f6750ff3-d2db-42b7-9c85-7896c9ba67c9\") " pod="openshift-multus/multus-additional-cni-plugins-l67jk"
Oct 30 21:27:27.480654 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.480621    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-multus-socket-dir-parent\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:27:27.481009 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.480661    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-host-var-lib-kubelet\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:27:27.481009 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.480694    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-host-run-multus-certs\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:27:27.481009 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.480719    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/352b5b4c-c640-4300-abf3-aa70b08cbb4b-metrics-certs\") pod \"network-metrics-daemon-vwxrd\" (UID: \"352b5b4c-c640-4300-abf3-aa70b08cbb4b\") " pod="openshift-multus/network-metrics-daemon-vwxrd"
Oct 30 21:27:27.481009 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.480769    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96bzs\" (UniqueName: \"kubernetes.io/projected/352b5b4c-c640-4300-abf3-aa70b08cbb4b-kube-api-access-96bzs\") pod \"network-metrics-daemon-vwxrd\" (UID: \"352b5b4c-c640-4300-abf3-aa70b08cbb4b\") " pod="openshift-multus/network-metrics-daemon-vwxrd"
Oct 30 21:27:27.481009 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.480797    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5tf5\" (UniqueName: \"kubernetes.io/projected/84a732a8-045d-4a8b-bb54-859c8d46f08d-kube-api-access-w5tf5\") pod \"iptables-alerter-bn9ng\" (UID: \"84a732a8-045d-4a8b-bb54-859c8d46f08d\") " pod="openshift-network-operator/iptables-alerter-bn9ng"
Oct 30 21:27:27.481009 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.480822    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cf90e1fc-bac9-4c33-8dc4-dbca18974399-etc-selinux\") pod \"aws-ebs-csi-driver-node-qcj76\" (UID: \"cf90e1fc-bac9-4c33-8dc4-dbca18974399\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qcj76"
Oct 30 21:27:27.481009 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.480871    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ct4zm\" (UniqueName: \"kubernetes.io/projected/cf90e1fc-bac9-4c33-8dc4-dbca18974399-kube-api-access-ct4zm\") pod \"aws-ebs-csi-driver-node-qcj76\" (UID: \"cf90e1fc-bac9-4c33-8dc4-dbca18974399\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qcj76"
Oct 30 21:27:27.481009 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.480898    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f6750ff3-d2db-42b7-9c85-7896c9ba67c9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-l67jk\" (UID: \"f6750ff3-d2db-42b7-9c85-7896c9ba67c9\") " pod="openshift-multus/multus-additional-cni-plugins-l67jk"
Oct 30 21:27:27.481009 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.480926    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhvst\" (UniqueName: \"kubernetes.io/projected/f6750ff3-d2db-42b7-9c85-7896c9ba67c9-kube-api-access-qhvst\") pod \"multus-additional-cni-plugins-l67jk\" (UID: \"f6750ff3-d2db-42b7-9c85-7896c9ba67c9\") " pod="openshift-multus/multus-additional-cni-plugins-l67jk"
Oct 30 21:27:27.481009 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.480950    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5e93253e-a238-4426-a584-b9de8e931cf7-multus-daemon-config\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:27:27.481009 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.480976    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2trdz\" (UniqueName: \"kubernetes.io/projected/5e93253e-a238-4426-a584-b9de8e931cf7-kube-api-access-2trdz\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:27:27.481702 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.481032    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/84a732a8-045d-4a8b-bb54-859c8d46f08d-iptables-alerter-script\") pod \"iptables-alerter-bn9ng\" (UID: \"84a732a8-045d-4a8b-bb54-859c8d46f08d\") " pod="openshift-network-operator/iptables-alerter-bn9ng"
Oct 30 21:27:27.481702 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.481084    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cf90e1fc-bac9-4c33-8dc4-dbca18974399-socket-dir\") pod \"aws-ebs-csi-driver-node-qcj76\" (UID: \"cf90e1fc-bac9-4c33-8dc4-dbca18974399\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qcj76"
Oct 30 21:27:27.481702 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.481117    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cf90e1fc-bac9-4c33-8dc4-dbca18974399-registration-dir\") pod \"aws-ebs-csi-driver-node-qcj76\" (UID: \"cf90e1fc-bac9-4c33-8dc4-dbca18974399\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qcj76"
Oct 30 21:27:27.481702 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.481148    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cf90e1fc-bac9-4c33-8dc4-dbca18974399-sys-fs\") pod \"aws-ebs-csi-driver-node-qcj76\" (UID: \"cf90e1fc-bac9-4c33-8dc4-dbca18974399\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qcj76"
Oct 30 21:27:27.481702 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.481183    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-host-run-k8s-cni-cncf-io\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:27:27.481702 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.481246    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-multus-conf-dir\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:27:27.481702 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.481290    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cf90e1fc-bac9-4c33-8dc4-dbca18974399-device-dir\") pod \"aws-ebs-csi-driver-node-qcj76\" (UID: \"cf90e1fc-bac9-4c33-8dc4-dbca18974399\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qcj76"
Oct 30 21:27:27.481702 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.481337    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f6750ff3-d2db-42b7-9c85-7896c9ba67c9-system-cni-dir\") pod \"multus-additional-cni-plugins-l67jk\" (UID: \"f6750ff3-d2db-42b7-9c85-7896c9ba67c9\") " pod="openshift-multus/multus-additional-cni-plugins-l67jk"
Oct 30 21:27:27.481702 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.481366    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-cnibin\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:27:27.481702 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.481461    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-host-run-netns\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:27:27.481702 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.481565    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-host-var-lib-cni-multus\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:27:27.481702 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.481592    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-hostroot\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:27:27.481702 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.481637    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f6750ff3-d2db-42b7-9c85-7896c9ba67c9-cnibin\") pod \"multus-additional-cni-plugins-l67jk\" (UID: \"f6750ff3-d2db-42b7-9c85-7896c9ba67c9\") " pod="openshift-multus/multus-additional-cni-plugins-l67jk"
Oct 30 21:27:27.482934 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.482916    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-csbxh"
Oct 30 21:27:27.483067 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:27.482984    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-csbxh" podUID="5f171661-33a2-4631-81c7-bc10a5db791f"
Oct 30 21:27:27.485961 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.485936    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:27:27.488564 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.488514    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-97tsr"
Oct 30 21:27:27.489107 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.489086    2573 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 21:27:27.517167 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.517140    2573 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-v2xzv"
Oct 30 21:27:27.526174 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.526143    2573 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-v2xzv"
Oct 30 21:27:27.569709 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.569676    2573 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 21:27:27.578522 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.578490    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 21:27:27.578667 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.578586    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 21:27:27.578667 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.578637    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 21:27:27.578789 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.578711    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-d996q\""
Oct 30 21:27:27.578789 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.578760    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 21:27:27.578909 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.578892    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 21:27:27.578971 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.578961    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-pqqf8\""
Oct 30 21:27:27.579021 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.578998    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 21:27:27.579088 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.579065    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 21:27:27.579141 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.579099    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:27:27.579141 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.579131    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 21:27:27.579299 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.579196    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-ct8zc\""
Oct 30 21:27:27.579299 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.579248    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-99k6q\""
Oct 30 21:27:27.579299 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.579285    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 21:27:27.579479 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.579309    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 21:27:27.579479 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.579356    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 21:27:27.579479 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.579390    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 21:27:27.579479 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.579455    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:27:27.579681 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.579496    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 21:27:27.579681 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.579553    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-wxqgz\""
Oct 30 21:27:27.579781 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.579690    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 21:27:27.579781 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.579733    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-2pcx6\""
Oct 30 21:27:27.580337 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.580317    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 21:27:27.580415 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.580381    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 21:27:27.582733 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.582708    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-host-run-k8s-cni-cncf-io\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:27:27.582838 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.582754    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f171661-33a2-4631-81c7-bc10a5db791f-original-pull-secret\") pod \"global-pull-secret-syncer-csbxh\" (UID: \"5f171661-33a2-4631-81c7-bc10a5db791f\") " pod="kube-system/global-pull-secret-syncer-csbxh"
Oct 30 21:27:27.582897 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.582849    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-host\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:27:27.582967 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.582912    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-host-cni-netd\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:27:27.582967 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.582944    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-cnibin\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:27:27.583076 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.582963    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-host-run-k8s-cni-cncf-io\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:27:27.583076 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.582970    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-host-run-netns\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:27:27.583076 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.583013    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-hostroot\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:27:27.583076 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.583022    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-host-run-netns\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:27:27.583076 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.583043    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f6750ff3-d2db-42b7-9c85-7896c9ba67c9-cnibin\") pod \"multus-additional-cni-plugins-l67jk\" (UID: \"f6750ff3-d2db-42b7-9c85-7896c9ba67c9\") " pod="openshift-multus/multus-additional-cni-plugins-l67jk"
Oct 30 21:27:27.583076 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.583065    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5f171661-33a2-4631-81c7-bc10a5db791f-dbus\") pod \"global-pull-secret-syncer-csbxh\" (UID: \"5f171661-33a2-4631-81c7-bc10a5db791f\") " pod="kube-system/global-pull-secret-syncer-csbxh"
Oct 30 21:27:27.583379 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.583130    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-cnibin\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:27:27.583379 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.583165    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f6750ff3-d2db-42b7-9c85-7896c9ba67c9-cnibin\") pod \"multus-additional-cni-plugins-l67jk\" (UID: \"f6750ff3-d2db-42b7-9c85-7896c9ba67c9\") " pod="openshift-multus/multus-additional-cni-plugins-l67jk"
Oct 30 21:27:27.583379 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.583197    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-hostroot\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:27:27.583379 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.583228    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-etc-sysconfig\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:27:27.583379 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.583259    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-log-socket\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:27:27.583379 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.583286    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/84a732a8-045d-4a8b-bb54-859c8d46f08d-host-slash\") pod \"iptables-alerter-bn9ng\" (UID: \"84a732a8-045d-4a8b-bb54-859c8d46f08d\") " pod="openshift-network-operator/iptables-alerter-bn9ng"
Oct 30 21:27:27.583379 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.583328    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cf90e1fc-bac9-4c33-8dc4-dbca18974399-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qcj76\" (UID: \"cf90e1fc-bac9-4c33-8dc4-dbca18974399\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qcj76"
Oct 30 21:27:27.583379 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.583356    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-host-run-ovn-kubernetes\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:27:27.583820 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.583393    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5d90d533-ebca-46db-ab59-733837db23e6-ovnkube-config\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:27:27.583820 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.583436    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5d90d533-ebca-46db-ab59-733837db23e6-env-overrides\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:27:27.583820 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.583496    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rf7bl\" (UniqueName: \"kubernetes.io/projected/5d90d533-ebca-46db-ab59-733837db23e6-kube-api-access-rf7bl\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:27:27.583820 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.583525    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ms8rv\" (UniqueName: \"kubernetes.io/projected/3913f7d4-86cb-4e86-af5a-debd8b859252-kube-api-access-ms8rv\") pod \"network-check-target-4cvlt\" (UID: \"3913f7d4-86cb-4e86-af5a-debd8b859252\") " pod="openshift-network-diagnostics/network-check-target-4cvlt"
Oct 30 21:27:27.583820 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.583553    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dv6wd\" (UniqueName: \"kubernetes.io/projected/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-kube-api-access-dv6wd\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:27:27.583820 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.583579    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/baa8d0c0-d477-4f30-a0f6-375fc84cc448-hosts-file\") pod \"node-resolver-97tsr\" (UID: \"baa8d0c0-d477-4f30-a0f6-375fc84cc448\") " pod="openshift-dns/node-resolver-97tsr"
Oct 30 21:27:27.583820 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.583603    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klwsc\" (UniqueName: \"kubernetes.io/projected/baa8d0c0-d477-4f30-a0f6-375fc84cc448-kube-api-access-klwsc\") pod \"node-resolver-97tsr\" (UID: \"baa8d0c0-d477-4f30-a0f6-375fc84cc448\") " pod="openshift-dns/node-resolver-97tsr"
Oct 30 21:27:27.583820 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.583630    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ee6de179-7e35-46a5-86e1-c7c31428c149-konnectivity-ca\") pod \"konnectivity-agent-l8g24\" (UID: \"ee6de179-7e35-46a5-86e1-c7c31428c149\") " pod="kube-system/konnectivity-agent-l8g24"
Oct 30 21:27:27.583820 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.583653    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/48efa9dc-3c20-4f81-86f3-c87ad697772c-host\") pod \"node-ca-jgns8\" (UID: \"48efa9dc-3c20-4f81-86f3-c87ad697772c\") " pod="openshift-image-registry/node-ca-jgns8"
Oct 30 21:27:27.583820 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.583699    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/84a732a8-045d-4a8b-bb54-859c8d46f08d-host-slash\") pod \"iptables-alerter-bn9ng\" (UID: \"84a732a8-045d-4a8b-bb54-859c8d46f08d\") " pod="openshift-network-operator/iptables-alerter-bn9ng"
Oct 30 21:27:27.583820 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.583747    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cf90e1fc-bac9-4c33-8dc4-dbca18974399-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qcj76\" (UID: \"cf90e1fc-bac9-4c33-8dc4-dbca18974399\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qcj76"
Oct 30 21:27:27.584351 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.584002    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-etc-kubernetes\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:27:27.584351 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.584032    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f6750ff3-d2db-42b7-9c85-7896c9ba67c9-cni-binary-copy\") pod \"multus-additional-cni-plugins-l67jk\" (UID: \"f6750ff3-d2db-42b7-9c85-7896c9ba67c9\") " pod="openshift-multus/multus-additional-cni-plugins-l67jk"
Oct 30 21:27:27.584351 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.584051    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f6750ff3-d2db-42b7-9c85-7896c9ba67c9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-l67jk\" (UID: \"f6750ff3-d2db-42b7-9c85-7896c9ba67c9\") " pod="openshift-multus/multus-additional-cni-plugins-l67jk"
Oct 30 21:27:27.584351 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.584086    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5f171661-33a2-4631-81c7-bc10a5db791f-kubelet-config\") pod \"global-pull-secret-syncer-csbxh\" (UID: \"5f171661-33a2-4631-81c7-bc10a5db791f\") " pod="kube-system/global-pull-secret-syncer-csbxh"
Oct 30 21:27:27.584351 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.584104    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-systemd-units\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:27:27.584351 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.584129    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-var-lib-openvswitch\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:27:27.584351 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.584153    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-host-run-multus-certs\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:27:27.584351 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.584190    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/352b5b4c-c640-4300-abf3-aa70b08cbb4b-metrics-certs\") pod \"network-metrics-daemon-vwxrd\" (UID: \"352b5b4c-c640-4300-abf3-aa70b08cbb4b\") " pod="openshift-multus/network-metrics-daemon-vwxrd"
Oct 30 21:27:27.584351 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.584214    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-etc-openvswitch\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:27:27.584351 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.584236    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5tf5\" (UniqueName: \"kubernetes.io/projected/84a732a8-045d-4a8b-bb54-859c8d46f08d-kube-api-access-w5tf5\") pod \"iptables-alerter-bn9ng\" (UID: \"84a732a8-045d-4a8b-bb54-859c8d46f08d\") " pod="openshift-network-operator/iptables-alerter-bn9ng"
Oct 30 21:27:27.584351 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.584257    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-etc-kubernetes\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:27:27.584351 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.584283    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5e93253e-a238-4426-a584-b9de8e931cf7-multus-daemon-config\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:27:27.584351 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.584300    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2trdz\" (UniqueName: \"kubernetes.io/projected/5e93253e-a238-4426-a584-b9de8e931cf7-kube-api-access-2trdz\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:27:27.584351 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.584325    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cf90e1fc-bac9-4c33-8dc4-dbca18974399-socket-dir\") pod \"aws-ebs-csi-driver-node-qcj76\" (UID: \"cf90e1fc-bac9-4c33-8dc4-dbca18974399\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qcj76"
Oct 30 21:27:27.584351 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.584347    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-lib-modules\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:27:27.585083 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.584412    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-var-lib-kubelet\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:27:27.585083 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.584453    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-host-kubelet\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:27:27.585083 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.584467    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-host-run-multus-certs\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:27:27.585083 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.584482    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:27:27.585083 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:27.584538    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:27:27.585083 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:27.584614    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/352b5b4c-c640-4300-abf3-aa70b08cbb4b-metrics-certs podName:352b5b4c-c640-4300-abf3-aa70b08cbb4b nodeName:}" failed. No retries permitted until 2026-10-30 21:27:28.084592461 +0000 UTC m=+2.965655415 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/352b5b4c-c640-4300-abf3-aa70b08cbb4b-metrics-certs") pod "network-metrics-daemon-vwxrd" (UID: "352b5b4c-c640-4300-abf3-aa70b08cbb4b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:27:27.585083 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.584611    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cf90e1fc-bac9-4c33-8dc4-dbca18974399-socket-dir\") pod \"aws-ebs-csi-driver-node-qcj76\" (UID: \"cf90e1fc-bac9-4c33-8dc4-dbca18974399\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qcj76"
Oct 30 21:27:27.585083 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.584642    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-etc-kubernetes\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:27:27.585083 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.584785    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-multus-conf-dir\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:27:27.585083 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.584845    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-multus-conf-dir\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:27:27.585602 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.584918    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cf90e1fc-bac9-4c33-8dc4-dbca18974399-device-dir\") pod \"aws-ebs-csi-driver-node-qcj76\" (UID: \"cf90e1fc-bac9-4c33-8dc4-dbca18974399\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qcj76"
Oct 30 21:27:27.585602 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.585364    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f6750ff3-d2db-42b7-9c85-7896c9ba67c9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-l67jk\" (UID: \"f6750ff3-d2db-42b7-9c85-7896c9ba67c9\") " pod="openshift-multus/multus-additional-cni-plugins-l67jk"
Oct 30 21:27:27.585602 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.585384    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cf90e1fc-bac9-4c33-8dc4-dbca18974399-device-dir\") pod \"aws-ebs-csi-driver-node-qcj76\" (UID: \"cf90e1fc-bac9-4c33-8dc4-dbca18974399\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qcj76"
Oct 30 21:27:27.585602 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.585428    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f6750ff3-d2db-42b7-9c85-7896c9ba67c9-system-cni-dir\") pod \"multus-additional-cni-plugins-l67jk\" (UID: \"f6750ff3-d2db-42b7-9c85-7896c9ba67c9\") " pod="openshift-multus/multus-additional-cni-plugins-l67jk"
Oct 30 21:27:27.585809 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.585608    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f6750ff3-d2db-42b7-9c85-7896c9ba67c9-cni-binary-copy\") pod \"multus-additional-cni-plugins-l67jk\" (UID: \"f6750ff3-d2db-42b7-9c85-7896c9ba67c9\") " pod="openshift-multus/multus-additional-cni-plugins-l67jk"
Oct 30 21:27:27.585809 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.585603    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5e93253e-a238-4426-a584-b9de8e931cf7-multus-daemon-config\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:27:27.585809 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.585669    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f6750ff3-d2db-42b7-9c85-7896c9ba67c9-system-cni-dir\") pod \"multus-additional-cni-plugins-l67jk\" (UID: \"f6750ff3-d2db-42b7-9c85-7896c9ba67c9\") " pod="openshift-multus/multus-additional-cni-plugins-l67jk"
Oct 30 21:27:27.585809 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.585670    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-etc-tuned\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:27:27.585809 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.585704    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-host-var-lib-cni-multus\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:27:27.585809 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.585758    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-etc-modprobe-d\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:27:27.585809 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.585789    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-run-systemd\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:27:27.586152 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.585853    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5d90d533-ebca-46db-ab59-733837db23e6-ovn-node-metrics-cert\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:27:27.586152 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.585892    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f6750ff3-d2db-42b7-9c85-7896c9ba67c9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-l67jk\" (UID: \"f6750ff3-d2db-42b7-9c85-7896c9ba67c9\") " pod="openshift-multus/multus-additional-cni-plugins-l67jk"
Oct 30 21:27:27.586152 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.585921    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5e93253e-a238-4426-a584-b9de8e931cf7-cni-binary-copy\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:27:27.586152 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.586017    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-host-var-lib-cni-bin\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:27:27.586152 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.586052    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-run\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:27:27.586152 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.586088    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ee6de179-7e35-46a5-86e1-c7c31428c149-agent-certs\") pod \"konnectivity-agent-l8g24\" (UID: \"ee6de179-7e35-46a5-86e1-c7c31428c149\") " pod="kube-system/konnectivity-agent-l8g24"
Oct 30 21:27:27.586152 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.586135    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-host-run-netns\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:27:27.586523 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.586160    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-host-cni-bin\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:27:27.586523 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.586193    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-system-cni-dir\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:27:27.586523 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.586227    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-multus-cni-dir\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:27:27.586675 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.586624    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-etc-systemd\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:27:27.586675 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.586652    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-tmp\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:27:27.586774 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.586683    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-node-log\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:27:27.586774 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.586718    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-multus-socket-dir-parent\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:27:27.586774 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.586750    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-host-var-lib-kubelet\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:27:27.586930 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.586786    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96bzs\" (UniqueName: \"kubernetes.io/projected/352b5b4c-c640-4300-abf3-aa70b08cbb4b-kube-api-access-96bzs\") pod \"network-metrics-daemon-vwxrd\" (UID: \"352b5b4c-c640-4300-abf3-aa70b08cbb4b\") " pod="openshift-multus/network-metrics-daemon-vwxrd"
Oct 30 21:27:27.586930 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.586819    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/baa8d0c0-d477-4f30-a0f6-375fc84cc448-tmp-dir\") pod \"node-resolver-97tsr\" (UID: \"baa8d0c0-d477-4f30-a0f6-375fc84cc448\") " pod="openshift-dns/node-resolver-97tsr"
Oct 30 21:27:27.586930 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.586851    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtmzr\" (UniqueName: \"kubernetes.io/projected/48efa9dc-3c20-4f81-86f3-c87ad697772c-kube-api-access-vtmzr\") pod \"node-ca-jgns8\" (UID: \"48efa9dc-3c20-4f81-86f3-c87ad697772c\") " pod="openshift-image-registry/node-ca-jgns8"
Oct 30 21:27:27.586930 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.586878    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-host-slash\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:27:27.586930 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.586914    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-run-ovn\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:27:27.587181 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.586946    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-sys\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:27:27.587181 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.586980    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cf90e1fc-bac9-4c33-8dc4-dbca18974399-etc-selinux\") pod \"aws-ebs-csi-driver-node-qcj76\" (UID: \"cf90e1fc-bac9-4c33-8dc4-dbca18974399\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qcj76"
Oct 30 21:27:27.587181 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.587009    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ct4zm\" (UniqueName: \"kubernetes.io/projected/cf90e1fc-bac9-4c33-8dc4-dbca18974399-kube-api-access-ct4zm\") pod \"aws-ebs-csi-driver-node-qcj76\" (UID: \"cf90e1fc-bac9-4c33-8dc4-dbca18974399\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qcj76"
Oct 30 21:27:27.587181 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.587042    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f6750ff3-d2db-42b7-9c85-7896c9ba67c9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-l67jk\" (UID: \"f6750ff3-d2db-42b7-9c85-7896c9ba67c9\") " pod="openshift-multus/multus-additional-cni-plugins-l67jk"
Oct 30 21:27:27.587375 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.587346    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhvst\" (UniqueName: \"kubernetes.io/projected/f6750ff3-d2db-42b7-9c85-7896c9ba67c9-kube-api-access-qhvst\") pod \"multus-additional-cni-plugins-l67jk\" (UID: \"f6750ff3-d2db-42b7-9c85-7896c9ba67c9\") " pod="openshift-multus/multus-additional-cni-plugins-l67jk"
Oct 30 21:27:27.587466 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.587384    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-etc-sysctl-d\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:27:27.587466 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.587417    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-etc-sysctl-conf\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:27:27.587575 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.587466    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-run-openvswitch\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:27:27.587575 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.587501    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/84a732a8-045d-4a8b-bb54-859c8d46f08d-iptables-alerter-script\") pod \"iptables-alerter-bn9ng\" (UID: \"84a732a8-045d-4a8b-bb54-859c8d46f08d\") " pod="openshift-network-operator/iptables-alerter-bn9ng"
Oct 30 21:27:27.587575 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.587534    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cf90e1fc-bac9-4c33-8dc4-dbca18974399-registration-dir\") pod \"aws-ebs-csi-driver-node-qcj76\" (UID: \"cf90e1fc-bac9-4c33-8dc4-dbca18974399\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qcj76"
Oct 30 21:27:27.587575 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.587561    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cf90e1fc-bac9-4c33-8dc4-dbca18974399-sys-fs\") pod \"aws-ebs-csi-driver-node-qcj76\" (UID: \"cf90e1fc-bac9-4c33-8dc4-dbca18974399\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qcj76"
Oct 30 21:27:27.587780 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.587593    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/48efa9dc-3c20-4f81-86f3-c87ad697772c-serviceca\") pod \"node-ca-jgns8\" (UID: \"48efa9dc-3c20-4f81-86f3-c87ad697772c\") " pod="openshift-image-registry/node-ca-jgns8"
Oct 30 21:27:27.587780 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.587627    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5d90d533-ebca-46db-ab59-733837db23e6-ovnkube-script-lib\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:27:27.587900 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.587880    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f6750ff3-d2db-42b7-9c85-7896c9ba67c9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-l67jk\" (UID: \"f6750ff3-d2db-42b7-9c85-7896c9ba67c9\") " pod="openshift-multus/multus-additional-cni-plugins-l67jk"
Oct 30 21:27:27.589889 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.588062    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-host-var-lib-kubelet\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:27:27.589889 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.588220    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-multus-socket-dir-parent\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:27:27.589889 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.588623    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cf90e1fc-bac9-4c33-8dc4-dbca18974399-etc-selinux\") pod \"aws-ebs-csi-driver-node-qcj76\" (UID: \"cf90e1fc-bac9-4c33-8dc4-dbca18974399\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qcj76"
Oct 30 21:27:27.589889 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.588633    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f6750ff3-d2db-42b7-9c85-7896c9ba67c9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-l67jk\" (UID: \"f6750ff3-d2db-42b7-9c85-7896c9ba67c9\") " pod="openshift-multus/multus-additional-cni-plugins-l67jk"
Oct 30 21:27:27.589889 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.588731    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-system-cni-dir\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:27:27.589889 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.588782    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-host-var-lib-cni-bin\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:27:27.589889 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.588790    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5e93253e-a238-4426-a584-b9de8e931cf7-cni-binary-copy\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:27:27.589889 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.588834    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-host-var-lib-cni-multus\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:27:27.589889 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.588909    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cf90e1fc-bac9-4c33-8dc4-dbca18974399-sys-fs\") pod \"aws-ebs-csi-driver-node-qcj76\" (UID: \"cf90e1fc-bac9-4c33-8dc4-dbca18974399\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qcj76"
Oct 30 21:27:27.589889 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.588917    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cf90e1fc-bac9-4c33-8dc4-dbca18974399-registration-dir\") pod \"aws-ebs-csi-driver-node-qcj76\" (UID: \"cf90e1fc-bac9-4c33-8dc4-dbca18974399\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qcj76"
Oct 30 21:27:27.589889 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.588997    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-multus-cni-dir\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:27:27.589889 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.589459    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/84a732a8-045d-4a8b-bb54-859c8d46f08d-iptables-alerter-script\") pod \"iptables-alerter-bn9ng\" (UID: \"84a732a8-045d-4a8b-bb54-859c8d46f08d\") " pod="openshift-network-operator/iptables-alerter-bn9ng"
Oct 30 21:27:27.603131 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.603094    2573 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 21:27:27.607949 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.607918    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5tf5\" (UniqueName: \"kubernetes.io/projected/84a732a8-045d-4a8b-bb54-859c8d46f08d-kube-api-access-w5tf5\") pod \"iptables-alerter-bn9ng\" (UID: \"84a732a8-045d-4a8b-bb54-859c8d46f08d\") " pod="openshift-network-operator/iptables-alerter-bn9ng"
Oct 30 21:27:27.617364 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:27.617329    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:27:27.617364 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:27.617365    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:27:27.617599 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:27.617382    2573 projected.go:196] Error preparing data for projected volume kube-api-access-ms8rv for pod openshift-network-diagnostics/network-check-target-4cvlt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:27:27.617599 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:27.617484    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3913f7d4-86cb-4e86-af5a-debd8b859252-kube-api-access-ms8rv podName:3913f7d4-86cb-4e86-af5a-debd8b859252 nodeName:}" failed. No retries permitted until 2026-10-30 21:27:28.117462831 +0000 UTC m=+2.998525787 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-ms8rv" (UniqueName: "kubernetes.io/projected/3913f7d4-86cb-4e86-af5a-debd8b859252-kube-api-access-ms8rv") pod "network-check-target-4cvlt" (UID: "3913f7d4-86cb-4e86-af5a-debd8b859252") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:27:27.619884 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.619844    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2trdz\" (UniqueName: \"kubernetes.io/projected/5e93253e-a238-4426-a584-b9de8e931cf7-kube-api-access-2trdz\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:27:27.620705 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.620649    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-96bzs\" (UniqueName: \"kubernetes.io/projected/352b5b4c-c640-4300-abf3-aa70b08cbb4b-kube-api-access-96bzs\") pod \"network-metrics-daemon-vwxrd\" (UID: \"352b5b4c-c640-4300-abf3-aa70b08cbb4b\") " pod="openshift-multus/network-metrics-daemon-vwxrd"
Oct 30 21:27:27.622019 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.621991    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ct4zm\" (UniqueName: \"kubernetes.io/projected/cf90e1fc-bac9-4c33-8dc4-dbca18974399-kube-api-access-ct4zm\") pod \"aws-ebs-csi-driver-node-qcj76\" (UID: \"cf90e1fc-bac9-4c33-8dc4-dbca18974399\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qcj76"
Oct 30 21:27:27.624908 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.624859    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhvst\" (UniqueName: \"kubernetes.io/projected/f6750ff3-d2db-42b7-9c85-7896c9ba67c9-kube-api-access-qhvst\") pod \"multus-additional-cni-plugins-l67jk\" (UID: \"f6750ff3-d2db-42b7-9c85-7896c9ba67c9\") " pod="openshift-multus/multus-additional-cni-plugins-l67jk"
Oct 30 21:27:27.662612 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.662553    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-195.us-east-2.compute.internal" event={"ID":"7efe602e8ba1524abe391aabbb698457","Type":"ContainerStarted","Data":"b9c929b2079c2178d1e0dbb3a0cd5c7965226441feeb7576f7b91f47724b22ff"}
Oct 30 21:27:27.663785 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.663754    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-195.us-east-2.compute.internal" event={"ID":"7c1602df5a81ac24aa1efba1059dbbe8","Type":"ContainerStarted","Data":"674d20fe97c246ae471b27f055ab576608b0a4cad5fabb0dca5b5ee7a31bfc48"}
Oct 30 21:27:27.689850 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.688451    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-etc-sysctl-d\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:27:27.689850 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.688516    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-etc-sysctl-conf\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:27:27.689850 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.688545    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-run-openvswitch\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:27:27.689850 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.688577    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/48efa9dc-3c20-4f81-86f3-c87ad697772c-serviceca\") pod \"node-ca-jgns8\" (UID: \"48efa9dc-3c20-4f81-86f3-c87ad697772c\") " pod="openshift-image-registry/node-ca-jgns8"
Oct 30 21:27:27.689850 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.688602    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5d90d533-ebca-46db-ab59-733837db23e6-ovnkube-script-lib\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:27:27.689850 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.688630    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f171661-33a2-4631-81c7-bc10a5db791f-original-pull-secret\") pod \"global-pull-secret-syncer-csbxh\" (UID: \"5f171661-33a2-4631-81c7-bc10a5db791f\") " pod="kube-system/global-pull-secret-syncer-csbxh"
Oct 30 21:27:27.689850 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.688655    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-host\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:27:27.689850 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.688659    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-etc-sysctl-d\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:27:27.689850 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.688682    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-host-cni-netd\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:27:27.689850 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.688715    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5f171661-33a2-4631-81c7-bc10a5db791f-dbus\") pod \"global-pull-secret-syncer-csbxh\" (UID: \"5f171661-33a2-4631-81c7-bc10a5db791f\") " pod="kube-system/global-pull-secret-syncer-csbxh"
Oct 30 21:27:27.689850 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.688738    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-etc-sysconfig\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:27:27.689850 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.688760    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-log-socket\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:27:27.689850 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.688783    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-host-run-ovn-kubernetes\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:27:27.689850 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.688807    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5d90d533-ebca-46db-ab59-733837db23e6-ovnkube-config\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:27:27.689850 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.688833    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5d90d533-ebca-46db-ab59-733837db23e6-env-overrides\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:27:27.689850 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.688858    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rf7bl\" (UniqueName: \"kubernetes.io/projected/5d90d533-ebca-46db-ab59-733837db23e6-kube-api-access-rf7bl\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:27:27.689850 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.688896    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dv6wd\" (UniqueName: \"kubernetes.io/projected/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-kube-api-access-dv6wd\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:27:27.690818 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.688919    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/baa8d0c0-d477-4f30-a0f6-375fc84cc448-hosts-file\") pod \"node-resolver-97tsr\" (UID: \"baa8d0c0-d477-4f30-a0f6-375fc84cc448\") " pod="openshift-dns/node-resolver-97tsr"
Oct 30 21:27:27.690818 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.688941    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klwsc\" (UniqueName: \"kubernetes.io/projected/baa8d0c0-d477-4f30-a0f6-375fc84cc448-kube-api-access-klwsc\") pod \"node-resolver-97tsr\" (UID: \"baa8d0c0-d477-4f30-a0f6-375fc84cc448\") " pod="openshift-dns/node-resolver-97tsr"
Oct 30 21:27:27.690818 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.688971    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ee6de179-7e35-46a5-86e1-c7c31428c149-konnectivity-ca\") pod \"konnectivity-agent-l8g24\" (UID: \"ee6de179-7e35-46a5-86e1-c7c31428c149\") " pod="kube-system/konnectivity-agent-l8g24"
Oct 30 21:27:27.690818 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.688997    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/48efa9dc-3c20-4f81-86f3-c87ad697772c-host\") pod \"node-ca-jgns8\" (UID: \"48efa9dc-3c20-4f81-86f3-c87ad697772c\") " pod="openshift-image-registry/node-ca-jgns8"
Oct 30 21:27:27.690818 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.689028    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5f171661-33a2-4631-81c7-bc10a5db791f-kubelet-config\") pod \"global-pull-secret-syncer-csbxh\" (UID: \"5f171661-33a2-4631-81c7-bc10a5db791f\") " pod="kube-system/global-pull-secret-syncer-csbxh"
Oct 30 21:27:27.690818 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.689054    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-systemd-units\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:27:27.690818 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.689077    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-var-lib-openvswitch\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:27:27.690818 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.689127    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-etc-openvswitch\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:27:27.690818 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.689153    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-etc-kubernetes\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:27:27.690818 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.689181    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-lib-modules\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:27:27.690818 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.689206    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-var-lib-kubelet\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:27:27.690818 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.689231    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-host-kubelet\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:27:27.690818 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.689261    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:27:27.690818 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.689297    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-etc-tuned\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:27:27.690818 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.689326    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-etc-modprobe-d\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:27:27.690818 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.689365    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-run-systemd\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:27:27.690818 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.689390    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5d90d533-ebca-46db-ab59-733837db23e6-ovn-node-metrics-cert\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:27:27.691551 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.689422    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-run\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:27:27.691551 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.689421    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-run-openvswitch\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:27:27.691551 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.689469    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ee6de179-7e35-46a5-86e1-c7c31428c149-agent-certs\") pod \"konnectivity-agent-l8g24\" (UID: \"ee6de179-7e35-46a5-86e1-c7c31428c149\") " pod="kube-system/konnectivity-agent-l8g24"
Oct 30 21:27:27.691551 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.689500    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-log-socket\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:27:27.691551 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.689544    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-etc-sysctl-conf\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:27:27.691551 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.689560    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-host-run-netns\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:27:27.691551 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.689582    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-lib-modules\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:27:27.691551 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.689618    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-var-lib-kubelet\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:27:27.691551 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.689643    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-etc-sysconfig\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:27:27.691551 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.689658    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/48efa9dc-3c20-4f81-86f3-c87ad697772c-host\") pod \"node-ca-jgns8\" (UID: \"48efa9dc-3c20-4f81-86f3-c87ad697772c\") " pod="openshift-image-registry/node-ca-jgns8"
Oct 30 21:27:27.691551 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.689704    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/baa8d0c0-d477-4f30-a0f6-375fc84cc448-hosts-file\") pod \"node-resolver-97tsr\" (UID: \"baa8d0c0-d477-4f30-a0f6-375fc84cc448\") " pod="openshift-dns/node-resolver-97tsr"
Oct 30 21:27:27.691551 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.689712    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:27:27.691551 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.689743    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-etc-openvswitch\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:27:27.691551 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.689494    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-host-run-netns\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:27:27.691551 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.689802    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-run-systemd\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:27:27.691551 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.689816    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-host-cni-bin\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:27:27.691551 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.689850    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-etc-systemd\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:27:27.691551 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.689860    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-host-cni-netd\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:27:27.692588 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.689921    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-tmp\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:27:27.692588 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.689927    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5f171661-33a2-4631-81c7-bc10a5db791f-dbus\") pod \"global-pull-secret-syncer-csbxh\" (UID: \"5f171661-33a2-4631-81c7-bc10a5db791f\") " pod="kube-system/global-pull-secret-syncer-csbxh"
Oct 30 21:27:27.692588 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.689950    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-node-log\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:27:27.692588 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.689971    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-etc-systemd\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:27:27.692588 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.689982    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/baa8d0c0-d477-4f30-a0f6-375fc84cc448-tmp-dir\") pod \"node-resolver-97tsr\" (UID: \"baa8d0c0-d477-4f30-a0f6-375fc84cc448\") " pod="openshift-dns/node-resolver-97tsr"
Oct 30 21:27:27.692588 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.690009    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtmzr\" (UniqueName: \"kubernetes.io/projected/48efa9dc-3c20-4f81-86f3-c87ad697772c-kube-api-access-vtmzr\") pod \"node-ca-jgns8\" (UID: \"48efa9dc-3c20-4f81-86f3-c87ad697772c\") " pod="openshift-image-registry/node-ca-jgns8"
Oct 30 21:27:27.692588 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.690036    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-host-slash\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:27:27.692588 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.690110    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5d90d533-ebca-46db-ab59-733837db23e6-ovnkube-script-lib\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:27:27.692588 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.690170    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-var-lib-openvswitch\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:27:27.692588 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.690260    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-run-ovn\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:27:27.692588 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.690290    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-sys\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:27:27.692588 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.690439    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-sys\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:27:27.692588 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.690479    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-host-slash\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:27:27.692588 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.690524    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-host-kubelet\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:27:27.692588 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.690535    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/48efa9dc-3c20-4f81-86f3-c87ad697772c-serviceca\") pod \"node-ca-jgns8\" (UID: \"48efa9dc-3c20-4f81-86f3-c87ad697772c\") " pod="openshift-image-registry/node-ca-jgns8"
Oct 30 21:27:27.692588 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.690554    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-run\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:27:27.692588 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.690570    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5f171661-33a2-4631-81c7-bc10a5db791f-kubelet-config\") pod \"global-pull-secret-syncer-csbxh\" (UID: \"5f171661-33a2-4631-81c7-bc10a5db791f\") " pod="kube-system/global-pull-secret-syncer-csbxh"
Oct 30 21:27:27.692588 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.690602    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ee6de179-7e35-46a5-86e1-c7c31428c149-konnectivity-ca\") pod \"konnectivity-agent-l8g24\" (UID: \"ee6de179-7e35-46a5-86e1-c7c31428c149\") " pod="kube-system/konnectivity-agent-l8g24"
Oct 30 21:27:27.693550 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.690610    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-host-run-ovn-kubernetes\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:27:27.693550 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.690624    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-etc-kubernetes\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:27:27.693550 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:27.690661    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:27:27.693550 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.690745    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-systemd-units\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:27:27.693550 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.690746    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-run-ovn\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:27:27.693550 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.690786    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-node-log\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:27:27.693550 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.691037    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-etc-modprobe-d\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:27:27.693550 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.691062    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/baa8d0c0-d477-4f30-a0f6-375fc84cc448-tmp-dir\") pod \"node-resolver-97tsr\" (UID: \"baa8d0c0-d477-4f30-a0f6-375fc84cc448\") " pod="openshift-dns/node-resolver-97tsr"
Oct 30 21:27:27.693550 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.691082    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-host-cni-bin\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:27:27.693550 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.691130    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-host\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:27:27.693550 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:27.691182    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f171661-33a2-4631-81c7-bc10a5db791f-original-pull-secret podName:5f171661-33a2-4631-81c7-bc10a5db791f nodeName:}" failed. No retries permitted until 2026-10-30 21:27:28.191169383 +0000 UTC m=+3.072232320 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5f171661-33a2-4631-81c7-bc10a5db791f-original-pull-secret") pod "global-pull-secret-syncer-csbxh" (UID: "5f171661-33a2-4631-81c7-bc10a5db791f") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:27:27.693550 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.691636    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5d90d533-ebca-46db-ab59-733837db23e6-env-overrides\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:27:27.693550 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.691967    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5d90d533-ebca-46db-ab59-733837db23e6-ovnkube-config\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:27:27.694143 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.694030    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-tmp\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:27:27.694277 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.694251    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ee6de179-7e35-46a5-86e1-c7c31428c149-agent-certs\") pod \"konnectivity-agent-l8g24\" (UID: \"ee6de179-7e35-46a5-86e1-c7c31428c149\") " pod="kube-system/konnectivity-agent-l8g24"
Oct 30 21:27:27.695055 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.695029    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-etc-tuned\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:27:27.696810 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.696748    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5d90d533-ebca-46db-ab59-733837db23e6-ovn-node-metrics-cert\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:27:27.705572 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.705538    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtmzr\" (UniqueName: \"kubernetes.io/projected/48efa9dc-3c20-4f81-86f3-c87ad697772c-kube-api-access-vtmzr\") pod \"node-ca-jgns8\" (UID: \"48efa9dc-3c20-4f81-86f3-c87ad697772c\") " pod="openshift-image-registry/node-ca-jgns8"
Oct 30 21:27:27.705728 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.705607    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dv6wd\" (UniqueName: \"kubernetes.io/projected/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-kube-api-access-dv6wd\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:27:27.706924 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.706896    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-klwsc\" (UniqueName: \"kubernetes.io/projected/baa8d0c0-d477-4f30-a0f6-375fc84cc448-kube-api-access-klwsc\") pod \"node-resolver-97tsr\" (UID: \"baa8d0c0-d477-4f30-a0f6-375fc84cc448\") " pod="openshift-dns/node-resolver-97tsr"
Oct 30 21:27:27.707788 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.707760    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rf7bl\" (UniqueName: \"kubernetes.io/projected/5d90d533-ebca-46db-ab59-733837db23e6-kube-api-access-rf7bl\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:27:27.787260 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.787222    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qcj76"
Oct 30 21:27:27.799651 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:27.799616    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf90e1fc_bac9_4c33_8dc4_dbca18974399.slice/crio-868d0e77ebffb05386c7b1e5376b26ba106aa3b18521844b1d33ca992dd6d4cc WatchSource:0}: Error finding container 868d0e77ebffb05386c7b1e5376b26ba106aa3b18521844b1d33ca992dd6d4cc: Status 404 returned error can't find the container with id 868d0e77ebffb05386c7b1e5376b26ba106aa3b18521844b1d33ca992dd6d4cc
Oct 30 21:27:27.809201 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.809169    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-d9b8n"
Oct 30 21:27:27.814922 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.814896    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-l67jk"
Oct 30 21:27:27.815042 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.814989    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bn9ng"
Oct 30 21:27:27.822019 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:27.821990    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e93253e_a238_4426_a584_b9de8e931cf7.slice/crio-04cbe888f55f0793a5c9408b6f252956dba81d6385e86f4366157ebd66e9a3e6 WatchSource:0}: Error finding container 04cbe888f55f0793a5c9408b6f252956dba81d6385e86f4366157ebd66e9a3e6: Status 404 returned error can't find the container with id 04cbe888f55f0793a5c9408b6f252956dba81d6385e86f4366157ebd66e9a3e6
Oct 30 21:27:27.823028 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.823005    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-jgns8"
Oct 30 21:27:27.831579 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:27.831549    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6750ff3_d2db_42b7_9c85_7896c9ba67c9.slice/crio-d122ee4dad1efd8d9cfa8ca3bf428e14dcdb5a02d16d145d71957b4666f0361f WatchSource:0}: Error finding container d122ee4dad1efd8d9cfa8ca3bf428e14dcdb5a02d16d145d71957b4666f0361f: Status 404 returned error can't find the container with id d122ee4dad1efd8d9cfa8ca3bf428e14dcdb5a02d16d145d71957b4666f0361f
Oct 30 21:27:27.837927 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:27.837897    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84a732a8_045d_4a8b_bb54_859c8d46f08d.slice/crio-5f2f9bf4612d88a6d984149720b0481036f148988b145cddf801e00076f200b1 WatchSource:0}: Error finding container 5f2f9bf4612d88a6d984149720b0481036f148988b145cddf801e00076f200b1: Status 404 returned error can't find the container with id 5f2f9bf4612d88a6d984149720b0481036f148988b145cddf801e00076f200b1
Oct 30 21:27:27.842190 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:27.842133    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48efa9dc_3c20_4f81_86f3_c87ad697772c.slice/crio-f0e4195569bff5c7940a6afbb3d373bff17293a533a3c98574dcfcdb118b3a57 WatchSource:0}: Error finding container f0e4195569bff5c7940a6afbb3d373bff17293a533a3c98574dcfcdb118b3a57: Status 404 returned error can't find the container with id f0e4195569bff5c7940a6afbb3d373bff17293a533a3c98574dcfcdb118b3a57
Oct 30 21:27:27.850164 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.849968    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-l8g24"
Oct 30 21:27:27.852589 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.852566    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:27:27.856373 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.856348    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:27:27.861115 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.861091    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-97tsr"
Oct 30 21:27:27.867889 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:27.867856    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee6de179_7e35_46a5_86e1_c7c31428c149.slice/crio-1567ec928ebf4029842c9202f9fbc1ca86870cc5b45a9cf2214aa801ce31a73f WatchSource:0}: Error finding container 1567ec928ebf4029842c9202f9fbc1ca86870cc5b45a9cf2214aa801ce31a73f: Status 404 returned error can't find the container with id 1567ec928ebf4029842c9202f9fbc1ca86870cc5b45a9cf2214aa801ce31a73f
Oct 30 21:27:27.869718 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:27.869689    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d90d533_ebca_46db_ab59_733837db23e6.slice/crio-28d6df74a321bc7402b0991709090e293ce1a21aea1c20e6a53e533ecaaf619e WatchSource:0}: Error finding container 28d6df74a321bc7402b0991709090e293ce1a21aea1c20e6a53e533ecaaf619e: Status 404 returned error can't find the container with id 28d6df74a321bc7402b0991709090e293ce1a21aea1c20e6a53e533ecaaf619e
Oct 30 21:27:27.878107 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:27.877972    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7deac9ed_2d3a_48c3_bcf8_566ffdef0869.slice/crio-875cac0c37f15cc8b02b2df5c450e217b16f7254dbcda0ae6f6ea00c5be62bd6 WatchSource:0}: Error finding container 875cac0c37f15cc8b02b2df5c450e217b16f7254dbcda0ae6f6ea00c5be62bd6: Status 404 returned error can't find the container with id 875cac0c37f15cc8b02b2df5c450e217b16f7254dbcda0ae6f6ea00c5be62bd6
Oct 30 21:27:27.883088 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:27.883058    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbaa8d0c0_d477_4f30_a0f6_375fc84cc448.slice/crio-57dbd87c6c23e36cb2934dbe29990855b3d7f1a4fedbd560cc649beff9706264 WatchSource:0}: Error finding container 57dbd87c6c23e36cb2934dbe29990855b3d7f1a4fedbd560cc649beff9706264: Status 404 returned error can't find the container with id 57dbd87c6c23e36cb2934dbe29990855b3d7f1a4fedbd560cc649beff9706264
Oct 30 21:27:27.989655 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:27.989576    2573 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:27:28.094699 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:28.094650    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/352b5b4c-c640-4300-abf3-aa70b08cbb4b-metrics-certs\") pod \"network-metrics-daemon-vwxrd\" (UID: \"352b5b4c-c640-4300-abf3-aa70b08cbb4b\") " pod="openshift-multus/network-metrics-daemon-vwxrd"
Oct 30 21:27:28.094895 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:28.094845    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:27:28.094973 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:28.094929    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/352b5b4c-c640-4300-abf3-aa70b08cbb4b-metrics-certs podName:352b5b4c-c640-4300-abf3-aa70b08cbb4b nodeName:}" failed. No retries permitted until 2026-10-30 21:27:29.094907953 +0000 UTC m=+3.975970906 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/352b5b4c-c640-4300-abf3-aa70b08cbb4b-metrics-certs") pod "network-metrics-daemon-vwxrd" (UID: "352b5b4c-c640-4300-abf3-aa70b08cbb4b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:27:28.195820 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:28.195772    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f171661-33a2-4631-81c7-bc10a5db791f-original-pull-secret\") pod \"global-pull-secret-syncer-csbxh\" (UID: \"5f171661-33a2-4631-81c7-bc10a5db791f\") " pod="kube-system/global-pull-secret-syncer-csbxh"
Oct 30 21:27:28.195820 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:28.195829    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ms8rv\" (UniqueName: \"kubernetes.io/projected/3913f7d4-86cb-4e86-af5a-debd8b859252-kube-api-access-ms8rv\") pod \"network-check-target-4cvlt\" (UID: \"3913f7d4-86cb-4e86-af5a-debd8b859252\") " pod="openshift-network-diagnostics/network-check-target-4cvlt"
Oct 30 21:27:28.196072 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:28.195951    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:27:28.196072 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:28.195967    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:27:28.196072 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:28.195977    2573 projected.go:196] Error preparing data for projected volume kube-api-access-ms8rv for pod openshift-network-diagnostics/network-check-target-4cvlt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:27:28.196072 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:28.195952    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:27:28.196072 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:28.196027    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3913f7d4-86cb-4e86-af5a-debd8b859252-kube-api-access-ms8rv podName:3913f7d4-86cb-4e86-af5a-debd8b859252 nodeName:}" failed. No retries permitted until 2026-10-30 21:27:29.196012062 +0000 UTC m=+4.077074995 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-ms8rv" (UniqueName: "kubernetes.io/projected/3913f7d4-86cb-4e86-af5a-debd8b859252-kube-api-access-ms8rv") pod "network-check-target-4cvlt" (UID: "3913f7d4-86cb-4e86-af5a-debd8b859252") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:27:28.196345 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:28.196085    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f171661-33a2-4631-81c7-bc10a5db791f-original-pull-secret podName:5f171661-33a2-4631-81c7-bc10a5db791f nodeName:}" failed. No retries permitted until 2026-10-30 21:27:29.19605089 +0000 UTC m=+4.077113823 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5f171661-33a2-4631-81c7-bc10a5db791f-original-pull-secret") pod "global-pull-secret-syncer-csbxh" (UID: "5f171661-33a2-4631-81c7-bc10a5db791f") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:27:28.529447 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:28.529279    2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 21:22:27 +0000 UTC" deadline="2028-07-08 13:35:45.374780362 +0000 UTC"
Oct 30 21:27:28.529447 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:28.529388    2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14800h8m16.845412596s"
Oct 30 21:27:28.530404 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:28.529796    2573 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:27:28.678225 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:28.678181    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-97tsr" event={"ID":"baa8d0c0-d477-4f30-a0f6-375fc84cc448","Type":"ContainerStarted","Data":"57dbd87c6c23e36cb2934dbe29990855b3d7f1a4fedbd560cc649beff9706264"}
Oct 30 21:27:28.683090 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:28.683049    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gl47s" event={"ID":"5d90d533-ebca-46db-ab59-733837db23e6","Type":"ContainerStarted","Data":"28d6df74a321bc7402b0991709090e293ce1a21aea1c20e6a53e533ecaaf619e"}
Oct 30 21:27:28.686371 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:28.686334    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-jgns8" event={"ID":"48efa9dc-3c20-4f81-86f3-c87ad697772c","Type":"ContainerStarted","Data":"f0e4195569bff5c7940a6afbb3d373bff17293a533a3c98574dcfcdb118b3a57"}
Oct 30 21:27:28.705522 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:28.705376    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bn9ng" event={"ID":"84a732a8-045d-4a8b-bb54-859c8d46f08d","Type":"ContainerStarted","Data":"5f2f9bf4612d88a6d984149720b0481036f148988b145cddf801e00076f200b1"}
Oct 30 21:27:28.708086 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:28.707952    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l67jk" event={"ID":"f6750ff3-d2db-42b7-9c85-7896c9ba67c9","Type":"ContainerStarted","Data":"d122ee4dad1efd8d9cfa8ca3bf428e14dcdb5a02d16d145d71957b4666f0361f"}
Oct 30 21:27:28.732820 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:28.732718    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-d9b8n" event={"ID":"5e93253e-a238-4426-a584-b9de8e931cf7","Type":"ContainerStarted","Data":"04cbe888f55f0793a5c9408b6f252956dba81d6385e86f4366157ebd66e9a3e6"}
Oct 30 21:27:28.740999 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:28.740656    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7ntsz" event={"ID":"7deac9ed-2d3a-48c3-bcf8-566ffdef0869","Type":"ContainerStarted","Data":"875cac0c37f15cc8b02b2df5c450e217b16f7254dbcda0ae6f6ea00c5be62bd6"}
Oct 30 21:27:28.749022 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:28.748768    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-l8g24" event={"ID":"ee6de179-7e35-46a5-86e1-c7c31428c149","Type":"ContainerStarted","Data":"1567ec928ebf4029842c9202f9fbc1ca86870cc5b45a9cf2214aa801ce31a73f"}
Oct 30 21:27:28.759684 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:28.759643    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qcj76" event={"ID":"cf90e1fc-bac9-4c33-8dc4-dbca18974399","Type":"ContainerStarted","Data":"868d0e77ebffb05386c7b1e5376b26ba106aa3b18521844b1d33ca992dd6d4cc"}
Oct 30 21:27:29.105636 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:29.105593    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/352b5b4c-c640-4300-abf3-aa70b08cbb4b-metrics-certs\") pod \"network-metrics-daemon-vwxrd\" (UID: \"352b5b4c-c640-4300-abf3-aa70b08cbb4b\") " pod="openshift-multus/network-metrics-daemon-vwxrd"
Oct 30 21:27:29.105816 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:29.105759    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:27:29.105874 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:29.105825    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/352b5b4c-c640-4300-abf3-aa70b08cbb4b-metrics-certs podName:352b5b4c-c640-4300-abf3-aa70b08cbb4b nodeName:}" failed. No retries permitted until 2026-10-30 21:27:31.105806431 +0000 UTC m=+5.986869378 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/352b5b4c-c640-4300-abf3-aa70b08cbb4b-metrics-certs") pod "network-metrics-daemon-vwxrd" (UID: "352b5b4c-c640-4300-abf3-aa70b08cbb4b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:27:29.208521 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:29.206808    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f171661-33a2-4631-81c7-bc10a5db791f-original-pull-secret\") pod \"global-pull-secret-syncer-csbxh\" (UID: \"5f171661-33a2-4631-81c7-bc10a5db791f\") " pod="kube-system/global-pull-secret-syncer-csbxh"
Oct 30 21:27:29.208521 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:29.206871    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ms8rv\" (UniqueName: \"kubernetes.io/projected/3913f7d4-86cb-4e86-af5a-debd8b859252-kube-api-access-ms8rv\") pod \"network-check-target-4cvlt\" (UID: \"3913f7d4-86cb-4e86-af5a-debd8b859252\") " pod="openshift-network-diagnostics/network-check-target-4cvlt"
Oct 30 21:27:29.208521 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:29.207070    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:27:29.208521 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:29.207095    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:27:29.208521 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:29.207110    2573 projected.go:196] Error preparing data for projected volume kube-api-access-ms8rv for pod openshift-network-diagnostics/network-check-target-4cvlt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:27:29.208521 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:29.207170    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3913f7d4-86cb-4e86-af5a-debd8b859252-kube-api-access-ms8rv podName:3913f7d4-86cb-4e86-af5a-debd8b859252 nodeName:}" failed. No retries permitted until 2026-10-30 21:27:31.207150203 +0000 UTC m=+6.088213151 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-ms8rv" (UniqueName: "kubernetes.io/projected/3913f7d4-86cb-4e86-af5a-debd8b859252-kube-api-access-ms8rv") pod "network-check-target-4cvlt" (UID: "3913f7d4-86cb-4e86-af5a-debd8b859252") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:27:29.208521 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:29.207599    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:27:29.208521 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:29.207648    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f171661-33a2-4631-81c7-bc10a5db791f-original-pull-secret podName:5f171661-33a2-4631-81c7-bc10a5db791f nodeName:}" failed. No retries permitted until 2026-10-30 21:27:31.207633118 +0000 UTC m=+6.088696066 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5f171661-33a2-4631-81c7-bc10a5db791f-original-pull-secret") pod "global-pull-secret-syncer-csbxh" (UID: "5f171661-33a2-4631-81c7-bc10a5db791f") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:27:29.529604 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:29.529559    2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 21:22:27 +0000 UTC" deadline="2028-05-03 13:42:32.210463396 +0000 UTC"
Oct 30 21:27:29.529604 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:29.529584    2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13216h15m2.680882574s"
Oct 30 21:27:29.662431 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:29.660932    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vwxrd"
Oct 30 21:27:29.662431 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:29.661317    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vwxrd" podUID="352b5b4c-c640-4300-abf3-aa70b08cbb4b"
Oct 30 21:27:29.662431 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:29.661827    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-csbxh"
Oct 30 21:27:29.662431 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:29.661928    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-csbxh" podUID="5f171661-33a2-4631-81c7-bc10a5db791f"
Oct 30 21:27:29.662431 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:29.661988    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4cvlt"
Oct 30 21:27:29.662431 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:29.662059    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4cvlt" podUID="3913f7d4-86cb-4e86-af5a-debd8b859252"
Oct 30 21:27:29.774561 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:29.774344    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-195.us-east-2.compute.internal" event={"ID":"7c1602df5a81ac24aa1efba1059dbbe8","Type":"ContainerStarted","Data":"ef221f135be982cbcdfd9ff62f4c5f763aedc599e41c092987c770d1bebb1a93"}
Oct 30 21:27:29.777205 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:29.777134    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-195.us-east-2.compute.internal" event={"ID":"7efe602e8ba1524abe391aabbb698457","Type":"ContainerStarted","Data":"a1435fa8c0ca5dc2d2b220c80044bade18bae4c3b25fd17682d861f4c413c59a"}
Oct 30 21:27:29.794626 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:29.794552    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-195.us-east-2.compute.internal" podStartSLOduration=2.794532361 podStartE2EDuration="2.794532361s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:27:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:27:29.793823967 +0000 UTC m=+4.674886925" watchObservedRunningTime="2026-10-30 21:27:29.794532361 +0000 UTC m=+4.675595318"
Oct 30 21:27:30.801133 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:30.800314    2573 generic.go:356] "Generic (PLEG): container finished" podID="7efe602e8ba1524abe391aabbb698457" containerID="a1435fa8c0ca5dc2d2b220c80044bade18bae4c3b25fd17682d861f4c413c59a" exitCode=0
Oct 30 21:27:30.802200 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:30.801785    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-195.us-east-2.compute.internal" event={"ID":"7efe602e8ba1524abe391aabbb698457","Type":"ContainerDied","Data":"a1435fa8c0ca5dc2d2b220c80044bade18bae4c3b25fd17682d861f4c413c59a"}
Oct 30 21:27:31.127098 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:31.126826    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/352b5b4c-c640-4300-abf3-aa70b08cbb4b-metrics-certs\") pod \"network-metrics-daemon-vwxrd\" (UID: \"352b5b4c-c640-4300-abf3-aa70b08cbb4b\") " pod="openshift-multus/network-metrics-daemon-vwxrd"
Oct 30 21:27:31.127098 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:31.127017    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:27:31.127288 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:31.127110    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/352b5b4c-c640-4300-abf3-aa70b08cbb4b-metrics-certs podName:352b5b4c-c640-4300-abf3-aa70b08cbb4b nodeName:}" failed. No retries permitted until 2026-10-30 21:27:35.1270719 +0000 UTC m=+10.008134858 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/352b5b4c-c640-4300-abf3-aa70b08cbb4b-metrics-certs") pod "network-metrics-daemon-vwxrd" (UID: "352b5b4c-c640-4300-abf3-aa70b08cbb4b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:27:31.229738 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:31.229677    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f171661-33a2-4631-81c7-bc10a5db791f-original-pull-secret\") pod \"global-pull-secret-syncer-csbxh\" (UID: \"5f171661-33a2-4631-81c7-bc10a5db791f\") " pod="kube-system/global-pull-secret-syncer-csbxh"
Oct 30 21:27:31.229903 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:31.229783    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ms8rv\" (UniqueName: \"kubernetes.io/projected/3913f7d4-86cb-4e86-af5a-debd8b859252-kube-api-access-ms8rv\") pod \"network-check-target-4cvlt\" (UID: \"3913f7d4-86cb-4e86-af5a-debd8b859252\") " pod="openshift-network-diagnostics/network-check-target-4cvlt"
Oct 30 21:27:31.230093 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:31.229983    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:27:31.230093 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:31.230026    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:27:31.230093 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:31.230041    2573 projected.go:196] Error preparing data for projected volume kube-api-access-ms8rv for pod openshift-network-diagnostics/network-check-target-4cvlt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:27:31.230279 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:31.230121    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3913f7d4-86cb-4e86-af5a-debd8b859252-kube-api-access-ms8rv podName:3913f7d4-86cb-4e86-af5a-debd8b859252 nodeName:}" failed. No retries permitted until 2026-10-30 21:27:35.230100716 +0000 UTC m=+10.111163664 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-ms8rv" (UniqueName: "kubernetes.io/projected/3913f7d4-86cb-4e86-af5a-debd8b859252-kube-api-access-ms8rv") pod "network-check-target-4cvlt" (UID: "3913f7d4-86cb-4e86-af5a-debd8b859252") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:27:31.230561 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:31.230541    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:27:31.230640 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:31.230598    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f171661-33a2-4631-81c7-bc10a5db791f-original-pull-secret podName:5f171661-33a2-4631-81c7-bc10a5db791f nodeName:}" failed. No retries permitted until 2026-10-30 21:27:35.230583347 +0000 UTC m=+10.111646296 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5f171661-33a2-4631-81c7-bc10a5db791f-original-pull-secret") pod "global-pull-secret-syncer-csbxh" (UID: "5f171661-33a2-4631-81c7-bc10a5db791f") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:27:31.659583 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:31.659546    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4cvlt"
Oct 30 21:27:31.659788 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:31.659606    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-csbxh"
Oct 30 21:27:31.659788 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:31.659694    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4cvlt" podUID="3913f7d4-86cb-4e86-af5a-debd8b859252"
Oct 30 21:27:31.659788 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:31.659783    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vwxrd"
Oct 30 21:27:31.660042 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:31.660009    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vwxrd" podUID="352b5b4c-c640-4300-abf3-aa70b08cbb4b"
Oct 30 21:27:31.660181 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:31.660107    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-csbxh" podUID="5f171661-33a2-4631-81c7-bc10a5db791f"
Oct 30 21:27:31.815457 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:31.815383    2573 generic.go:356] "Generic (PLEG): container finished" podID="f6750ff3-d2db-42b7-9c85-7896c9ba67c9" containerID="1cc6b317cd9fddeb36db820d12c3cfb4540385856cdea18e9fa0cbb43d467e5b" exitCode=0
Oct 30 21:27:31.815966 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:31.815521    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l67jk" event={"ID":"f6750ff3-d2db-42b7-9c85-7896c9ba67c9","Type":"ContainerDied","Data":"1cc6b317cd9fddeb36db820d12c3cfb4540385856cdea18e9fa0cbb43d467e5b"}
Oct 30 21:27:31.823299 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:31.823266    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-195.us-east-2.compute.internal" event={"ID":"7efe602e8ba1524abe391aabbb698457","Type":"ContainerStarted","Data":"b7f568d38ac5e35b04ee68347b079798ac640c11b6fa76d1e54e73b9d3905ba9"}
Oct 30 21:27:32.836085 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:32.836052    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-l8g24" event={"ID":"ee6de179-7e35-46a5-86e1-c7c31428c149","Type":"ContainerStarted","Data":"acee3bf558a41f0256e7cafad94a593b2336827a978d0cd5b1ba0dc3a4042efa"}
Oct 30 21:27:32.853735 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:32.853674    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-195.us-east-2.compute.internal" podStartSLOduration=5.853655667 podStartE2EDuration="5.853655667s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:27:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:27:31.873134366 +0000 UTC m=+6.754197326" watchObservedRunningTime="2026-10-30 21:27:32.853655667 +0000 UTC m=+7.734718622"
Oct 30 21:27:32.854201 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:32.854162    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-l8g24" podStartSLOduration=2.651635563 podStartE2EDuration="6.854148177s" totalImagesPullingTime="4.202512614s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:27:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:27:32.853399504 +0000 UTC m=+7.734462455" watchObservedRunningTime="2026-10-30 21:27:32.854148177 +0000 UTC m=+7.735211126"
Oct 30 21:27:33.657334 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:33.657254    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4cvlt"
Oct 30 21:27:33.657510 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:33.657264    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-csbxh"
Oct 30 21:27:33.657510 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:33.657376    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4cvlt" podUID="3913f7d4-86cb-4e86-af5a-debd8b859252"
Oct 30 21:27:33.657510 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:33.657468    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-csbxh" podUID="5f171661-33a2-4631-81c7-bc10a5db791f"
Oct 30 21:27:33.657510 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:33.657264    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vwxrd"
Oct 30 21:27:33.657779 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:33.657572    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vwxrd" podUID="352b5b4c-c640-4300-abf3-aa70b08cbb4b"
Oct 30 21:27:33.841594 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:33.841542    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-jgns8" event={"ID":"48efa9dc-3c20-4f81-86f3-c87ad697772c","Type":"ContainerStarted","Data":"59c39d406198d390a643f6559cb366ebd5e6dee39acef89e6bf247bfc72f3564"}
Oct 30 21:27:34.845961 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:34.845924    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qcj76" event={"ID":"cf90e1fc-bac9-4c33-8dc4-dbca18974399","Type":"ContainerStarted","Data":"85c5b6cc286d7b9a8fc4e5ecf744db5b805223acafaecaf76f581367932ac35f"}
Oct 30 21:27:35.162676 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:35.162594    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/352b5b4c-c640-4300-abf3-aa70b08cbb4b-metrics-certs\") pod \"network-metrics-daemon-vwxrd\" (UID: \"352b5b4c-c640-4300-abf3-aa70b08cbb4b\") " pod="openshift-multus/network-metrics-daemon-vwxrd"
Oct 30 21:27:35.162843 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:35.162754    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:27:35.162843 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:35.162817    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/352b5b4c-c640-4300-abf3-aa70b08cbb4b-metrics-certs podName:352b5b4c-c640-4300-abf3-aa70b08cbb4b nodeName:}" failed. No retries permitted until 2026-10-30 21:27:43.162798195 +0000 UTC m=+18.043861170 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/352b5b4c-c640-4300-abf3-aa70b08cbb4b-metrics-certs") pod "network-metrics-daemon-vwxrd" (UID: "352b5b4c-c640-4300-abf3-aa70b08cbb4b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:27:35.238218 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:35.238184    2573 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-l8g24"
Oct 30 21:27:35.238979 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:35.238954    2573 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-l8g24"
Oct 30 21:27:35.256652 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:35.256593    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-jgns8" podStartSLOduration=3.697520185 podStartE2EDuration="9.256575056s" totalImagesPullingTime="5.559054871s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:27:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:27:33.859429534 +0000 UTC m=+8.740492488" watchObservedRunningTime="2026-10-30 21:27:35.256575056 +0000 UTC m=+10.137638013"
Oct 30 21:27:35.263726 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:35.263608    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f171661-33a2-4631-81c7-bc10a5db791f-original-pull-secret\") pod \"global-pull-secret-syncer-csbxh\" (UID: \"5f171661-33a2-4631-81c7-bc10a5db791f\") " pod="kube-system/global-pull-secret-syncer-csbxh"
Oct 30 21:27:35.263726 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:35.263664    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ms8rv\" (UniqueName: \"kubernetes.io/projected/3913f7d4-86cb-4e86-af5a-debd8b859252-kube-api-access-ms8rv\") pod \"network-check-target-4cvlt\" (UID: \"3913f7d4-86cb-4e86-af5a-debd8b859252\") " pod="openshift-network-diagnostics/network-check-target-4cvlt"
Oct 30 21:27:35.263914 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:35.263758    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:27:35.263914 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:35.263776    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:27:35.263914 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:35.263798    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:27:35.263914 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:35.263810    2573 projected.go:196] Error preparing data for projected volume kube-api-access-ms8rv for pod openshift-network-diagnostics/network-check-target-4cvlt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:27:35.263914 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:35.263833    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f171661-33a2-4631-81c7-bc10a5db791f-original-pull-secret podName:5f171661-33a2-4631-81c7-bc10a5db791f nodeName:}" failed. No retries permitted until 2026-10-30 21:27:43.263815782 +0000 UTC m=+18.144878725 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5f171661-33a2-4631-81c7-bc10a5db791f-original-pull-secret") pod "global-pull-secret-syncer-csbxh" (UID: "5f171661-33a2-4631-81c7-bc10a5db791f") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:27:35.263914 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:35.263850    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3913f7d4-86cb-4e86-af5a-debd8b859252-kube-api-access-ms8rv podName:3913f7d4-86cb-4e86-af5a-debd8b859252 nodeName:}" failed. No retries permitted until 2026-10-30 21:27:43.26384121 +0000 UTC m=+18.144904143 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-ms8rv" (UniqueName: "kubernetes.io/projected/3913f7d4-86cb-4e86-af5a-debd8b859252-kube-api-access-ms8rv") pod "network-check-target-4cvlt" (UID: "3913f7d4-86cb-4e86-af5a-debd8b859252") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:27:35.662691 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:35.662648    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4cvlt"
Oct 30 21:27:35.662783 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:35.662764    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4cvlt" podUID="3913f7d4-86cb-4e86-af5a-debd8b859252"
Oct 30 21:27:35.663153 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:35.663132    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vwxrd"
Oct 30 21:27:35.663229 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:35.663206    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vwxrd" podUID="352b5b4c-c640-4300-abf3-aa70b08cbb4b"
Oct 30 21:27:35.663275 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:35.663246    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-csbxh"
Oct 30 21:27:35.663305 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:35.663290    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-csbxh" podUID="5f171661-33a2-4631-81c7-bc10a5db791f"
Oct 30 21:27:35.683669 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:35.683641    2573 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 21:27:35.851191 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:35.851124    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qcj76" event={"ID":"cf90e1fc-bac9-4c33-8dc4-dbca18974399","Type":"ContainerStarted","Data":"4afbe576ccc0a6132c67df919af00f8cb1a859bf8cda51d321470829a0e7c457"}
Oct 30 21:27:36.679170 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:36.678847    2573 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T21:27:35.683661331Z","UUID":"7c1e47be-33e0-4983-b0fc-7976fa851ada","Handler":null,"Name":"","Endpoint":""}
Oct 30 21:27:36.680833 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:36.680810    2573 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 21:27:36.680981 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:36.680843    2573 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 21:27:36.858693 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:36.854425    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-97tsr" event={"ID":"baa8d0c0-d477-4f30-a0f6-375fc84cc448","Type":"ContainerStarted","Data":"ba402b4206cc30f26fada147122ed2d4682602c529a27b1cdd75a436f5f272f3"}
Oct 30 21:27:36.858693 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:36.857726    2573 generic.go:356] "Generic (PLEG): container finished" podID="f6750ff3-d2db-42b7-9c85-7896c9ba67c9" containerID="718d207492d450b1620381f78079cb900c0ea4c80634af5286e4a9875eb3af50" exitCode=0
Oct 30 21:27:36.859345 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:36.859165    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l67jk" event={"ID":"f6750ff3-d2db-42b7-9c85-7896c9ba67c9","Type":"ContainerDied","Data":"718d207492d450b1620381f78079cb900c0ea4c80634af5286e4a9875eb3af50"}
Oct 30 21:27:36.862748 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:36.862725    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-d9b8n" event={"ID":"5e93253e-a238-4426-a584-b9de8e931cf7","Type":"ContainerStarted","Data":"d4e8d4e974da261ca0c7af1410408f399612f8128fc2c7a1b680b678ee8d1b39"}
Oct 30 21:27:36.865159 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:36.865136    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qcj76" event={"ID":"cf90e1fc-bac9-4c33-8dc4-dbca18974399","Type":"ContainerStarted","Data":"4c09f05386d7568920cf4e22cf25e1375fef0c49b37b12879f288a8248e49463"}
Oct 30 21:27:36.900247 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:36.900186    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-97tsr" podStartSLOduration=2.551197324 podStartE2EDuration="10.900168504s" totalImagesPullingTime="8.34897118s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:27:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:27:36.876545738 +0000 UTC m=+11.757608692" watchObservedRunningTime="2026-10-30 21:27:36.900168504 +0000 UTC m=+11.781231462"
Oct 30 21:27:36.921569 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:36.921509    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qcj76" podStartSLOduration=2.412886997 podStartE2EDuration="10.921490997s" totalImagesPullingTime="8.508604s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:27:26 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:27:36.921125552 +0000 UTC m=+11.802188505" watchObservedRunningTime="2026-10-30 21:27:36.921490997 +0000 UTC m=+11.802553965"
Oct 30 21:27:36.941414 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:36.941331    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-d9b8n" podStartSLOduration=2.823758691 podStartE2EDuration="10.941314076s" totalImagesPullingTime="8.117555385s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:27:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:27:36.941071861 +0000 UTC m=+11.822134820" watchObservedRunningTime="2026-10-30 21:27:36.941314076 +0000 UTC m=+11.822377031"
Oct 30 21:27:37.658231 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:37.658193    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4cvlt"
Oct 30 21:27:37.658231 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:37.658214    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-csbxh"
Oct 30 21:27:37.658465 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:37.658324    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4cvlt" podUID="3913f7d4-86cb-4e86-af5a-debd8b859252"
Oct 30 21:27:37.658465 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:37.658393    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vwxrd"
Oct 30 21:27:37.658581 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:37.658501    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vwxrd" podUID="352b5b4c-c640-4300-abf3-aa70b08cbb4b"
Oct 30 21:27:37.658649 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:37.658587    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-csbxh" podUID="5f171661-33a2-4631-81c7-bc10a5db791f"
Oct 30 21:27:37.874200 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:37.874163    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bn9ng" event={"ID":"84a732a8-045d-4a8b-bb54-859c8d46f08d","Type":"ContainerStarted","Data":"93096cdab65a19116edfad5bbc9f06ca44a072c6ea42f133c56e1db981812387"}
Oct 30 21:27:37.877152 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:37.877119    2573 generic.go:356] "Generic (PLEG): container finished" podID="f6750ff3-d2db-42b7-9c85-7896c9ba67c9" containerID="ee78ec47141b73f993c9bea8c2dcf0df41883085d8accde67385072b698f82e7" exitCode=0
Oct 30 21:27:37.877583 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:37.877316    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l67jk" event={"ID":"f6750ff3-d2db-42b7-9c85-7896c9ba67c9","Type":"ContainerDied","Data":"ee78ec47141b73f993c9bea8c2dcf0df41883085d8accde67385072b698f82e7"}
Oct 30 21:27:37.891596 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:37.891542    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-bn9ng" podStartSLOduration=3.708482631 podStartE2EDuration="11.891528683s" totalImagesPullingTime="8.183046052s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:27:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:27:37.890722568 +0000 UTC m=+12.771785521" watchObservedRunningTime="2026-10-30 21:27:37.891528683 +0000 UTC m=+12.772591635"
Oct 30 21:27:38.880938 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:38.880910    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7ntsz" event={"ID":"7deac9ed-2d3a-48c3-bcf8-566ffdef0869","Type":"ContainerStarted","Data":"80b21eed99fa9f7f9513c194578b79c55c6f03895221f286676fe22590390c74"}
Oct 30 21:27:38.902215 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:38.902164    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-7ntsz" podStartSLOduration=2.343154162 podStartE2EDuration="12.902146503s" totalImagesPullingTime="10.558992341s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:27:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:27:38.902051753 +0000 UTC m=+13.783114705" watchObservedRunningTime="2026-10-30 21:27:38.902146503 +0000 UTC m=+13.783209460"
Oct 30 21:27:39.660562 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:39.660519    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4cvlt"
Oct 30 21:27:39.660735 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:39.660594    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-csbxh"
Oct 30 21:27:39.660735 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:39.660694    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vwxrd"
Oct 30 21:27:39.660735 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:39.660695    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4cvlt" podUID="3913f7d4-86cb-4e86-af5a-debd8b859252"
Oct 30 21:27:39.660899 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:39.660810    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vwxrd" podUID="352b5b4c-c640-4300-abf3-aa70b08cbb4b"
Oct 30 21:27:39.660899 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:39.660867    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-csbxh" podUID="5f171661-33a2-4631-81c7-bc10a5db791f"
Oct 30 21:27:39.885999 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:39.885967    2573 generic.go:356] "Generic (PLEG): container finished" podID="f6750ff3-d2db-42b7-9c85-7896c9ba67c9" containerID="601fac517d55b2bdddecc7d87a3b14947fc0797f8747034ff725d4044efa8c95" exitCode=0
Oct 30 21:27:39.886433 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:39.886042    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l67jk" event={"ID":"f6750ff3-d2db-42b7-9c85-7896c9ba67c9","Type":"ContainerDied","Data":"601fac517d55b2bdddecc7d87a3b14947fc0797f8747034ff725d4044efa8c95"}
Oct 30 21:27:41.661788 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:41.661762    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4cvlt"
Oct 30 21:27:41.661788 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:41.661780    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-csbxh"
Oct 30 21:27:41.662309 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:41.661875    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4cvlt" podUID="3913f7d4-86cb-4e86-af5a-debd8b859252"
Oct 30 21:27:41.662309 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:41.661913    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vwxrd"
Oct 30 21:27:41.662309 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:41.661998    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vwxrd" podUID="352b5b4c-c640-4300-abf3-aa70b08cbb4b"
Oct 30 21:27:41.662309 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:41.662053    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-csbxh" podUID="5f171661-33a2-4631-81c7-bc10a5db791f"
Oct 30 21:27:42.893723 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:42.893553    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gl47s" event={"ID":"5d90d533-ebca-46db-ab59-733837db23e6","Type":"ContainerStarted","Data":"6be2e251cb893f2ab7c4ff8d56c110d1fee0435da0988027af0e67592593265c"}
Oct 30 21:27:42.894061 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:42.893792    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gl47s" event={"ID":"5d90d533-ebca-46db-ab59-733837db23e6","Type":"ContainerStarted","Data":"1b0c60dc23aedf72b3f0c55c37f76d6228b29894726e956e216fb2ecaf4d388c"}
Oct 30 21:27:42.896160 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:42.896139    2573 generic.go:356] "Generic (PLEG): container finished" podID="f6750ff3-d2db-42b7-9c85-7896c9ba67c9" containerID="e9e39c89fd50eacd050925cff6dd2516294bcbed1507a3777a28e1f700a36c6f" exitCode=0
Oct 30 21:27:42.896241 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:42.896183    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l67jk" event={"ID":"f6750ff3-d2db-42b7-9c85-7896c9ba67c9","Type":"ContainerDied","Data":"e9e39c89fd50eacd050925cff6dd2516294bcbed1507a3777a28e1f700a36c6f"}
Oct 30 21:27:43.225212 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:43.225169    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/352b5b4c-c640-4300-abf3-aa70b08cbb4b-metrics-certs\") pod \"network-metrics-daemon-vwxrd\" (UID: \"352b5b4c-c640-4300-abf3-aa70b08cbb4b\") " pod="openshift-multus/network-metrics-daemon-vwxrd"
Oct 30 21:27:43.225380 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:43.225347    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:27:43.225437 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:43.225427    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/352b5b4c-c640-4300-abf3-aa70b08cbb4b-metrics-certs podName:352b5b4c-c640-4300-abf3-aa70b08cbb4b nodeName:}" failed. No retries permitted until 2026-10-30 21:27:59.225412641 +0000 UTC m=+34.106475587 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/352b5b4c-c640-4300-abf3-aa70b08cbb4b-metrics-certs") pod "network-metrics-daemon-vwxrd" (UID: "352b5b4c-c640-4300-abf3-aa70b08cbb4b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:27:43.326046 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:43.325960    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f171661-33a2-4631-81c7-bc10a5db791f-original-pull-secret\") pod \"global-pull-secret-syncer-csbxh\" (UID: \"5f171661-33a2-4631-81c7-bc10a5db791f\") " pod="kube-system/global-pull-secret-syncer-csbxh"
Oct 30 21:27:43.326046 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:43.326006    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ms8rv\" (UniqueName: \"kubernetes.io/projected/3913f7d4-86cb-4e86-af5a-debd8b859252-kube-api-access-ms8rv\") pod \"network-check-target-4cvlt\" (UID: \"3913f7d4-86cb-4e86-af5a-debd8b859252\") " pod="openshift-network-diagnostics/network-check-target-4cvlt"
Oct 30 21:27:43.326212 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:43.326118    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:27:43.326212 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:43.326137    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:27:43.326212 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:43.326155    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:27:43.326212 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:43.326167    2573 projected.go:196] Error preparing data for projected volume kube-api-access-ms8rv for pod openshift-network-diagnostics/network-check-target-4cvlt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:27:43.326212 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:43.326186    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f171661-33a2-4631-81c7-bc10a5db791f-original-pull-secret podName:5f171661-33a2-4631-81c7-bc10a5db791f nodeName:}" failed. No retries permitted until 2026-10-30 21:27:59.326169716 +0000 UTC m=+34.207232669 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5f171661-33a2-4631-81c7-bc10a5db791f-original-pull-secret") pod "global-pull-secret-syncer-csbxh" (UID: "5f171661-33a2-4631-81c7-bc10a5db791f") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:27:43.326212 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:43.326205    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3913f7d4-86cb-4e86-af5a-debd8b859252-kube-api-access-ms8rv podName:3913f7d4-86cb-4e86-af5a-debd8b859252 nodeName:}" failed. No retries permitted until 2026-10-30 21:27:59.326196175 +0000 UTC m=+34.207259110 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-ms8rv" (UniqueName: "kubernetes.io/projected/3913f7d4-86cb-4e86-af5a-debd8b859252-kube-api-access-ms8rv") pod "network-check-target-4cvlt" (UID: "3913f7d4-86cb-4e86-af5a-debd8b859252") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:27:43.645633 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:43.645549    2573 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-l8g24"
Oct 30 21:27:43.645766 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:43.645693    2573 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 21:27:43.646740 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:43.646718    2573 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-l8g24"
Oct 30 21:27:43.657235 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:43.657219    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-csbxh"
Oct 30 21:27:43.657235 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:43.657232    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vwxrd"
Oct 30 21:27:43.657331 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:43.657300    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-csbxh" podUID="5f171661-33a2-4631-81c7-bc10a5db791f"
Oct 30 21:27:43.657400 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:43.657353    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4cvlt"
Oct 30 21:27:43.657463 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:43.657447    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vwxrd" podUID="352b5b4c-c640-4300-abf3-aa70b08cbb4b"
Oct 30 21:27:43.657517 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:43.657504    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4cvlt" podUID="3913f7d4-86cb-4e86-af5a-debd8b859252"
Oct 30 21:27:43.900888 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:43.900806    2573 generic.go:356] "Generic (PLEG): container finished" podID="f6750ff3-d2db-42b7-9c85-7896c9ba67c9" containerID="4b1645c2c8087cf36011a87d81b2947223e8d7b98752f433167910385073a1b6" exitCode=0
Oct 30 21:27:43.901554 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:43.900900    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l67jk" event={"ID":"f6750ff3-d2db-42b7-9c85-7896c9ba67c9","Type":"ContainerDied","Data":"4b1645c2c8087cf36011a87d81b2947223e8d7b98752f433167910385073a1b6"}
Oct 30 21:27:43.904051 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:43.904026    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gl47s" event={"ID":"5d90d533-ebca-46db-ab59-733837db23e6","Type":"ContainerStarted","Data":"cf3d17b0590a954fe923f47baede50becbe2dcecb8eef56ad79fea65187e9cc1"}
Oct 30 21:27:43.904140 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:43.904090    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gl47s" event={"ID":"5d90d533-ebca-46db-ab59-733837db23e6","Type":"ContainerStarted","Data":"a682561e81a0b08287c4d1ca26b4759f4b4071973c1cfe0c1c71115b36c4e6bf"}
Oct 30 21:27:43.904140 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:43.904119    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gl47s" event={"ID":"5d90d533-ebca-46db-ab59-733837db23e6","Type":"ContainerStarted","Data":"b5ed111379f42fcff26c3b623a08c6b4713a57b65de7511fc581a01570ba1b85"}
Oct 30 21:27:43.904219 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:43.904153    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gl47s" event={"ID":"5d90d533-ebca-46db-ab59-733837db23e6","Type":"ContainerStarted","Data":"4f799d0d719d542af11d95391347498e0ad9846e77186bf48bc0f19d6a7fbeeb"}
Oct 30 21:27:44.911323 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:44.911289    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l67jk" event={"ID":"f6750ff3-d2db-42b7-9c85-7896c9ba67c9","Type":"ContainerStarted","Data":"9b73a05f23fde46fa1f9c59bcc1ee701a461411d6ac29f1689fe685e75986932"}
Oct 30 21:27:44.939530 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:44.939479    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-l67jk" podStartSLOduration=8.043589939 podStartE2EDuration="18.939465686s" totalImagesPullingTime="10.895875747s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:27:26 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:27:44.939136355 +0000 UTC m=+19.820199308" watchObservedRunningTime="2026-10-30 21:27:44.939465686 +0000 UTC m=+19.820528636"
Oct 30 21:27:45.658663 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:45.658629    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4cvlt"
Oct 30 21:27:45.658826 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:45.658725    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4cvlt" podUID="3913f7d4-86cb-4e86-af5a-debd8b859252"
Oct 30 21:27:45.658826 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:45.658813    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-csbxh"
Oct 30 21:27:45.658961 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:45.658921    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-csbxh" podUID="5f171661-33a2-4631-81c7-bc10a5db791f"
Oct 30 21:27:45.659026 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:45.658971    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vwxrd"
Oct 30 21:27:45.659080 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:45.659063    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vwxrd" podUID="352b5b4c-c640-4300-abf3-aa70b08cbb4b"
Oct 30 21:27:45.916480 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:45.916387    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gl47s" event={"ID":"5d90d533-ebca-46db-ab59-733837db23e6","Type":"ContainerStarted","Data":"54c236892d81dd23ac9817b54504dbd666df4beb5774b8a3a3f9ef17e5377bba"}
Oct 30 21:27:47.657783 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:47.657533    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4cvlt"
Oct 30 21:27:47.657783 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:47.657559    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vwxrd"
Oct 30 21:27:47.658306 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:47.657784    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4cvlt" podUID="3913f7d4-86cb-4e86-af5a-debd8b859252"
Oct 30 21:27:47.658306 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:47.657593    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-csbxh"
Oct 30 21:27:47.658306 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:47.657864    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vwxrd" podUID="352b5b4c-c640-4300-abf3-aa70b08cbb4b"
Oct 30 21:27:47.658306 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:47.657968    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-csbxh" podUID="5f171661-33a2-4631-81c7-bc10a5db791f"
Oct 30 21:27:47.924558 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:47.924485    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gl47s" event={"ID":"5d90d533-ebca-46db-ab59-733837db23e6","Type":"ContainerStarted","Data":"b62d0b59ffbb945eb5829d53ede1ffc4b652980d695782539281592f934f0159"}
Oct 30 21:27:47.924739 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:47.924722    2573 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:27:47.924821 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:47.924794    2573 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:27:47.924873 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:47.924844    2573 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:27:47.941183 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:47.941157    2573 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:27:47.941316 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:47.941257    2573 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:27:47.959505 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:47.959461    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-gl47s" podStartSLOduration=7.073391459 podStartE2EDuration="21.959447898s" totalImagesPullingTime="14.886056439s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:27:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:27:47.95891442 +0000 UTC m=+22.839977373" watchObservedRunningTime="2026-10-30 21:27:47.959447898 +0000 UTC m=+22.840510851"
Oct 30 21:27:49.097629 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:49.097596    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-csbxh"]
Oct 30 21:27:49.098093 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:49.097768    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-csbxh"
Oct 30 21:27:49.098093 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:49.097873    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-csbxh" podUID="5f171661-33a2-4631-81c7-bc10a5db791f"
Oct 30 21:27:49.100952 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:49.100929    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vwxrd"]
Oct 30 21:27:49.101055 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:49.101042    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vwxrd"
Oct 30 21:27:49.101147 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:49.101131    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vwxrd" podUID="352b5b4c-c640-4300-abf3-aa70b08cbb4b"
Oct 30 21:27:49.103905 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:49.103884    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4cvlt"]
Oct 30 21:27:49.104026 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:49.103981    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4cvlt"
Oct 30 21:27:49.104091 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:49.104073    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4cvlt" podUID="3913f7d4-86cb-4e86-af5a-debd8b859252"
Oct 30 21:27:50.657775 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:50.657728    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-csbxh"
Oct 30 21:27:50.658207 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:50.657758    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4cvlt"
Oct 30 21:27:50.658207 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:50.657850    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vwxrd"
Oct 30 21:27:50.658207 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:50.657860    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-csbxh" podUID="5f171661-33a2-4631-81c7-bc10a5db791f"
Oct 30 21:27:50.658207 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:50.657918    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4cvlt" podUID="3913f7d4-86cb-4e86-af5a-debd8b859252"
Oct 30 21:27:50.658207 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:50.657996    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vwxrd" podUID="352b5b4c-c640-4300-abf3-aa70b08cbb4b"
Oct 30 21:27:52.657920 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:52.657882    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4cvlt"
Oct 30 21:27:52.657920 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:52.657915    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-csbxh"
Oct 30 21:27:52.658553 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:52.657988    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vwxrd"
Oct 30 21:27:52.658553 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:52.658008    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4cvlt" podUID="3913f7d4-86cb-4e86-af5a-debd8b859252"
Oct 30 21:27:52.658553 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:52.658079    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vwxrd" podUID="352b5b4c-c640-4300-abf3-aa70b08cbb4b"
Oct 30 21:27:52.658553 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:52.658121    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-csbxh" podUID="5f171661-33a2-4631-81c7-bc10a5db791f"
Oct 30 21:27:54.657842 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:54.657756    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-csbxh"
Oct 30 21:27:54.658257 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:54.657856    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-csbxh" podUID="5f171661-33a2-4631-81c7-bc10a5db791f"
Oct 30 21:27:54.658257 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:54.657767    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vwxrd"
Oct 30 21:27:54.658257 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:54.657756    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4cvlt"
Oct 30 21:27:54.658257 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:54.657968    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vwxrd" podUID="352b5b4c-c640-4300-abf3-aa70b08cbb4b"
Oct 30 21:27:54.658257 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:54.658030    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4cvlt" podUID="3913f7d4-86cb-4e86-af5a-debd8b859252"
Oct 30 21:27:55.462734 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.462707    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-195.us-east-2.compute.internal" event="NodeReady"
Oct 30 21:27:55.462893 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.462833    2573 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Oct 30 21:27:55.498071 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.498037    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-9mt44"]
Oct 30 21:27:55.500986 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.500969    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-9mt44"
Oct 30 21:27:55.503083 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.503058    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-l4dbk"]
Oct 30 21:27:55.506124 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.506103    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-5bbfd9465-x28lh"]
Oct 30 21:27:55.506283 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.506260    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-l4dbk"
Oct 30 21:27:55.509026 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.509008    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-7c4986d449-7cgk9"]
Oct 30 21:27:55.509148 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.509129    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5bbfd9465-x28lh"
Oct 30 21:27:55.512436 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.512415    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-zvqvt"]
Oct 30 21:27:55.512567 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.512537    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-7cgk9"
Oct 30 21:27:55.515603 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.515583    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-9p8mb"]
Oct 30 21:27:55.515797 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.515766    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-zvqvt"
Oct 30 21:27:55.517222 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.517079    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\""
Oct 30 21:27:55.518632 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.518601    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\""
Oct 30 21:27:55.519180 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.519004    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Oct 30 21:27:55.519738 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.519719    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Oct 30 21:27:55.520812 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.519257    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Oct 30 21:27:55.521320 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.521300    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-kdrrd"]
Oct 30 21:27:55.522173 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.522155    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-9p8mb"
Oct 30 21:27:55.529166 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.529144    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-lbjhv"]
Oct 30 21:27:55.529273 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.529252    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-kdrrd"
Oct 30 21:27:55.532062 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.532044    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-sklxd"]
Oct 30 21:27:55.532164 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.532141    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-lbjhv"
Oct 30 21:27:55.534828 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.534810    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-76679b6db7-ltmt6"]
Oct 30 21:27:55.534934 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.534907    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-sklxd"
Oct 30 21:27:55.537617 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.537590    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-lwm89"]
Oct 30 21:27:55.537729 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.537638    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-wst66\""
Oct 30 21:27:55.537789 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.537746    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76679b6db7-ltmt6"
Oct 30 21:27:55.540326 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.540310    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-tdmc8"]
Oct 30 21:27:55.540433 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.540416    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-lwm89"
Oct 30 21:27:55.543100 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.543074    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-9r5sb"]
Oct 30 21:27:55.543789 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.543757    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-tdmc8"
Oct 30 21:27:55.546076 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.546057    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-9qppl"]
Oct 30 21:27:55.546198 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.546185    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-9r5sb"
Oct 30 21:27:55.548932 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.548914    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-86f87fb58b-5clbv"]
Oct 30 21:27:55.549092 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.549076    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-9qppl"
Oct 30 21:27:55.554150 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.554106    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-9mt44"]
Oct 30 21:27:55.554150 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.554148    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-l4dbk"]
Oct 30 21:27:55.554480 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.554169    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-zvqvt"]
Oct 30 21:27:55.554480 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.554189    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-7cgk9"]
Oct 30 21:27:55.554480 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.554209    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5bbfd9465-x28lh"]
Oct 30 21:27:55.554480 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.554231    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-9p8mb"]
Oct 30 21:27:55.554480 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.554340    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-86f87fb58b-5clbv"
Oct 30 21:27:55.558464 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.558429    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-hzvrj"]
Oct 30 21:27:55.562039 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.562018    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-tdmc8"]
Oct 30 21:27:55.562157 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.562059    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-sbhch"]
Oct 30 21:27:55.562324 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.562170    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hzvrj"
Oct 30 21:27:55.566953 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.566933    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-sklxd"]
Oct 30 21:27:55.567068 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.566962    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-9r5sb"]
Oct 30 21:27:55.567068 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.566976    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-9qppl"]
Oct 30 21:27:55.567068 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.567046    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-sbhch"
Oct 30 21:27:55.576326 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.576298    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-lwm89"]
Oct 30 21:27:55.581239 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.581179    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-kdrrd"]
Oct 30 21:27:55.583024 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.583002    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-76679b6db7-ltmt6"]
Oct 30 21:27:55.583895 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.583871    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-86f87fb58b-5clbv"]
Oct 30 21:27:55.587004 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.586988    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-lbjhv"]
Oct 30 21:27:55.587548 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.587532    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hzvrj"]
Oct 30 21:27:55.588400 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.588382    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-sbhch"]
Oct 30 21:27:55.609260 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.609233    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/67292bdc-7fe1-449d-af75-2baa90c16ea9-config\") pod \"console-operator-7cd56b8dbc-sklxd\" (UID: \"67292bdc-7fe1-449d-af75-2baa90c16ea9\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-sklxd"
Oct 30 21:27:55.609391 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.609264    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffq8c\" (UniqueName: \"kubernetes.io/projected/fb314739-c678-4da8-9d54-147dc0612ee2-kube-api-access-ffq8c\") pod \"kas-connection-checker-685895c7c9-l4dbk\" (UID: \"fb314739-c678-4da8-9d54-147dc0612ee2\") " pod="kube-system/kas-connection-checker-685895c7c9-l4dbk"
Oct 30 21:27:55.609391 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.609285    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pls28\" (UniqueName: \"kubernetes.io/projected/3c835b41-25cb-4f0c-8e7c-4ac9e2fee318-kube-api-access-pls28\") pod \"kas-connection-checker-685895c7c9-9mt44\" (UID: \"3c835b41-25cb-4f0c-8e7c-4ac9e2fee318\") " pod="kube-system/kas-connection-checker-685895c7c9-9mt44"
Oct 30 21:27:55.609391 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.609302    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzmzn\" (UniqueName: \"kubernetes.io/projected/e9d2f04b-bb7d-422f-b497-5fa61580f95a-kube-api-access-bzmzn\") pod \"volume-data-source-validator-58c9f5f46d-lbjhv\" (UID: \"e9d2f04b-bb7d-422f-b497-5fa61580f95a\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-lbjhv"
Oct 30 21:27:55.609391 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.609320    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/039cf439-d12d-450a-80c1-eb3efeb50e8a-installation-pull-secrets\") pod \"image-registry-76679b6db7-ltmt6\" (UID: \"039cf439-d12d-450a-80c1-eb3efeb50e8a\") " pod="openshift-image-registry/image-registry-76679b6db7-ltmt6"
Oct 30 21:27:55.609553 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.609422    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b4f9524-db75-4206-aa76-04400d95706f-metrics-certs\") pod \"router-default-5bbfd9465-x28lh\" (UID: \"9b4f9524-db75-4206-aa76-04400d95706f\") " pod="openshift-ingress/router-default-5bbfd9465-x28lh"
Oct 30 21:27:55.609553 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.609519    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d6ca21b9-20a4-4887-9d08-19d9c491a09e-tmp\") pod \"insights-operator-7c4986d449-7cgk9\" (UID: \"d6ca21b9-20a4-4887-9d08-19d9c491a09e\") " pod="openshift-insights/insights-operator-7c4986d449-7cgk9"
Oct 30 21:27:55.609651 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.609556    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/90937bc9-0051-407d-9c82-014f0a9c65d4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-kdrrd\" (UID: \"90937bc9-0051-407d-9c82-014f0a9c65d4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-kdrrd"
Oct 30 21:27:55.609651 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.609582    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/753d4639-09e6-4850-9f4d-ed39b0406439-nginx-conf\") pod \"networking-console-plugin-86c478bb88-lwm89\" (UID: \"753d4639-09e6-4850-9f4d-ed39b0406439\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-lwm89"
Oct 30 21:27:55.609651 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.609637    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/039cf439-d12d-450a-80c1-eb3efeb50e8a-registry-tls\") pod \"image-registry-76679b6db7-ltmt6\" (UID: \"039cf439-d12d-450a-80c1-eb3efeb50e8a\") " pod="openshift-image-registry/image-registry-76679b6db7-ltmt6"
Oct 30 21:27:55.609767 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.609741    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfrc4\" (UniqueName: \"kubernetes.io/projected/9b4f9524-db75-4206-aa76-04400d95706f-kube-api-access-wfrc4\") pod \"router-default-5bbfd9465-x28lh\" (UID: \"9b4f9524-db75-4206-aa76-04400d95706f\") " pod="openshift-ingress/router-default-5bbfd9465-x28lh"
Oct 30 21:27:55.609867 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.609795    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jsdkt\" (UniqueName: \"kubernetes.io/projected/8873f9b9-2ac5-409c-9299-fa024e00189c-kube-api-access-jsdkt\") pod \"service-ca-operator-544d749c8c-9r5sb\" (UID: \"8873f9b9-2ac5-409c-9299-fa024e00189c\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-9r5sb"
Oct 30 21:27:55.609867 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.609823    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8bb24a0d-3af0-4a05-bba4-4f9630b04bf3-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-9p8mb\" (UID: \"8bb24a0d-3af0-4a05-bba4-4f9630b04bf3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-9p8mb"
Oct 30 21:27:55.609867 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.609855    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pklf4\" (UniqueName: \"kubernetes.io/projected/d6ca21b9-20a4-4887-9d08-19d9c491a09e-kube-api-access-pklf4\") pod \"insights-operator-7c4986d449-7cgk9\" (UID: \"d6ca21b9-20a4-4887-9d08-19d9c491a09e\") " pod="openshift-insights/insights-operator-7c4986d449-7cgk9"
Oct 30 21:27:55.609983 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.609892    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/d6ca21b9-20a4-4887-9d08-19d9c491a09e-snapshots\") pod \"insights-operator-7c4986d449-7cgk9\" (UID: \"d6ca21b9-20a4-4887-9d08-19d9c491a09e\") " pod="openshift-insights/insights-operator-7c4986d449-7cgk9"
Oct 30 21:27:55.609983 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.609917    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56qdr\" (UniqueName: \"kubernetes.io/projected/90937bc9-0051-407d-9c82-014f0a9c65d4-kube-api-access-56qdr\") pod \"cluster-monitoring-operator-884576c57-kdrrd\" (UID: \"90937bc9-0051-407d-9c82-014f0a9c65d4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-kdrrd"
Oct 30 21:27:55.610073 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.610050    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6ca21b9-20a4-4887-9d08-19d9c491a09e-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-7cgk9\" (UID: \"d6ca21b9-20a4-4887-9d08-19d9c491a09e\") " pod="openshift-insights/insights-operator-7c4986d449-7cgk9"
Oct 30 21:27:55.610110 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.610086    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8873f9b9-2ac5-409c-9299-fa024e00189c-serving-cert\") pod \"service-ca-operator-544d749c8c-9r5sb\" (UID: \"8873f9b9-2ac5-409c-9299-fa024e00189c\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-9r5sb"
Oct 30 21:27:55.610150 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.610114    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6ca21b9-20a4-4887-9d08-19d9c491a09e-service-ca-bundle\") pod \"insights-operator-7c4986d449-7cgk9\" (UID: \"d6ca21b9-20a4-4887-9d08-19d9c491a09e\") " pod="openshift-insights/insights-operator-7c4986d449-7cgk9"
Oct 30 21:27:55.610196 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.610156    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/9b4f9524-db75-4206-aa76-04400d95706f-stats-auth\") pod \"router-default-5bbfd9465-x28lh\" (UID: \"9b4f9524-db75-4206-aa76-04400d95706f\") " pod="openshift-ingress/router-default-5bbfd9465-x28lh"
Oct 30 21:27:55.610196 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.610181    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/67292bdc-7fe1-449d-af75-2baa90c16ea9-trusted-ca\") pod \"console-operator-7cd56b8dbc-sklxd\" (UID: \"67292bdc-7fe1-449d-af75-2baa90c16ea9\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-sklxd"
Oct 30 21:27:55.610294 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.610205    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/039cf439-d12d-450a-80c1-eb3efeb50e8a-bound-sa-token\") pod \"image-registry-76679b6db7-ltmt6\" (UID: \"039cf439-d12d-450a-80c1-eb3efeb50e8a\") " pod="openshift-image-registry/image-registry-76679b6db7-ltmt6"
Oct 30 21:27:55.610344 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.610295    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/90937bc9-0051-407d-9c82-014f0a9c65d4-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-kdrrd\" (UID: \"90937bc9-0051-407d-9c82-014f0a9c65d4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-kdrrd"
Oct 30 21:27:55.610344 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.610331    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/039cf439-d12d-450a-80c1-eb3efeb50e8a-image-registry-private-configuration\") pod \"image-registry-76679b6db7-ltmt6\" (UID: \"039cf439-d12d-450a-80c1-eb3efeb50e8a\") " pod="openshift-image-registry/image-registry-76679b6db7-ltmt6"
Oct 30 21:27:55.610499 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.610480    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/039cf439-d12d-450a-80c1-eb3efeb50e8a-trusted-ca\") pod \"image-registry-76679b6db7-ltmt6\" (UID: \"039cf439-d12d-450a-80c1-eb3efeb50e8a\") " pod="openshift-image-registry/image-registry-76679b6db7-ltmt6"
Oct 30 21:27:55.610556 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.610543    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcgbg\" (UniqueName: \"kubernetes.io/projected/bba1a087-6e04-4db8-ba82-7260a88e2843-kube-api-access-qcgbg\") pod \"network-check-source-6f78d59f99-tdmc8\" (UID: \"bba1a087-6e04-4db8-ba82-7260a88e2843\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-tdmc8"
Oct 30 21:27:55.610593 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.610581    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8km8\" (UniqueName: \"kubernetes.io/projected/67292bdc-7fe1-449d-af75-2baa90c16ea9-kube-api-access-m8km8\") pod \"console-operator-7cd56b8dbc-sklxd\" (UID: \"67292bdc-7fe1-449d-af75-2baa90c16ea9\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-sklxd"
Oct 30 21:27:55.610631 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.610614    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/039cf439-d12d-450a-80c1-eb3efeb50e8a-ca-trust-extracted\") pod \"image-registry-76679b6db7-ltmt6\" (UID: \"039cf439-d12d-450a-80c1-eb3efeb50e8a\") " pod="openshift-image-registry/image-registry-76679b6db7-ltmt6"
Oct 30 21:27:55.610695 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.610677    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/9b4f9524-db75-4206-aa76-04400d95706f-default-certificate\") pod \"router-default-5bbfd9465-x28lh\" (UID: \"9b4f9524-db75-4206-aa76-04400d95706f\") " pod="openshift-ingress/router-default-5bbfd9465-x28lh"
Oct 30 21:27:55.610730 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.610711    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrrdm\" (UniqueName: \"kubernetes.io/projected/039cf439-d12d-450a-80c1-eb3efeb50e8a-kube-api-access-zrrdm\") pod \"image-registry-76679b6db7-ltmt6\" (UID: \"039cf439-d12d-450a-80c1-eb3efeb50e8a\") " pod="openshift-image-registry/image-registry-76679b6db7-ltmt6"
Oct 30 21:27:55.610778 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.610747    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8bb24a0d-3af0-4a05-bba4-4f9630b04bf3-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-9p8mb\" (UID: \"8bb24a0d-3af0-4a05-bba4-4f9630b04bf3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-9p8mb"
Oct 30 21:27:55.610827 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.610814    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wp72\" (UniqueName: \"kubernetes.io/projected/0d83c70b-fab8-4872-969f-11f74c0ddea0-kube-api-access-9wp72\") pod \"kas-connection-checker-685895c7c9-zvqvt\" (UID: \"0d83c70b-fab8-4872-969f-11f74c0ddea0\") " pod="kube-system/kas-connection-checker-685895c7c9-zvqvt"
Oct 30 21:27:55.610863 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.610841    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b4f9524-db75-4206-aa76-04400d95706f-service-ca-bundle\") pod \"router-default-5bbfd9465-x28lh\" (UID: \"9b4f9524-db75-4206-aa76-04400d95706f\") " pod="openshift-ingress/router-default-5bbfd9465-x28lh"
Oct 30 21:27:55.610902 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.610866    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spppx\" (UniqueName: \"kubernetes.io/projected/8bb24a0d-3af0-4a05-bba4-4f9630b04bf3-kube-api-access-spppx\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-9p8mb\" (UID: \"8bb24a0d-3af0-4a05-bba4-4f9630b04bf3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-9p8mb"
Oct 30 21:27:55.610941 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.610921    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/67292bdc-7fe1-449d-af75-2baa90c16ea9-serving-cert\") pod \"console-operator-7cd56b8dbc-sklxd\" (UID: \"67292bdc-7fe1-449d-af75-2baa90c16ea9\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-sklxd"
Oct 30 21:27:55.610981 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.610957    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/039cf439-d12d-450a-80c1-eb3efeb50e8a-registry-certificates\") pod \"image-registry-76679b6db7-ltmt6\" (UID: \"039cf439-d12d-450a-80c1-eb3efeb50e8a\") " pod="openshift-image-registry/image-registry-76679b6db7-ltmt6"
Oct 30 21:27:55.611044 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.611026    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8873f9b9-2ac5-409c-9299-fa024e00189c-config\") pod \"service-ca-operator-544d749c8c-9r5sb\" (UID: \"8873f9b9-2ac5-409c-9299-fa024e00189c\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-9r5sb"
Oct 30 21:27:55.611106 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.611084    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d6ca21b9-20a4-4887-9d08-19d9c491a09e-serving-cert\") pod \"insights-operator-7c4986d449-7cgk9\" (UID: \"d6ca21b9-20a4-4887-9d08-19d9c491a09e\") " pod="openshift-insights/insights-operator-7c4986d449-7cgk9"
Oct 30 21:27:55.611169 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.611126    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/753d4639-09e6-4850-9f4d-ed39b0406439-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-lwm89\" (UID: \"753d4639-09e6-4850-9f4d-ed39b0406439\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-lwm89"
Oct 30 21:27:55.617586 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.617561    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Oct 30 21:27:55.617822 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.617799    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Oct 30 21:27:55.617822 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.617803    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 21:27:55.618059 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.618044    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 21:27:55.621671 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.621652    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Oct 30 21:27:55.621851 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.621662    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 21:27:55.621941 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.621853    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 21:27:55.621941 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.621683    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:27:55.621941 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.621695    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Oct 30 21:27:55.621941 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.621707    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Oct 30 21:27:55.621941 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.621711    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:27:55.621941 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.621900    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:27:55.621941 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.621720    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:27:55.621941 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.621752    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Oct 30 21:27:55.621941 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.621786    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Oct 30 21:27:55.621941 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.621662    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 21:27:55.622347 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.621951    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Oct 30 21:27:55.622347 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.621994    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Oct 30 21:27:55.622347 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.621955    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Oct 30 21:27:55.622347 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.622025    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Oct 30 21:27:55.622347 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.622210    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 21:27:55.622347 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.622224    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\""
Oct 30 21:27:55.622347 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.622245    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Oct 30 21:27:55.622347 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.622262    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Oct 30 21:27:55.622347 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.622264    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Oct 30 21:27:55.622347 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.622276    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Oct 30 21:27:55.622650 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.622401    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 21:27:55.623286 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.623267    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Oct 30 21:27:55.623415 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.623303    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:27:55.627715 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.627695    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Oct 30 21:27:55.627886 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.627871    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Oct 30 21:27:55.628109 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.627816    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 21:27:55.628109 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.627891    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 21:27:55.628235 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.627936    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 21:27:55.628235 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.627954    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 21:27:55.628235 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.627764    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Oct 30 21:27:55.628235 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.628025    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Oct 30 21:27:55.628235 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.628047    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Oct 30 21:27:55.628235 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.628059    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-bvjbv\""
Oct 30 21:27:55.628235 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.628067    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Oct 30 21:27:55.628235 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.628075    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Oct 30 21:27:55.628564 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.628253    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-knzjw\""
Oct 30 21:27:55.631246 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.631190    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-qcj4q\""
Oct 30 21:27:55.631246 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.631190    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-vnct9\""
Oct 30 21:27:55.631246 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.631196    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Oct 30 21:27:55.631246 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.631234    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-r96r5\""
Oct 30 21:27:55.631246 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.631239    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Oct 30 21:27:55.631606 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.631252    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-dockercfg-7m5rm\""
Oct 30 21:27:55.631606 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.631352    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Oct 30 21:27:55.632577 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.632557    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Oct 30 21:27:55.711618 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.711581    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/9b4f9524-db75-4206-aa76-04400d95706f-stats-auth\") pod \"router-default-5bbfd9465-x28lh\" (UID: \"9b4f9524-db75-4206-aa76-04400d95706f\") " pod="openshift-ingress/router-default-5bbfd9465-x28lh"
Oct 30 21:27:55.711618 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.711615    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/67292bdc-7fe1-449d-af75-2baa90c16ea9-trusted-ca\") pod \"console-operator-7cd56b8dbc-sklxd\" (UID: \"67292bdc-7fe1-449d-af75-2baa90c16ea9\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-sklxd"
Oct 30 21:27:55.712163 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.711635    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/039cf439-d12d-450a-80c1-eb3efeb50e8a-bound-sa-token\") pod \"image-registry-76679b6db7-ltmt6\" (UID: \"039cf439-d12d-450a-80c1-eb3efeb50e8a\") " pod="openshift-image-registry/image-registry-76679b6db7-ltmt6"
Oct 30 21:27:55.712163 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.711666    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/94ee3757-5440-42e1-8617-2a68744ef3f4-registry-certificates\") pod \"image-registry-86f87fb58b-5clbv\" (UID: \"94ee3757-5440-42e1-8617-2a68744ef3f4\") " pod="openshift-image-registry/image-registry-86f87fb58b-5clbv"
Oct 30 21:27:55.712163 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.711690    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6lpv\" (UniqueName: \"kubernetes.io/projected/94ee3757-5440-42e1-8617-2a68744ef3f4-kube-api-access-t6lpv\") pod \"image-registry-86f87fb58b-5clbv\" (UID: \"94ee3757-5440-42e1-8617-2a68744ef3f4\") " pod="openshift-image-registry/image-registry-86f87fb58b-5clbv"
Oct 30 21:27:55.712163 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.711712    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08498890-fb63-4aea-9cf7-a2ade5397df2-tmp-dir\") pod \"dns-default-sbhch\" (UID: \"08498890-fb63-4aea-9cf7-a2ade5397df2\") " pod="openshift-dns/dns-default-sbhch"
Oct 30 21:27:55.712163 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.711736    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdf4s\" (UniqueName: \"kubernetes.io/projected/08498890-fb63-4aea-9cf7-a2ade5397df2-kube-api-access-hdf4s\") pod \"dns-default-sbhch\" (UID: \"08498890-fb63-4aea-9cf7-a2ade5397df2\") " pod="openshift-dns/dns-default-sbhch"
Oct 30 21:27:55.712163 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.711807    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/94ee3757-5440-42e1-8617-2a68744ef3f4-trusted-ca\") pod \"image-registry-86f87fb58b-5clbv\" (UID: \"94ee3757-5440-42e1-8617-2a68744ef3f4\") " pod="openshift-image-registry/image-registry-86f87fb58b-5clbv"
Oct 30 21:27:55.712163 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.711846    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/90937bc9-0051-407d-9c82-014f0a9c65d4-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-kdrrd\" (UID: \"90937bc9-0051-407d-9c82-014f0a9c65d4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-kdrrd"
Oct 30 21:27:55.712163 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.711889    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/039cf439-d12d-450a-80c1-eb3efeb50e8a-image-registry-private-configuration\") pod \"image-registry-76679b6db7-ltmt6\" (UID: \"039cf439-d12d-450a-80c1-eb3efeb50e8a\") " pod="openshift-image-registry/image-registry-76679b6db7-ltmt6"
Oct 30 21:27:55.712163 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.711987    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/039cf439-d12d-450a-80c1-eb3efeb50e8a-trusted-ca\") pod \"image-registry-76679b6db7-ltmt6\" (UID: \"039cf439-d12d-450a-80c1-eb3efeb50e8a\") " pod="openshift-image-registry/image-registry-76679b6db7-ltmt6"
Oct 30 21:27:55.712629 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.712228    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcgbg\" (UniqueName: \"kubernetes.io/projected/bba1a087-6e04-4db8-ba82-7260a88e2843-kube-api-access-qcgbg\") pod \"network-check-source-6f78d59f99-tdmc8\" (UID: \"bba1a087-6e04-4db8-ba82-7260a88e2843\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-tdmc8"
Oct 30 21:27:55.712629 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.712533    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8km8\" (UniqueName: \"kubernetes.io/projected/67292bdc-7fe1-449d-af75-2baa90c16ea9-kube-api-access-m8km8\") pod \"console-operator-7cd56b8dbc-sklxd\" (UID: \"67292bdc-7fe1-449d-af75-2baa90c16ea9\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-sklxd"
Oct 30 21:27:55.712629 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.712567    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/039cf439-d12d-450a-80c1-eb3efeb50e8a-ca-trust-extracted\") pod \"image-registry-76679b6db7-ltmt6\" (UID: \"039cf439-d12d-450a-80c1-eb3efeb50e8a\") " pod="openshift-image-registry/image-registry-76679b6db7-ltmt6"
Oct 30 21:27:55.712629 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.712602    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/9b4f9524-db75-4206-aa76-04400d95706f-default-certificate\") pod \"router-default-5bbfd9465-x28lh\" (UID: \"9b4f9524-db75-4206-aa76-04400d95706f\") " pod="openshift-ingress/router-default-5bbfd9465-x28lh"
Oct 30 21:27:55.712629 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.712628    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrrdm\" (UniqueName: \"kubernetes.io/projected/039cf439-d12d-450a-80c1-eb3efeb50e8a-kube-api-access-zrrdm\") pod \"image-registry-76679b6db7-ltmt6\" (UID: \"039cf439-d12d-450a-80c1-eb3efeb50e8a\") " pod="openshift-image-registry/image-registry-76679b6db7-ltmt6"
Oct 30 21:27:55.712869 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.712643    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/67292bdc-7fe1-449d-af75-2baa90c16ea9-trusted-ca\") pod \"console-operator-7cd56b8dbc-sklxd\" (UID: \"67292bdc-7fe1-449d-af75-2baa90c16ea9\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-sklxd"
Oct 30 21:27:55.712869 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.712661    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8bb24a0d-3af0-4a05-bba4-4f9630b04bf3-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-9p8mb\" (UID: \"8bb24a0d-3af0-4a05-bba4-4f9630b04bf3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-9p8mb"
Oct 30 21:27:55.712869 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.712686    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/90937bc9-0051-407d-9c82-014f0a9c65d4-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-kdrrd\" (UID: \"90937bc9-0051-407d-9c82-014f0a9c65d4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-kdrrd"
Oct 30 21:27:55.712869 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.712709    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wp72\" (UniqueName: \"kubernetes.io/projected/0d83c70b-fab8-4872-969f-11f74c0ddea0-kube-api-access-9wp72\") pod \"kas-connection-checker-685895c7c9-zvqvt\" (UID: \"0d83c70b-fab8-4872-969f-11f74c0ddea0\") " pod="kube-system/kas-connection-checker-685895c7c9-zvqvt"
Oct 30 21:27:55.712869 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.712740    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/94ee3757-5440-42e1-8617-2a68744ef3f4-ca-trust-extracted\") pod \"image-registry-86f87fb58b-5clbv\" (UID: \"94ee3757-5440-42e1-8617-2a68744ef3f4\") " pod="openshift-image-registry/image-registry-86f87fb58b-5clbv"
Oct 30 21:27:55.712869 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.712769    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b4f9524-db75-4206-aa76-04400d95706f-service-ca-bundle\") pod \"router-default-5bbfd9465-x28lh\" (UID: \"9b4f9524-db75-4206-aa76-04400d95706f\") " pod="openshift-ingress/router-default-5bbfd9465-x28lh"
Oct 30 21:27:55.712869 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.712797    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spppx\" (UniqueName: \"kubernetes.io/projected/8bb24a0d-3af0-4a05-bba4-4f9630b04bf3-kube-api-access-spppx\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-9p8mb\" (UID: \"8bb24a0d-3af0-4a05-bba4-4f9630b04bf3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-9p8mb"
Oct 30 21:27:55.712869 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.712823    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/67292bdc-7fe1-449d-af75-2baa90c16ea9-serving-cert\") pod \"console-operator-7cd56b8dbc-sklxd\" (UID: \"67292bdc-7fe1-449d-af75-2baa90c16ea9\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-sklxd"
Oct 30 21:27:55.712869 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.712848    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/039cf439-d12d-450a-80c1-eb3efeb50e8a-registry-certificates\") pod \"image-registry-76679b6db7-ltmt6\" (UID: \"039cf439-d12d-450a-80c1-eb3efeb50e8a\") " pod="openshift-image-registry/image-registry-76679b6db7-ltmt6"
Oct 30 21:27:55.713279 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.712892    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8873f9b9-2ac5-409c-9299-fa024e00189c-config\") pod \"service-ca-operator-544d749c8c-9r5sb\" (UID: \"8873f9b9-2ac5-409c-9299-fa024e00189c\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-9r5sb"
Oct 30 21:27:55.713279 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.712931    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d6ca21b9-20a4-4887-9d08-19d9c491a09e-serving-cert\") pod \"insights-operator-7c4986d449-7cgk9\" (UID: \"d6ca21b9-20a4-4887-9d08-19d9c491a09e\") " pod="openshift-insights/insights-operator-7c4986d449-7cgk9"
Oct 30 21:27:55.713279 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.712962    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/753d4639-09e6-4850-9f4d-ed39b0406439-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-lwm89\" (UID: \"753d4639-09e6-4850-9f4d-ed39b0406439\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-lwm89"
Oct 30 21:27:55.713279 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.712990    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfphc\" (UniqueName: \"kubernetes.io/projected/3175ff47-1038-4139-9c0a-87062bb38dc2-kube-api-access-zfphc\") pod \"ingress-canary-hzvrj\" (UID: \"3175ff47-1038-4139-9c0a-87062bb38dc2\") " pod="openshift-ingress-canary/ingress-canary-hzvrj"
Oct 30 21:27:55.713279 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.713013    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08498890-fb63-4aea-9cf7-a2ade5397df2-metrics-tls\") pod \"dns-default-sbhch\" (UID: \"08498890-fb63-4aea-9cf7-a2ade5397df2\") " pod="openshift-dns/dns-default-sbhch"
Oct 30 21:27:55.713279 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.713001    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/039cf439-d12d-450a-80c1-eb3efeb50e8a-ca-trust-extracted\") pod \"image-registry-76679b6db7-ltmt6\" (UID: \"039cf439-d12d-450a-80c1-eb3efeb50e8a\") " pod="openshift-image-registry/image-registry-76679b6db7-ltmt6"
Oct 30 21:27:55.713279 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.713048    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqwd9\" (UniqueName: \"kubernetes.io/projected/9f54b2c6-b1f9-42da-abdc-275fc8827b71-kube-api-access-jqwd9\") pod \"cluster-samples-operator-6ddb6c8b94-9qppl\" (UID: \"9f54b2c6-b1f9-42da-abdc-275fc8827b71\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-9qppl"
Oct 30 21:27:55.713279 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.713079    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/67292bdc-7fe1-449d-af75-2baa90c16ea9-config\") pod \"console-operator-7cd56b8dbc-sklxd\" (UID: \"67292bdc-7fe1-449d-af75-2baa90c16ea9\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-sklxd"
Oct 30 21:27:55.713279 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.713102    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/08498890-fb63-4aea-9cf7-a2ade5397df2-config-volume\") pod \"dns-default-sbhch\" (UID: \"08498890-fb63-4aea-9cf7-a2ade5397df2\") " pod="openshift-dns/dns-default-sbhch"
Oct 30 21:27:55.713279 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.713133    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffq8c\" (UniqueName: \"kubernetes.io/projected/fb314739-c678-4da8-9d54-147dc0612ee2-kube-api-access-ffq8c\") pod \"kas-connection-checker-685895c7c9-l4dbk\" (UID: \"fb314739-c678-4da8-9d54-147dc0612ee2\") " pod="kube-system/kas-connection-checker-685895c7c9-l4dbk"
Oct 30 21:27:55.713279 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.713159    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pls28\" (UniqueName: \"kubernetes.io/projected/3c835b41-25cb-4f0c-8e7c-4ac9e2fee318-kube-api-access-pls28\") pod \"kas-connection-checker-685895c7c9-9mt44\" (UID: \"3c835b41-25cb-4f0c-8e7c-4ac9e2fee318\") " pod="kube-system/kas-connection-checker-685895c7c9-9mt44"
Oct 30 21:27:55.713279 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.713212    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzmzn\" (UniqueName: \"kubernetes.io/projected/e9d2f04b-bb7d-422f-b497-5fa61580f95a-kube-api-access-bzmzn\") pod \"volume-data-source-validator-58c9f5f46d-lbjhv\" (UID: \"e9d2f04b-bb7d-422f-b497-5fa61580f95a\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-lbjhv"
Oct 30 21:27:55.713279 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.713229    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/039cf439-d12d-450a-80c1-eb3efeb50e8a-trusted-ca\") pod \"image-registry-76679b6db7-ltmt6\" (UID: \"039cf439-d12d-450a-80c1-eb3efeb50e8a\") " pod="openshift-image-registry/image-registry-76679b6db7-ltmt6"
Oct 30 21:27:55.713279 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.713237    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/039cf439-d12d-450a-80c1-eb3efeb50e8a-installation-pull-secrets\") pod \"image-registry-76679b6db7-ltmt6\" (UID: \"039cf439-d12d-450a-80c1-eb3efeb50e8a\") " pod="openshift-image-registry/image-registry-76679b6db7-ltmt6"
Oct 30 21:27:55.713965 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.713284    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b4f9524-db75-4206-aa76-04400d95706f-metrics-certs\") pod \"router-default-5bbfd9465-x28lh\" (UID: \"9b4f9524-db75-4206-aa76-04400d95706f\") " pod="openshift-ingress/router-default-5bbfd9465-x28lh"
Oct 30 21:27:55.713965 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.713334    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/94ee3757-5440-42e1-8617-2a68744ef3f4-installation-pull-secrets\") pod \"image-registry-86f87fb58b-5clbv\" (UID: \"94ee3757-5440-42e1-8617-2a68744ef3f4\") " pod="openshift-image-registry/image-registry-86f87fb58b-5clbv"
Oct 30 21:27:55.713965 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.713391    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d6ca21b9-20a4-4887-9d08-19d9c491a09e-tmp\") pod \"insights-operator-7c4986d449-7cgk9\" (UID: \"d6ca21b9-20a4-4887-9d08-19d9c491a09e\") " pod="openshift-insights/insights-operator-7c4986d449-7cgk9"
Oct 30 21:27:55.713965 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.713668    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/90937bc9-0051-407d-9c82-014f0a9c65d4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-kdrrd\" (UID: \"90937bc9-0051-407d-9c82-014f0a9c65d4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-kdrrd"
Oct 30 21:27:55.713965 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.713700    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/753d4639-09e6-4850-9f4d-ed39b0406439-nginx-conf\") pod \"networking-console-plugin-86c478bb88-lwm89\" (UID: \"753d4639-09e6-4850-9f4d-ed39b0406439\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-lwm89"
Oct 30 21:27:55.713965 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.713724    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/039cf439-d12d-450a-80c1-eb3efeb50e8a-registry-tls\") pod \"image-registry-76679b6db7-ltmt6\" (UID: \"039cf439-d12d-450a-80c1-eb3efeb50e8a\") " pod="openshift-image-registry/image-registry-76679b6db7-ltmt6"
Oct 30 21:27:55.713965 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.713751    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3175ff47-1038-4139-9c0a-87062bb38dc2-cert\") pod \"ingress-canary-hzvrj\" (UID: \"3175ff47-1038-4139-9c0a-87062bb38dc2\") " pod="openshift-ingress-canary/ingress-canary-hzvrj"
Oct 30 21:27:55.713965 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.713781    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfrc4\" (UniqueName: \"kubernetes.io/projected/9b4f9524-db75-4206-aa76-04400d95706f-kube-api-access-wfrc4\") pod \"router-default-5bbfd9465-x28lh\" (UID: \"9b4f9524-db75-4206-aa76-04400d95706f\") " pod="openshift-ingress/router-default-5bbfd9465-x28lh"
Oct 30 21:27:55.713965 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.713897    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/67292bdc-7fe1-449d-af75-2baa90c16ea9-config\") pod \"console-operator-7cd56b8dbc-sklxd\" (UID: \"67292bdc-7fe1-449d-af75-2baa90c16ea9\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-sklxd"
Oct 30 21:27:55.714390 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:55.714017    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9b4f9524-db75-4206-aa76-04400d95706f-service-ca-bundle podName:9b4f9524-db75-4206-aa76-04400d95706f nodeName:}" failed. No retries permitted until 2026-10-30 21:27:56.21399493 +0000 UTC m=+31.095057879 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9b4f9524-db75-4206-aa76-04400d95706f-service-ca-bundle") pod "router-default-5bbfd9465-x28lh" (UID: "9b4f9524-db75-4206-aa76-04400d95706f") : configmap references non-existent config key: service-ca.crt
Oct 30 21:27:55.714390 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.714079    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/039cf439-d12d-450a-80c1-eb3efeb50e8a-registry-certificates\") pod \"image-registry-76679b6db7-ltmt6\" (UID: \"039cf439-d12d-450a-80c1-eb3efeb50e8a\") " pod="openshift-image-registry/image-registry-76679b6db7-ltmt6"
Oct 30 21:27:55.714390 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.714132    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d6ca21b9-20a4-4887-9d08-19d9c491a09e-tmp\") pod \"insights-operator-7c4986d449-7cgk9\" (UID: \"d6ca21b9-20a4-4887-9d08-19d9c491a09e\") " pod="openshift-insights/insights-operator-7c4986d449-7cgk9"
Oct 30 21:27:55.714390 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:55.714239    2573 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 21:27:55.714390 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:55.714296    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/753d4639-09e6-4850-9f4d-ed39b0406439-networking-console-plugin-cert podName:753d4639-09e6-4850-9f4d-ed39b0406439 nodeName:}" failed. No retries permitted until 2026-10-30 21:27:56.214272015 +0000 UTC m=+31.095334960 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/753d4639-09e6-4850-9f4d-ed39b0406439-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-lwm89" (UID: "753d4639-09e6-4850-9f4d-ed39b0406439") : secret "networking-console-plugin-cert" not found
Oct 30 21:27:55.714662 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:55.714536    2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 21:27:55.714662 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:55.714550    2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76679b6db7-ltmt6: secret "image-registry-tls" not found
Oct 30 21:27:55.714662 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:55.714644    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/039cf439-d12d-450a-80c1-eb3efeb50e8a-registry-tls podName:039cf439-d12d-450a-80c1-eb3efeb50e8a nodeName:}" failed. No retries permitted until 2026-10-30 21:27:56.214595769 +0000 UTC m=+31.095658716 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/039cf439-d12d-450a-80c1-eb3efeb50e8a-registry-tls") pod "image-registry-76679b6db7-ltmt6" (UID: "039cf439-d12d-450a-80c1-eb3efeb50e8a") : secret "image-registry-tls" not found
Oct 30 21:27:55.714864 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.714838    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8873f9b9-2ac5-409c-9299-fa024e00189c-config\") pod \"service-ca-operator-544d749c8c-9r5sb\" (UID: \"8873f9b9-2ac5-409c-9299-fa024e00189c\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-9r5sb"
Oct 30 21:27:55.715860 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:55.715198    2573 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 21:27:55.715860 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:55.715253    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b4f9524-db75-4206-aa76-04400d95706f-metrics-certs podName:9b4f9524-db75-4206-aa76-04400d95706f nodeName:}" failed. No retries permitted until 2026-10-30 21:27:56.215237034 +0000 UTC m=+31.096299983 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9b4f9524-db75-4206-aa76-04400d95706f-metrics-certs") pod "router-default-5bbfd9465-x28lh" (UID: "9b4f9524-db75-4206-aa76-04400d95706f") : secret "router-metrics-certs-default" not found
Oct 30 21:27:55.715860 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:55.715758    2573 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 30 21:27:55.715860 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.715770    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/753d4639-09e6-4850-9f4d-ed39b0406439-nginx-conf\") pod \"networking-console-plugin-86c478bb88-lwm89\" (UID: \"753d4639-09e6-4850-9f4d-ed39b0406439\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-lwm89"
Oct 30 21:27:55.715860 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:55.715818    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90937bc9-0051-407d-9c82-014f0a9c65d4-cluster-monitoring-operator-tls podName:90937bc9-0051-407d-9c82-014f0a9c65d4 nodeName:}" failed. No retries permitted until 2026-10-30 21:27:56.215802967 +0000 UTC m=+31.096865916 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/90937bc9-0051-407d-9c82-014f0a9c65d4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-kdrrd" (UID: "90937bc9-0051-407d-9c82-014f0a9c65d4") : secret "cluster-monitoring-operator-tls" not found
Oct 30 21:27:55.716205 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.715872    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/94ee3757-5440-42e1-8617-2a68744ef3f4-image-registry-private-configuration\") pod \"image-registry-86f87fb58b-5clbv\" (UID: \"94ee3757-5440-42e1-8617-2a68744ef3f4\") " pod="openshift-image-registry/image-registry-86f87fb58b-5clbv"
Oct 30 21:27:55.716205 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.715918    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/94ee3757-5440-42e1-8617-2a68744ef3f4-registry-tls\") pod \"image-registry-86f87fb58b-5clbv\" (UID: \"94ee3757-5440-42e1-8617-2a68744ef3f4\") " pod="openshift-image-registry/image-registry-86f87fb58b-5clbv"
Oct 30 21:27:55.716205 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.715947    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/9f54b2c6-b1f9-42da-abdc-275fc8827b71-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-9qppl\" (UID: \"9f54b2c6-b1f9-42da-abdc-275fc8827b71\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-9qppl"
Oct 30 21:27:55.716205 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.715991    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jsdkt\" (UniqueName: \"kubernetes.io/projected/8873f9b9-2ac5-409c-9299-fa024e00189c-kube-api-access-jsdkt\") pod \"service-ca-operator-544d749c8c-9r5sb\" (UID: \"8873f9b9-2ac5-409c-9299-fa024e00189c\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-9r5sb"
Oct 30 21:27:55.716205 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.716024    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8bb24a0d-3af0-4a05-bba4-4f9630b04bf3-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-9p8mb\" (UID: \"8bb24a0d-3af0-4a05-bba4-4f9630b04bf3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-9p8mb"
Oct 30 21:27:55.717114 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.716447    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pklf4\" (UniqueName: \"kubernetes.io/projected/d6ca21b9-20a4-4887-9d08-19d9c491a09e-kube-api-access-pklf4\") pod \"insights-operator-7c4986d449-7cgk9\" (UID: \"d6ca21b9-20a4-4887-9d08-19d9c491a09e\") " pod="openshift-insights/insights-operator-7c4986d449-7cgk9"
Oct 30 21:27:55.717114 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.716699    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/d6ca21b9-20a4-4887-9d08-19d9c491a09e-snapshots\") pod \"insights-operator-7c4986d449-7cgk9\" (UID: \"d6ca21b9-20a4-4887-9d08-19d9c491a09e\") " pod="openshift-insights/insights-operator-7c4986d449-7cgk9"
Oct 30 21:27:55.717114 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.716754    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56qdr\" (UniqueName: \"kubernetes.io/projected/90937bc9-0051-407d-9c82-014f0a9c65d4-kube-api-access-56qdr\") pod \"cluster-monitoring-operator-884576c57-kdrrd\" (UID: \"90937bc9-0051-407d-9c82-014f0a9c65d4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-kdrrd"
Oct 30 21:27:55.717114 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.716783    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9f54b2c6-b1f9-42da-abdc-275fc8827b71-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-9qppl\" (UID: \"9f54b2c6-b1f9-42da-abdc-275fc8827b71\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-9qppl"
Oct 30 21:27:55.717114 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.716837    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6ca21b9-20a4-4887-9d08-19d9c491a09e-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-7cgk9\" (UID: \"d6ca21b9-20a4-4887-9d08-19d9c491a09e\") " pod="openshift-insights/insights-operator-7c4986d449-7cgk9"
Oct 30 21:27:55.717114 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.716853    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8bb24a0d-3af0-4a05-bba4-4f9630b04bf3-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-9p8mb\" (UID: \"8bb24a0d-3af0-4a05-bba4-4f9630b04bf3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-9p8mb"
Oct 30 21:27:55.717114 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.716887    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8bb24a0d-3af0-4a05-bba4-4f9630b04bf3-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-9p8mb\" (UID: \"8bb24a0d-3af0-4a05-bba4-4f9630b04bf3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-9p8mb"
Oct 30 21:27:55.717114 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.716906    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8873f9b9-2ac5-409c-9299-fa024e00189c-serving-cert\") pod \"service-ca-operator-544d749c8c-9r5sb\" (UID: \"8873f9b9-2ac5-409c-9299-fa024e00189c\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-9r5sb"
Oct 30 21:27:55.717114 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.716938    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6ca21b9-20a4-4887-9d08-19d9c491a09e-service-ca-bundle\") pod \"insights-operator-7c4986d449-7cgk9\" (UID: \"d6ca21b9-20a4-4887-9d08-19d9c491a09e\") " pod="openshift-insights/insights-operator-7c4986d449-7cgk9"
Oct 30 21:27:55.717114 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.716975    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/94ee3757-5440-42e1-8617-2a68744ef3f4-bound-sa-token\") pod \"image-registry-86f87fb58b-5clbv\" (UID: \"94ee3757-5440-42e1-8617-2a68744ef3f4\") " pod="openshift-image-registry/image-registry-86f87fb58b-5clbv"
Oct 30 21:27:55.717640 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.717412    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/9b4f9524-db75-4206-aa76-04400d95706f-default-certificate\") pod \"router-default-5bbfd9465-x28lh\" (UID: \"9b4f9524-db75-4206-aa76-04400d95706f\") " pod="openshift-ingress/router-default-5bbfd9465-x28lh"
Oct 30 21:27:55.718204 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.717756    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6ca21b9-20a4-4887-9d08-19d9c491a09e-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-7cgk9\" (UID: \"d6ca21b9-20a4-4887-9d08-19d9c491a09e\") " pod="openshift-insights/insights-operator-7c4986d449-7cgk9"
Oct 30 21:27:55.718204 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.717890    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/d6ca21b9-20a4-4887-9d08-19d9c491a09e-snapshots\") pod \"insights-operator-7c4986d449-7cgk9\" (UID: \"d6ca21b9-20a4-4887-9d08-19d9c491a09e\") " pod="openshift-insights/insights-operator-7c4986d449-7cgk9"
Oct 30 21:27:55.718204 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.718167    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6ca21b9-20a4-4887-9d08-19d9c491a09e-service-ca-bundle\") pod \"insights-operator-7c4986d449-7cgk9\" (UID: \"d6ca21b9-20a4-4887-9d08-19d9c491a09e\") " pod="openshift-insights/insights-operator-7c4986d449-7cgk9"
Oct 30 21:27:55.718438 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.718224    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/67292bdc-7fe1-449d-af75-2baa90c16ea9-serving-cert\") pod \"console-operator-7cd56b8dbc-sklxd\" (UID: \"67292bdc-7fe1-449d-af75-2baa90c16ea9\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-sklxd"
Oct 30 21:27:55.718438 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.718388    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d6ca21b9-20a4-4887-9d08-19d9c491a09e-serving-cert\") pod \"insights-operator-7c4986d449-7cgk9\" (UID: \"d6ca21b9-20a4-4887-9d08-19d9c491a09e\") " pod="openshift-insights/insights-operator-7c4986d449-7cgk9"
Oct 30 21:27:55.718565 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.718542    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/9b4f9524-db75-4206-aa76-04400d95706f-stats-auth\") pod \"router-default-5bbfd9465-x28lh\" (UID: \"9b4f9524-db75-4206-aa76-04400d95706f\") " pod="openshift-ingress/router-default-5bbfd9465-x28lh"
Oct 30 21:27:55.719285 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.719263    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/039cf439-d12d-450a-80c1-eb3efeb50e8a-installation-pull-secrets\") pod \"image-registry-76679b6db7-ltmt6\" (UID: \"039cf439-d12d-450a-80c1-eb3efeb50e8a\") " pod="openshift-image-registry/image-registry-76679b6db7-ltmt6"
Oct 30 21:27:55.720186 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.720157    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/039cf439-d12d-450a-80c1-eb3efeb50e8a-image-registry-private-configuration\") pod \"image-registry-76679b6db7-ltmt6\" (UID: \"039cf439-d12d-450a-80c1-eb3efeb50e8a\") " pod="openshift-image-registry/image-registry-76679b6db7-ltmt6"
Oct 30 21:27:55.720261 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.720167    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8873f9b9-2ac5-409c-9299-fa024e00189c-serving-cert\") pod \"service-ca-operator-544d749c8c-9r5sb\" (UID: \"8873f9b9-2ac5-409c-9299-fa024e00189c\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-9r5sb"
Oct 30 21:27:55.727320 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.727298    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/039cf439-d12d-450a-80c1-eb3efeb50e8a-bound-sa-token\") pod \"image-registry-76679b6db7-ltmt6\" (UID: \"039cf439-d12d-450a-80c1-eb3efeb50e8a\") " pod="openshift-image-registry/image-registry-76679b6db7-ltmt6"
Oct 30 21:27:55.728549 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.728528    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcgbg\" (UniqueName: \"kubernetes.io/projected/bba1a087-6e04-4db8-ba82-7260a88e2843-kube-api-access-qcgbg\") pod \"network-check-source-6f78d59f99-tdmc8\" (UID: \"bba1a087-6e04-4db8-ba82-7260a88e2843\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-tdmc8"
Oct 30 21:27:55.735314 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.735285    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wp72\" (UniqueName: \"kubernetes.io/projected/0d83c70b-fab8-4872-969f-11f74c0ddea0-kube-api-access-9wp72\") pod \"kas-connection-checker-685895c7c9-zvqvt\" (UID: \"0d83c70b-fab8-4872-969f-11f74c0ddea0\") " pod="kube-system/kas-connection-checker-685895c7c9-zvqvt"
Oct 30 21:27:55.735451 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.735387    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrrdm\" (UniqueName: \"kubernetes.io/projected/039cf439-d12d-450a-80c1-eb3efeb50e8a-kube-api-access-zrrdm\") pod \"image-registry-76679b6db7-ltmt6\" (UID: \"039cf439-d12d-450a-80c1-eb3efeb50e8a\") " pod="openshift-image-registry/image-registry-76679b6db7-ltmt6"
Oct 30 21:27:55.735709 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.735690    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfrc4\" (UniqueName: \"kubernetes.io/projected/9b4f9524-db75-4206-aa76-04400d95706f-kube-api-access-wfrc4\") pod \"router-default-5bbfd9465-x28lh\" (UID: \"9b4f9524-db75-4206-aa76-04400d95706f\") " pod="openshift-ingress/router-default-5bbfd9465-x28lh"
Oct 30 21:27:55.736399 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.736330    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pls28\" (UniqueName: \"kubernetes.io/projected/3c835b41-25cb-4f0c-8e7c-4ac9e2fee318-kube-api-access-pls28\") pod \"kas-connection-checker-685895c7c9-9mt44\" (UID: \"3c835b41-25cb-4f0c-8e7c-4ac9e2fee318\") " pod="kube-system/kas-connection-checker-685895c7c9-9mt44"
Oct 30 21:27:55.736631 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.736575    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-56qdr\" (UniqueName: \"kubernetes.io/projected/90937bc9-0051-407d-9c82-014f0a9c65d4-kube-api-access-56qdr\") pod \"cluster-monitoring-operator-884576c57-kdrrd\" (UID: \"90937bc9-0051-407d-9c82-014f0a9c65d4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-kdrrd"
Oct 30 21:27:55.737523 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.737499    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzmzn\" (UniqueName: \"kubernetes.io/projected/e9d2f04b-bb7d-422f-b497-5fa61580f95a-kube-api-access-bzmzn\") pod \"volume-data-source-validator-58c9f5f46d-lbjhv\" (UID: \"e9d2f04b-bb7d-422f-b497-5fa61580f95a\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-lbjhv"
Oct 30 21:27:55.737766 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.737711    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-spppx\" (UniqueName: \"kubernetes.io/projected/8bb24a0d-3af0-4a05-bba4-4f9630b04bf3-kube-api-access-spppx\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-9p8mb\" (UID: \"8bb24a0d-3af0-4a05-bba4-4f9630b04bf3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-9p8mb"
Oct 30 21:27:55.738689 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.738659    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pklf4\" (UniqueName: \"kubernetes.io/projected/d6ca21b9-20a4-4887-9d08-19d9c491a09e-kube-api-access-pklf4\") pod \"insights-operator-7c4986d449-7cgk9\" (UID: \"d6ca21b9-20a4-4887-9d08-19d9c491a09e\") " pod="openshift-insights/insights-operator-7c4986d449-7cgk9"
Oct 30 21:27:55.739325 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.739305    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffq8c\" (UniqueName: \"kubernetes.io/projected/fb314739-c678-4da8-9d54-147dc0612ee2-kube-api-access-ffq8c\") pod \"kas-connection-checker-685895c7c9-l4dbk\" (UID: \"fb314739-c678-4da8-9d54-147dc0612ee2\") " pod="kube-system/kas-connection-checker-685895c7c9-l4dbk"
Oct 30 21:27:55.739625 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.739609    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jsdkt\" (UniqueName: \"kubernetes.io/projected/8873f9b9-2ac5-409c-9299-fa024e00189c-kube-api-access-jsdkt\") pod \"service-ca-operator-544d749c8c-9r5sb\" (UID: \"8873f9b9-2ac5-409c-9299-fa024e00189c\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-9r5sb"
Oct 30 21:27:55.740045 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.740029    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8km8\" (UniqueName: \"kubernetes.io/projected/67292bdc-7fe1-449d-af75-2baa90c16ea9-kube-api-access-m8km8\") pod \"console-operator-7cd56b8dbc-sklxd\" (UID: \"67292bdc-7fe1-449d-af75-2baa90c16ea9\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-sklxd"
Oct 30 21:27:55.815320 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.815290    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-9mt44"
Oct 30 21:27:55.817687 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.817667    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3175ff47-1038-4139-9c0a-87062bb38dc2-cert\") pod \"ingress-canary-hzvrj\" (UID: \"3175ff47-1038-4139-9c0a-87062bb38dc2\") " pod="openshift-ingress-canary/ingress-canary-hzvrj"
Oct 30 21:27:55.817756 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.817699    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/94ee3757-5440-42e1-8617-2a68744ef3f4-image-registry-private-configuration\") pod \"image-registry-86f87fb58b-5clbv\" (UID: \"94ee3757-5440-42e1-8617-2a68744ef3f4\") " pod="openshift-image-registry/image-registry-86f87fb58b-5clbv"
Oct 30 21:27:55.817756 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.817719    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/94ee3757-5440-42e1-8617-2a68744ef3f4-registry-tls\") pod \"image-registry-86f87fb58b-5clbv\" (UID: \"94ee3757-5440-42e1-8617-2a68744ef3f4\") " pod="openshift-image-registry/image-registry-86f87fb58b-5clbv"
Oct 30 21:27:55.817854 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.817834    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/9f54b2c6-b1f9-42da-abdc-275fc8827b71-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-9qppl\" (UID: \"9f54b2c6-b1f9-42da-abdc-275fc8827b71\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-9qppl"
Oct 30 21:27:55.817972 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.817874    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9f54b2c6-b1f9-42da-abdc-275fc8827b71-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-9qppl\" (UID: \"9f54b2c6-b1f9-42da-abdc-275fc8827b71\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-9qppl"
Oct 30 21:27:55.817972 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:55.817894    2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 21:27:55.817972 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:55.817911    2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-86f87fb58b-5clbv: secret "image-registry-tls" not found
Oct 30 21:27:55.817972 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.817923    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/94ee3757-5440-42e1-8617-2a68744ef3f4-bound-sa-token\") pod \"image-registry-86f87fb58b-5clbv\" (UID: \"94ee3757-5440-42e1-8617-2a68744ef3f4\") " pod="openshift-image-registry/image-registry-86f87fb58b-5clbv"
Oct 30 21:27:55.817972 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.817963    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/94ee3757-5440-42e1-8617-2a68744ef3f4-registry-certificates\") pod \"image-registry-86f87fb58b-5clbv\" (UID: \"94ee3757-5440-42e1-8617-2a68744ef3f4\") " pod="openshift-image-registry/image-registry-86f87fb58b-5clbv"
Oct 30 21:27:55.818222 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:55.817981    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/94ee3757-5440-42e1-8617-2a68744ef3f4-registry-tls podName:94ee3757-5440-42e1-8617-2a68744ef3f4 nodeName:}" failed. No retries permitted until 2026-10-30 21:27:56.317961959 +0000 UTC m=+31.199024904 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/94ee3757-5440-42e1-8617-2a68744ef3f4-registry-tls") pod "image-registry-86f87fb58b-5clbv" (UID: "94ee3757-5440-42e1-8617-2a68744ef3f4") : secret "image-registry-tls" not found
Oct 30 21:27:55.818222 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:55.817998    2573 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 21:27:55.818222 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:55.818068    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f54b2c6-b1f9-42da-abdc-275fc8827b71-samples-operator-tls podName:9f54b2c6-b1f9-42da-abdc-275fc8827b71 nodeName:}" failed. No retries permitted until 2026-10-30 21:27:56.318052109 +0000 UTC m=+31.199115057 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9f54b2c6-b1f9-42da-abdc-275fc8827b71-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-9qppl" (UID: "9f54b2c6-b1f9-42da-abdc-275fc8827b71") : secret "samples-operator-tls" not found
Oct 30 21:27:55.818222 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.818015    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6lpv\" (UniqueName: \"kubernetes.io/projected/94ee3757-5440-42e1-8617-2a68744ef3f4-kube-api-access-t6lpv\") pod \"image-registry-86f87fb58b-5clbv\" (UID: \"94ee3757-5440-42e1-8617-2a68744ef3f4\") " pod="openshift-image-registry/image-registry-86f87fb58b-5clbv"
Oct 30 21:27:55.818222 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.818100    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08498890-fb63-4aea-9cf7-a2ade5397df2-tmp-dir\") pod \"dns-default-sbhch\" (UID: \"08498890-fb63-4aea-9cf7-a2ade5397df2\") " pod="openshift-dns/dns-default-sbhch"
Oct 30 21:27:55.818222 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.818134    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdf4s\" (UniqueName: \"kubernetes.io/projected/08498890-fb63-4aea-9cf7-a2ade5397df2-kube-api-access-hdf4s\") pod \"dns-default-sbhch\" (UID: \"08498890-fb63-4aea-9cf7-a2ade5397df2\") " pod="openshift-dns/dns-default-sbhch"
Oct 30 21:27:55.818222 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.818166    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/94ee3757-5440-42e1-8617-2a68744ef3f4-trusted-ca\") pod \"image-registry-86f87fb58b-5clbv\" (UID: \"94ee3757-5440-42e1-8617-2a68744ef3f4\") " pod="openshift-image-registry/image-registry-86f87fb58b-5clbv"
Oct 30 21:27:55.818222 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.818219    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/94ee3757-5440-42e1-8617-2a68744ef3f4-ca-trust-extracted\") pod \"image-registry-86f87fb58b-5clbv\" (UID: \"94ee3757-5440-42e1-8617-2a68744ef3f4\") " pod="openshift-image-registry/image-registry-86f87fb58b-5clbv"
Oct 30 21:27:55.818635 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.818294    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfphc\" (UniqueName: \"kubernetes.io/projected/3175ff47-1038-4139-9c0a-87062bb38dc2-kube-api-access-zfphc\") pod \"ingress-canary-hzvrj\" (UID: \"3175ff47-1038-4139-9c0a-87062bb38dc2\") " pod="openshift-ingress-canary/ingress-canary-hzvrj"
Oct 30 21:27:55.818635 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.818320    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08498890-fb63-4aea-9cf7-a2ade5397df2-metrics-tls\") pod \"dns-default-sbhch\" (UID: \"08498890-fb63-4aea-9cf7-a2ade5397df2\") " pod="openshift-dns/dns-default-sbhch"
Oct 30 21:27:55.818635 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.818351    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqwd9\" (UniqueName: \"kubernetes.io/projected/9f54b2c6-b1f9-42da-abdc-275fc8827b71-kube-api-access-jqwd9\") pod \"cluster-samples-operator-6ddb6c8b94-9qppl\" (UID: \"9f54b2c6-b1f9-42da-abdc-275fc8827b71\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-9qppl"
Oct 30 21:27:55.818635 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.818405    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/08498890-fb63-4aea-9cf7-a2ade5397df2-config-volume\") pod \"dns-default-sbhch\" (UID: \"08498890-fb63-4aea-9cf7-a2ade5397df2\") " pod="openshift-dns/dns-default-sbhch"
Oct 30 21:27:55.818635 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.818460    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/94ee3757-5440-42e1-8617-2a68744ef3f4-installation-pull-secrets\") pod \"image-registry-86f87fb58b-5clbv\" (UID: \"94ee3757-5440-42e1-8617-2a68744ef3f4\") " pod="openshift-image-registry/image-registry-86f87fb58b-5clbv"
Oct 30 21:27:55.818635 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.818559    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/94ee3757-5440-42e1-8617-2a68744ef3f4-registry-certificates\") pod \"image-registry-86f87fb58b-5clbv\" (UID: \"94ee3757-5440-42e1-8617-2a68744ef3f4\") " pod="openshift-image-registry/image-registry-86f87fb58b-5clbv"
Oct 30 21:27:55.818919 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.818637    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08498890-fb63-4aea-9cf7-a2ade5397df2-tmp-dir\") pod \"dns-default-sbhch\" (UID: \"08498890-fb63-4aea-9cf7-a2ade5397df2\") " pod="openshift-dns/dns-default-sbhch"
Oct 30 21:27:55.818919 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.818664    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/9f54b2c6-b1f9-42da-abdc-275fc8827b71-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-9qppl\" (UID: \"9f54b2c6-b1f9-42da-abdc-275fc8827b71\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-9qppl"
Oct 30 21:27:55.819018 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:55.818922    2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 21:27:55.819018 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:55.818968    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08498890-fb63-4aea-9cf7-a2ade5397df2-metrics-tls podName:08498890-fb63-4aea-9cf7-a2ade5397df2 nodeName:}" failed. No retries permitted until 2026-10-30 21:27:56.318954319 +0000 UTC m=+31.200017267 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/08498890-fb63-4aea-9cf7-a2ade5397df2-metrics-tls") pod "dns-default-sbhch" (UID: "08498890-fb63-4aea-9cf7-a2ade5397df2") : secret "dns-default-metrics-tls" not found
Oct 30 21:27:55.819018 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.818917    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/94ee3757-5440-42e1-8617-2a68744ef3f4-ca-trust-extracted\") pod \"image-registry-86f87fb58b-5clbv\" (UID: \"94ee3757-5440-42e1-8617-2a68744ef3f4\") " pod="openshift-image-registry/image-registry-86f87fb58b-5clbv"
Oct 30 21:27:55.819174 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.819151    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/08498890-fb63-4aea-9cf7-a2ade5397df2-config-volume\") pod \"dns-default-sbhch\" (UID: \"08498890-fb63-4aea-9cf7-a2ade5397df2\") " pod="openshift-dns/dns-default-sbhch"
Oct 30 21:27:55.819302 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.819279    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/94ee3757-5440-42e1-8617-2a68744ef3f4-trusted-ca\") pod \"image-registry-86f87fb58b-5clbv\" (UID: \"94ee3757-5440-42e1-8617-2a68744ef3f4\") " pod="openshift-image-registry/image-registry-86f87fb58b-5clbv"
Oct 30 21:27:55.820343 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.820326    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3175ff47-1038-4139-9c0a-87062bb38dc2-cert\") pod \"ingress-canary-hzvrj\" (UID: \"3175ff47-1038-4139-9c0a-87062bb38dc2\") " pod="openshift-ingress-canary/ingress-canary-hzvrj"
Oct 30 21:27:55.820792 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.820773    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/94ee3757-5440-42e1-8617-2a68744ef3f4-image-registry-private-configuration\") pod \"image-registry-86f87fb58b-5clbv\" (UID: \"94ee3757-5440-42e1-8617-2a68744ef3f4\") " pod="openshift-image-registry/image-registry-86f87fb58b-5clbv"
Oct 30 21:27:55.821485 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.821462    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/94ee3757-5440-42e1-8617-2a68744ef3f4-installation-pull-secrets\") pod \"image-registry-86f87fb58b-5clbv\" (UID: \"94ee3757-5440-42e1-8617-2a68744ef3f4\") " pod="openshift-image-registry/image-registry-86f87fb58b-5clbv"
Oct 30 21:27:55.828479 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.828460    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-l4dbk"
Oct 30 21:27:55.831603 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.831532    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqwd9\" (UniqueName: \"kubernetes.io/projected/9f54b2c6-b1f9-42da-abdc-275fc8827b71-kube-api-access-jqwd9\") pod \"cluster-samples-operator-6ddb6c8b94-9qppl\" (UID: \"9f54b2c6-b1f9-42da-abdc-275fc8827b71\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-9qppl"
Oct 30 21:27:55.831603 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.831569    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/94ee3757-5440-42e1-8617-2a68744ef3f4-bound-sa-token\") pod \"image-registry-86f87fb58b-5clbv\" (UID: \"94ee3757-5440-42e1-8617-2a68744ef3f4\") " pod="openshift-image-registry/image-registry-86f87fb58b-5clbv"
Oct 30 21:27:55.831771 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.831700    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfphc\" (UniqueName: \"kubernetes.io/projected/3175ff47-1038-4139-9c0a-87062bb38dc2-kube-api-access-zfphc\") pod \"ingress-canary-hzvrj\" (UID: \"3175ff47-1038-4139-9c0a-87062bb38dc2\") " pod="openshift-ingress-canary/ingress-canary-hzvrj"
Oct 30 21:27:55.831847 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.831830    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6lpv\" (UniqueName: \"kubernetes.io/projected/94ee3757-5440-42e1-8617-2a68744ef3f4-kube-api-access-t6lpv\") pod \"image-registry-86f87fb58b-5clbv\" (UID: \"94ee3757-5440-42e1-8617-2a68744ef3f4\") " pod="openshift-image-registry/image-registry-86f87fb58b-5clbv"
Oct 30 21:27:55.831954 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.831939    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdf4s\" (UniqueName: \"kubernetes.io/projected/08498890-fb63-4aea-9cf7-a2ade5397df2-kube-api-access-hdf4s\") pod \"dns-default-sbhch\" (UID: \"08498890-fb63-4aea-9cf7-a2ade5397df2\") " pod="openshift-dns/dns-default-sbhch"
Oct 30 21:27:55.841653 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.841634    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-7cgk9"
Oct 30 21:27:55.853064 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.852830    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-zvqvt"
Oct 30 21:27:55.861171 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.861144    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-9p8mb"
Oct 30 21:27:55.879859 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.879825    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-lbjhv"
Oct 30 21:27:55.893415 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.893311    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-sklxd"
Oct 30 21:27:55.925655 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.925397    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-9r5sb"
Oct 30 21:27:55.925655 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.925465    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-tdmc8"
Oct 30 21:27:55.948809 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:55.948265    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hzvrj"
Oct 30 21:27:56.120046 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:56.120019    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-l4dbk"]
Oct 30 21:27:56.123373 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:56.123333    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-9mt44"]
Oct 30 21:27:56.124535 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:56.124506    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb314739_c678_4da8_9d54_147dc0612ee2.slice/crio-f2508aca20577c2f409c061e53ac97f8b3464b791954ba175ee8f9ef6d4c76f8 WatchSource:0}: Error finding container f2508aca20577c2f409c061e53ac97f8b3464b791954ba175ee8f9ef6d4c76f8: Status 404 returned error can't find the container with id f2508aca20577c2f409c061e53ac97f8b3464b791954ba175ee8f9ef6d4c76f8
Oct 30 21:27:56.124965 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:56.124946    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c835b41_25cb_4f0c_8e7c_4ac9e2fee318.slice/crio-75526b7530220d62f568ed172aa859dcd9a25086d66f706ee52dc359e8bdc322 WatchSource:0}: Error finding container 75526b7530220d62f568ed172aa859dcd9a25086d66f706ee52dc359e8bdc322: Status 404 returned error can't find the container with id 75526b7530220d62f568ed172aa859dcd9a25086d66f706ee52dc359e8bdc322
Oct 30 21:27:56.148692 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:56.148661    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-7cgk9"]
Oct 30 21:27:56.150260 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:56.150225    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6ca21b9_20a4_4887_9d08_19d9c491a09e.slice/crio-914ffc5c554a5c4122f2c41a837189ca7dc51ff6991d0eb637f078079903684a WatchSource:0}: Error finding container 914ffc5c554a5c4122f2c41a837189ca7dc51ff6991d0eb637f078079903684a: Status 404 returned error can't find the container with id 914ffc5c554a5c4122f2c41a837189ca7dc51ff6991d0eb637f078079903684a
Oct 30 21:27:56.169919 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:56.169832    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-zvqvt"]
Oct 30 21:27:56.178864 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:56.178824    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d83c70b_fab8_4872_969f_11f74c0ddea0.slice/crio-ef6c6d4940b60a96a28dda299962451ccbe7af14eef1e415eaf6be3ef995c61d WatchSource:0}: Error finding container ef6c6d4940b60a96a28dda299962451ccbe7af14eef1e415eaf6be3ef995c61d: Status 404 returned error can't find the container with id ef6c6d4940b60a96a28dda299962451ccbe7af14eef1e415eaf6be3ef995c61d
Oct 30 21:27:56.192442 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:56.192412    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-9p8mb"]
Oct 30 21:27:56.208503 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:56.208476    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bb24a0d_3af0_4a05_bba4_4f9630b04bf3.slice/crio-01a6be223caaa40916fbe86ec4ffed50b3b9de9f3f8b0241a8c368c01fa597a7 WatchSource:0}: Error finding container 01a6be223caaa40916fbe86ec4ffed50b3b9de9f3f8b0241a8c368c01fa597a7: Status 404 returned error can't find the container with id 01a6be223caaa40916fbe86ec4ffed50b3b9de9f3f8b0241a8c368c01fa597a7
Oct 30 21:27:56.228729 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:56.228705    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b4f9524-db75-4206-aa76-04400d95706f-service-ca-bundle\") pod \"router-default-5bbfd9465-x28lh\" (UID: \"9b4f9524-db75-4206-aa76-04400d95706f\") " pod="openshift-ingress/router-default-5bbfd9465-x28lh"
Oct 30 21:27:56.228844 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:56.228783    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/753d4639-09e6-4850-9f4d-ed39b0406439-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-lwm89\" (UID: \"753d4639-09e6-4850-9f4d-ed39b0406439\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-lwm89"
Oct 30 21:27:56.228907 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:56.228888    2573 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 21:27:56.228950 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:56.228920    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9b4f9524-db75-4206-aa76-04400d95706f-service-ca-bundle podName:9b4f9524-db75-4206-aa76-04400d95706f nodeName:}" failed. No retries permitted until 2026-10-30 21:27:57.22889603 +0000 UTC m=+32.109958975 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9b4f9524-db75-4206-aa76-04400d95706f-service-ca-bundle") pod "router-default-5bbfd9465-x28lh" (UID: "9b4f9524-db75-4206-aa76-04400d95706f") : configmap references non-existent config key: service-ca.crt
Oct 30 21:27:56.228950 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:56.228948    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/753d4639-09e6-4850-9f4d-ed39b0406439-networking-console-plugin-cert podName:753d4639-09e6-4850-9f4d-ed39b0406439 nodeName:}" failed. No retries permitted until 2026-10-30 21:27:57.228937156 +0000 UTC m=+32.110000107 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/753d4639-09e6-4850-9f4d-ed39b0406439-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-lwm89" (UID: "753d4639-09e6-4850-9f4d-ed39b0406439") : secret "networking-console-plugin-cert" not found
Oct 30 21:27:56.229043 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:56.229010    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b4f9524-db75-4206-aa76-04400d95706f-metrics-certs\") pod \"router-default-5bbfd9465-x28lh\" (UID: \"9b4f9524-db75-4206-aa76-04400d95706f\") " pod="openshift-ingress/router-default-5bbfd9465-x28lh"
Oct 30 21:27:56.229095 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:56.229048    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/90937bc9-0051-407d-9c82-014f0a9c65d4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-kdrrd\" (UID: \"90937bc9-0051-407d-9c82-014f0a9c65d4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-kdrrd"
Oct 30 21:27:56.229095 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:56.229077    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/039cf439-d12d-450a-80c1-eb3efeb50e8a-registry-tls\") pod \"image-registry-76679b6db7-ltmt6\" (UID: \"039cf439-d12d-450a-80c1-eb3efeb50e8a\") " pod="openshift-image-registry/image-registry-76679b6db7-ltmt6"
Oct 30 21:27:56.229202 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:56.229187    2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 21:27:56.229252 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:56.229205    2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76679b6db7-ltmt6: secret "image-registry-tls" not found
Oct 30 21:27:56.229252 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:56.229242    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/039cf439-d12d-450a-80c1-eb3efeb50e8a-registry-tls podName:039cf439-d12d-450a-80c1-eb3efeb50e8a nodeName:}" failed. No retries permitted until 2026-10-30 21:27:57.229229447 +0000 UTC m=+32.110292392 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/039cf439-d12d-450a-80c1-eb3efeb50e8a-registry-tls") pod "image-registry-76679b6db7-ltmt6" (UID: "039cf439-d12d-450a-80c1-eb3efeb50e8a") : secret "image-registry-tls" not found
Oct 30 21:27:56.229413 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:56.229297    2573 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 21:27:56.229413 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:56.229318    2573 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 30 21:27:56.229413 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:56.229329    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b4f9524-db75-4206-aa76-04400d95706f-metrics-certs podName:9b4f9524-db75-4206-aa76-04400d95706f nodeName:}" failed. No retries permitted until 2026-10-30 21:27:57.229318636 +0000 UTC m=+32.110381581 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9b4f9524-db75-4206-aa76-04400d95706f-metrics-certs") pod "router-default-5bbfd9465-x28lh" (UID: "9b4f9524-db75-4206-aa76-04400d95706f") : secret "router-metrics-certs-default" not found
Oct 30 21:27:56.229413 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:56.229355    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90937bc9-0051-407d-9c82-014f0a9c65d4-cluster-monitoring-operator-tls podName:90937bc9-0051-407d-9c82-014f0a9c65d4 nodeName:}" failed. No retries permitted until 2026-10-30 21:27:57.229343792 +0000 UTC m=+32.110406732 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/90937bc9-0051-407d-9c82-014f0a9c65d4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-kdrrd" (UID: "90937bc9-0051-407d-9c82-014f0a9c65d4") : secret "cluster-monitoring-operator-tls" not found
Oct 30 21:27:56.257519 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:56.256433    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-lbjhv"]
Oct 30 21:27:56.259354 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:56.259322    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9d2f04b_bb7d_422f_b497_5fa61580f95a.slice/crio-0f738ceac995d4100163621b468e95def6099e0a6b0e9758ea5f3b383bb595f4 WatchSource:0}: Error finding container 0f738ceac995d4100163621b468e95def6099e0a6b0e9758ea5f3b383bb595f4: Status 404 returned error can't find the container with id 0f738ceac995d4100163621b468e95def6099e0a6b0e9758ea5f3b383bb595f4
Oct 30 21:27:56.265895 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:56.265866    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8873f9b9_2ac5_409c_9299_fa024e00189c.slice/crio-1c916fbc0fbaa7a6b041fa9c07924c5020e909cd2b5516de2fae05b8d63f51a4 WatchSource:0}: Error finding container 1c916fbc0fbaa7a6b041fa9c07924c5020e909cd2b5516de2fae05b8d63f51a4: Status 404 returned error can't find the container with id 1c916fbc0fbaa7a6b041fa9c07924c5020e909cd2b5516de2fae05b8d63f51a4
Oct 30 21:27:56.267634 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:56.267570    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-9r5sb"]
Oct 30 21:27:56.269448 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:56.269425    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-sklxd"]
Oct 30 21:27:56.272273 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:56.272248    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hzvrj"]
Oct 30 21:27:56.273388 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:56.273338    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67292bdc_7fe1_449d_af75_2baa90c16ea9.slice/crio-0bc1321b91785136139aed9902de94cdf1a2c849b8813d68d5d7a26b1d104efb WatchSource:0}: Error finding container 0bc1321b91785136139aed9902de94cdf1a2c849b8813d68d5d7a26b1d104efb: Status 404 returned error can't find the container with id 0bc1321b91785136139aed9902de94cdf1a2c849b8813d68d5d7a26b1d104efb
Oct 30 21:27:56.274332 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:56.274305    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3175ff47_1038_4139_9c0a_87062bb38dc2.slice/crio-f555f9b5cf699ae142da23c63edc76a2380d32f62058b986654cf82116d1c660 WatchSource:0}: Error finding container f555f9b5cf699ae142da23c63edc76a2380d32f62058b986654cf82116d1c660: Status 404 returned error can't find the container with id f555f9b5cf699ae142da23c63edc76a2380d32f62058b986654cf82116d1c660
Oct 30 21:27:56.283620 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:56.283599    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-tdmc8"]
Oct 30 21:27:56.294273 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:56.294247    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbba1a087_6e04_4db8_ba82_7260a88e2843.slice/crio-0bbe505a0a4b7594e8e39d86bde05cbb8a3dd7cb5cb29b5128cb8d3b9cd43991 WatchSource:0}: Error finding container 0bbe505a0a4b7594e8e39d86bde05cbb8a3dd7cb5cb29b5128cb8d3b9cd43991: Status 404 returned error can't find the container with id 0bbe505a0a4b7594e8e39d86bde05cbb8a3dd7cb5cb29b5128cb8d3b9cd43991
Oct 30 21:27:56.330052 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:56.330026    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08498890-fb63-4aea-9cf7-a2ade5397df2-metrics-tls\") pod \"dns-default-sbhch\" (UID: \"08498890-fb63-4aea-9cf7-a2ade5397df2\") " pod="openshift-dns/dns-default-sbhch"
Oct 30 21:27:56.330141 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:56.330102    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/94ee3757-5440-42e1-8617-2a68744ef3f4-registry-tls\") pod \"image-registry-86f87fb58b-5clbv\" (UID: \"94ee3757-5440-42e1-8617-2a68744ef3f4\") " pod="openshift-image-registry/image-registry-86f87fb58b-5clbv"
Oct 30 21:27:56.330230 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:56.330209    2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 21:27:56.330265 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:56.330249    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9f54b2c6-b1f9-42da-abdc-275fc8827b71-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-9qppl\" (UID: \"9f54b2c6-b1f9-42da-abdc-275fc8827b71\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-9qppl"
Oct 30 21:27:56.330312 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:56.330280    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08498890-fb63-4aea-9cf7-a2ade5397df2-metrics-tls podName:08498890-fb63-4aea-9cf7-a2ade5397df2 nodeName:}" failed. No retries permitted until 2026-10-30 21:27:57.330259975 +0000 UTC m=+32.211322908 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/08498890-fb63-4aea-9cf7-a2ade5397df2-metrics-tls") pod "dns-default-sbhch" (UID: "08498890-fb63-4aea-9cf7-a2ade5397df2") : secret "dns-default-metrics-tls" not found
Oct 30 21:27:56.330312 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:56.330218    2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 21:27:56.330424 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:56.330315    2573 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 21:27:56.330424 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:56.330320    2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-86f87fb58b-5clbv: secret "image-registry-tls" not found
Oct 30 21:27:56.330424 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:56.330355    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f54b2c6-b1f9-42da-abdc-275fc8827b71-samples-operator-tls podName:9f54b2c6-b1f9-42da-abdc-275fc8827b71 nodeName:}" failed. No retries permitted until 2026-10-30 21:27:57.330344925 +0000 UTC m=+32.211407870 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9f54b2c6-b1f9-42da-abdc-275fc8827b71-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-9qppl" (UID: "9f54b2c6-b1f9-42da-abdc-275fc8827b71") : secret "samples-operator-tls" not found
Oct 30 21:27:56.330548 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:56.330478    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/94ee3757-5440-42e1-8617-2a68744ef3f4-registry-tls podName:94ee3757-5440-42e1-8617-2a68744ef3f4 nodeName:}" failed. No retries permitted until 2026-10-30 21:27:57.330458255 +0000 UTC m=+32.211521203 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/94ee3757-5440-42e1-8617-2a68744ef3f4-registry-tls") pod "image-registry-86f87fb58b-5clbv" (UID: "94ee3757-5440-42e1-8617-2a68744ef3f4") : secret "image-registry-tls" not found
Oct 30 21:27:56.658099 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:56.658062    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4cvlt"
Oct 30 21:27:56.658099 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:56.658090    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vwxrd"
Oct 30 21:27:56.658309 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:56.658100    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-csbxh"
Oct 30 21:27:56.670810 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:56.670785    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 21:27:56.670810 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:56.670804    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-f4rkh\""
Oct 30 21:27:56.671098 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:56.671074    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 21:27:56.671202 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:56.671110    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-rpdb2\""
Oct 30 21:27:56.671202 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:56.671117    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-cjjj4\""
Oct 30 21:27:56.961601 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:56.961560    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-lbjhv" event={"ID":"e9d2f04b-bb7d-422f-b497-5fa61580f95a","Type":"ContainerStarted","Data":"0f738ceac995d4100163621b468e95def6099e0a6b0e9758ea5f3b383bb595f4"}
Oct 30 21:27:56.966072 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:56.966037    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-tdmc8" event={"ID":"bba1a087-6e04-4db8-ba82-7260a88e2843","Type":"ContainerStarted","Data":"0bbe505a0a4b7594e8e39d86bde05cbb8a3dd7cb5cb29b5128cb8d3b9cd43991"}
Oct 30 21:27:56.968807 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:56.968776    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-9r5sb" event={"ID":"8873f9b9-2ac5-409c-9299-fa024e00189c","Type":"ContainerStarted","Data":"1c916fbc0fbaa7a6b041fa9c07924c5020e909cd2b5516de2fae05b8d63f51a4"}
Oct 30 21:27:56.970672 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:56.970573    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-9p8mb" event={"ID":"8bb24a0d-3af0-4a05-bba4-4f9630b04bf3","Type":"ContainerStarted","Data":"01a6be223caaa40916fbe86ec4ffed50b3b9de9f3f8b0241a8c368c01fa597a7"}
Oct 30 21:27:56.974738 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:56.974704    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-zvqvt" event={"ID":"0d83c70b-fab8-4872-969f-11f74c0ddea0","Type":"ContainerStarted","Data":"b76ef3ffcdd1a221d6a9db76fa2aac43fa71db5b524d8eddc59022644c3bc2ea"}
Oct 30 21:27:56.974861 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:56.974749    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-zvqvt" event={"ID":"0d83c70b-fab8-4872-969f-11f74c0ddea0","Type":"ContainerStarted","Data":"ef6c6d4940b60a96a28dda299962451ccbe7af14eef1e415eaf6be3ef995c61d"}
Oct 30 21:27:56.978125 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:56.978047    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-l4dbk" event={"ID":"fb314739-c678-4da8-9d54-147dc0612ee2","Type":"ContainerStarted","Data":"ba221cb0b7fdd714fc410756dee54329980ca0ea9b644abdd01c4249bc9ee1bb"}
Oct 30 21:27:56.978125 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:56.978094    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-l4dbk" event={"ID":"fb314739-c678-4da8-9d54-147dc0612ee2","Type":"ContainerStarted","Data":"f2508aca20577c2f409c061e53ac97f8b3464b791954ba175ee8f9ef6d4c76f8"}
Oct 30 21:27:56.981398 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:56.981109    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-9mt44" event={"ID":"3c835b41-25cb-4f0c-8e7c-4ac9e2fee318","Type":"ContainerStarted","Data":"971519aa49ce2e584f7040649e1cdd7b72335a300caa7c7caa1f07d888adcda4"}
Oct 30 21:27:56.981398 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:56.981152    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-9mt44" event={"ID":"3c835b41-25cb-4f0c-8e7c-4ac9e2fee318","Type":"ContainerStarted","Data":"75526b7530220d62f568ed172aa859dcd9a25086d66f706ee52dc359e8bdc322"}
Oct 30 21:27:56.983935 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:56.983886    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hzvrj" event={"ID":"3175ff47-1038-4139-9c0a-87062bb38dc2","Type":"ContainerStarted","Data":"f555f9b5cf699ae142da23c63edc76a2380d32f62058b986654cf82116d1c660"}
Oct 30 21:27:56.985614 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:56.985592    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-7cgk9" event={"ID":"d6ca21b9-20a4-4887-9d08-19d9c491a09e","Type":"ContainerStarted","Data":"914ffc5c554a5c4122f2c41a837189ca7dc51ff6991d0eb637f078079903684a"}
Oct 30 21:27:56.986880 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:56.986856    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-sklxd" event={"ID":"67292bdc-7fe1-449d-af75-2baa90c16ea9","Type":"ContainerStarted","Data":"0bc1321b91785136139aed9902de94cdf1a2c849b8813d68d5d7a26b1d104efb"}
Oct 30 21:27:56.997997 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:56.997944    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-zvqvt" podStartSLOduration=362.997921466 podStartE2EDuration="6m2.997921466s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:21:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:27:56.996800698 +0000 UTC m=+31.877863652" watchObservedRunningTime="2026-10-30 21:27:56.997921466 +0000 UTC m=+31.878984422"
Oct 30 21:27:57.036316 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:57.036265    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-l4dbk" podStartSLOduration=363.036251852 podStartE2EDuration="6m3.036251852s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:21:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:27:57.018635578 +0000 UTC m=+31.899698532" watchObservedRunningTime="2026-10-30 21:27:57.036251852 +0000 UTC m=+31.917314805"
Oct 30 21:27:57.036649 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:57.036610    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-9mt44" podStartSLOduration=363.036602983 podStartE2EDuration="6m3.036602983s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:21:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:27:57.035930876 +0000 UTC m=+31.916993825" watchObservedRunningTime="2026-10-30 21:27:57.036602983 +0000 UTC m=+31.917665939"
Oct 30 21:27:57.241270 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:57.240584    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b4f9524-db75-4206-aa76-04400d95706f-metrics-certs\") pod \"router-default-5bbfd9465-x28lh\" (UID: \"9b4f9524-db75-4206-aa76-04400d95706f\") " pod="openshift-ingress/router-default-5bbfd9465-x28lh"
Oct 30 21:27:57.241270 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:57.240637    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/90937bc9-0051-407d-9c82-014f0a9c65d4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-kdrrd\" (UID: \"90937bc9-0051-407d-9c82-014f0a9c65d4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-kdrrd"
Oct 30 21:27:57.241270 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:57.240667    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/039cf439-d12d-450a-80c1-eb3efeb50e8a-registry-tls\") pod \"image-registry-76679b6db7-ltmt6\" (UID: \"039cf439-d12d-450a-80c1-eb3efeb50e8a\") " pod="openshift-image-registry/image-registry-76679b6db7-ltmt6"
Oct 30 21:27:57.241270 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:57.240735    2573 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 21:27:57.241270 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:57.240786    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b4f9524-db75-4206-aa76-04400d95706f-service-ca-bundle\") pod \"router-default-5bbfd9465-x28lh\" (UID: \"9b4f9524-db75-4206-aa76-04400d95706f\") " pod="openshift-ingress/router-default-5bbfd9465-x28lh"
Oct 30 21:27:57.241270 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:57.240813    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b4f9524-db75-4206-aa76-04400d95706f-metrics-certs podName:9b4f9524-db75-4206-aa76-04400d95706f nodeName:}" failed. No retries permitted until 2026-10-30 21:27:59.240795153 +0000 UTC m=+34.121858103 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9b4f9524-db75-4206-aa76-04400d95706f-metrics-certs") pod "router-default-5bbfd9465-x28lh" (UID: "9b4f9524-db75-4206-aa76-04400d95706f") : secret "router-metrics-certs-default" not found
Oct 30 21:27:57.241270 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:57.240858    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9b4f9524-db75-4206-aa76-04400d95706f-service-ca-bundle podName:9b4f9524-db75-4206-aa76-04400d95706f nodeName:}" failed. No retries permitted until 2026-10-30 21:27:59.240846343 +0000 UTC m=+34.121909287 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9b4f9524-db75-4206-aa76-04400d95706f-service-ca-bundle") pod "router-default-5bbfd9465-x28lh" (UID: "9b4f9524-db75-4206-aa76-04400d95706f") : configmap references non-existent config key: service-ca.crt
Oct 30 21:27:57.241270 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:57.240885    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/753d4639-09e6-4850-9f4d-ed39b0406439-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-lwm89\" (UID: \"753d4639-09e6-4850-9f4d-ed39b0406439\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-lwm89"
Oct 30 21:27:57.241270 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:57.240982    2573 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 21:27:57.241270 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:57.241015    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/753d4639-09e6-4850-9f4d-ed39b0406439-networking-console-plugin-cert podName:753d4639-09e6-4850-9f4d-ed39b0406439 nodeName:}" failed. No retries permitted until 2026-10-30 21:27:59.241004712 +0000 UTC m=+34.122067659 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/753d4639-09e6-4850-9f4d-ed39b0406439-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-lwm89" (UID: "753d4639-09e6-4850-9f4d-ed39b0406439") : secret "networking-console-plugin-cert" not found
Oct 30 21:27:57.241270 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:57.241064    2573 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 30 21:27:57.241270 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:57.241094    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90937bc9-0051-407d-9c82-014f0a9c65d4-cluster-monitoring-operator-tls podName:90937bc9-0051-407d-9c82-014f0a9c65d4 nodeName:}" failed. No retries permitted until 2026-10-30 21:27:59.241085405 +0000 UTC m=+34.122148336 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/90937bc9-0051-407d-9c82-014f0a9c65d4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-kdrrd" (UID: "90937bc9-0051-407d-9c82-014f0a9c65d4") : secret "cluster-monitoring-operator-tls" not found
Oct 30 21:27:57.241270 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:57.241148    2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 21:27:57.241270 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:57.241159    2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76679b6db7-ltmt6: secret "image-registry-tls" not found
Oct 30 21:27:57.241270 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:57.241189    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/039cf439-d12d-450a-80c1-eb3efeb50e8a-registry-tls podName:039cf439-d12d-450a-80c1-eb3efeb50e8a nodeName:}" failed. No retries permitted until 2026-10-30 21:27:59.241177779 +0000 UTC m=+34.122240709 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/039cf439-d12d-450a-80c1-eb3efeb50e8a-registry-tls") pod "image-registry-76679b6db7-ltmt6" (UID: "039cf439-d12d-450a-80c1-eb3efeb50e8a") : secret "image-registry-tls" not found
Oct 30 21:27:57.343016 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:57.341998    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08498890-fb63-4aea-9cf7-a2ade5397df2-metrics-tls\") pod \"dns-default-sbhch\" (UID: \"08498890-fb63-4aea-9cf7-a2ade5397df2\") " pod="openshift-dns/dns-default-sbhch"
Oct 30 21:27:57.343016 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:57.342120    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/94ee3757-5440-42e1-8617-2a68744ef3f4-registry-tls\") pod \"image-registry-86f87fb58b-5clbv\" (UID: \"94ee3757-5440-42e1-8617-2a68744ef3f4\") " pod="openshift-image-registry/image-registry-86f87fb58b-5clbv"
Oct 30 21:27:57.343016 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:57.342159    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9f54b2c6-b1f9-42da-abdc-275fc8827b71-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-9qppl\" (UID: \"9f54b2c6-b1f9-42da-abdc-275fc8827b71\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-9qppl"
Oct 30 21:27:57.343016 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:57.342372    2573 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 21:27:57.343016 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:57.342437    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f54b2c6-b1f9-42da-abdc-275fc8827b71-samples-operator-tls podName:9f54b2c6-b1f9-42da-abdc-275fc8827b71 nodeName:}" failed. No retries permitted until 2026-10-30 21:27:59.342419757 +0000 UTC m=+34.223482690 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9f54b2c6-b1f9-42da-abdc-275fc8827b71-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-9qppl" (UID: "9f54b2c6-b1f9-42da-abdc-275fc8827b71") : secret "samples-operator-tls" not found
Oct 30 21:27:57.343016 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:57.342825    2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 21:27:57.343016 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:57.342879    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08498890-fb63-4aea-9cf7-a2ade5397df2-metrics-tls podName:08498890-fb63-4aea-9cf7-a2ade5397df2 nodeName:}" failed. No retries permitted until 2026-10-30 21:27:59.34286187 +0000 UTC m=+34.223924804 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/08498890-fb63-4aea-9cf7-a2ade5397df2-metrics-tls") pod "dns-default-sbhch" (UID: "08498890-fb63-4aea-9cf7-a2ade5397df2") : secret "dns-default-metrics-tls" not found
Oct 30 21:27:57.343016 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:57.342939    2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 21:27:57.343016 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:57.342949    2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-86f87fb58b-5clbv: secret "image-registry-tls" not found
Oct 30 21:27:57.343016 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:57.342979    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/94ee3757-5440-42e1-8617-2a68744ef3f4-registry-tls podName:94ee3757-5440-42e1-8617-2a68744ef3f4 nodeName:}" failed. No retries permitted until 2026-10-30 21:27:59.342968827 +0000 UTC m=+34.224031773 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/94ee3757-5440-42e1-8617-2a68744ef3f4-registry-tls") pod "image-registry-86f87fb58b-5clbv" (UID: "94ee3757-5440-42e1-8617-2a68744ef3f4") : secret "image-registry-tls" not found
Oct 30 21:27:59.003377 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:59.002623    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-lbjhv" event={"ID":"e9d2f04b-bb7d-422f-b497-5fa61580f95a","Type":"ContainerStarted","Data":"2bcb7ac387c07ad2d5cef78c22996e150de3eac7adf81ebc8f2dfb3cc090ddac"}
Oct 30 21:27:59.022103 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:59.022047    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-lbjhv" podStartSLOduration=329.757570395 podStartE2EDuration="5m32.022031902s" totalImagesPullingTime="2.264461507s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:22:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:27:59.021559417 +0000 UTC m=+33.902622372" watchObservedRunningTime="2026-10-30 21:27:59.022031902 +0000 UTC m=+33.903094854"
Oct 30 21:27:59.264746 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:59.264706    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b4f9524-db75-4206-aa76-04400d95706f-metrics-certs\") pod \"router-default-5bbfd9465-x28lh\" (UID: \"9b4f9524-db75-4206-aa76-04400d95706f\") " pod="openshift-ingress/router-default-5bbfd9465-x28lh"
Oct 30 21:27:59.264964 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:59.264764    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/90937bc9-0051-407d-9c82-014f0a9c65d4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-kdrrd\" (UID: \"90937bc9-0051-407d-9c82-014f0a9c65d4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-kdrrd"
Oct 30 21:27:59.264964 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:59.264798    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/039cf439-d12d-450a-80c1-eb3efeb50e8a-registry-tls\") pod \"image-registry-76679b6db7-ltmt6\" (UID: \"039cf439-d12d-450a-80c1-eb3efeb50e8a\") " pod="openshift-image-registry/image-registry-76679b6db7-ltmt6"
Oct 30 21:27:59.264964 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:59.264876    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/352b5b4c-c640-4300-abf3-aa70b08cbb4b-metrics-certs\") pod \"network-metrics-daemon-vwxrd\" (UID: \"352b5b4c-c640-4300-abf3-aa70b08cbb4b\") " pod="openshift-multus/network-metrics-daemon-vwxrd"
Oct 30 21:27:59.264964 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:59.264935    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b4f9524-db75-4206-aa76-04400d95706f-service-ca-bundle\") pod \"router-default-5bbfd9465-x28lh\" (UID: \"9b4f9524-db75-4206-aa76-04400d95706f\") " pod="openshift-ingress/router-default-5bbfd9465-x28lh"
Oct 30 21:27:59.265121 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:59.264983    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/753d4639-09e6-4850-9f4d-ed39b0406439-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-lwm89\" (UID: \"753d4639-09e6-4850-9f4d-ed39b0406439\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-lwm89"
Oct 30 21:27:59.265155 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:59.265124    2573 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 21:27:59.265226 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:59.265186    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/753d4639-09e6-4850-9f4d-ed39b0406439-networking-console-plugin-cert podName:753d4639-09e6-4850-9f4d-ed39b0406439 nodeName:}" failed. No retries permitted until 2026-10-30 21:28:03.26516662 +0000 UTC m=+38.146229565 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/753d4639-09e6-4850-9f4d-ed39b0406439-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-lwm89" (UID: "753d4639-09e6-4850-9f4d-ed39b0406439") : secret "networking-console-plugin-cert" not found
Oct 30 21:27:59.265623 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:59.265602    2573 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 21:27:59.265716 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:59.265656    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b4f9524-db75-4206-aa76-04400d95706f-metrics-certs podName:9b4f9524-db75-4206-aa76-04400d95706f nodeName:}" failed. No retries permitted until 2026-10-30 21:28:03.265639811 +0000 UTC m=+38.146702756 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9b4f9524-db75-4206-aa76-04400d95706f-metrics-certs") pod "router-default-5bbfd9465-x28lh" (UID: "9b4f9524-db75-4206-aa76-04400d95706f") : secret "router-metrics-certs-default" not found
Oct 30 21:27:59.265716 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:59.265710    2573 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 30 21:27:59.265836 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:59.265745    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90937bc9-0051-407d-9c82-014f0a9c65d4-cluster-monitoring-operator-tls podName:90937bc9-0051-407d-9c82-014f0a9c65d4 nodeName:}" failed. No retries permitted until 2026-10-30 21:28:03.265732631 +0000 UTC m=+38.146795581 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/90937bc9-0051-407d-9c82-014f0a9c65d4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-kdrrd" (UID: "90937bc9-0051-407d-9c82-014f0a9c65d4") : secret "cluster-monitoring-operator-tls" not found
Oct 30 21:27:59.265836 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:59.265799    2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 21:27:59.265836 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:59.265806    2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76679b6db7-ltmt6: secret "image-registry-tls" not found
Oct 30 21:27:59.265836 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:59.265826    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/039cf439-d12d-450a-80c1-eb3efeb50e8a-registry-tls podName:039cf439-d12d-450a-80c1-eb3efeb50e8a nodeName:}" failed. No retries permitted until 2026-10-30 21:28:03.265819792 +0000 UTC m=+38.146882736 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/039cf439-d12d-450a-80c1-eb3efeb50e8a-registry-tls") pod "image-registry-76679b6db7-ltmt6" (UID: "039cf439-d12d-450a-80c1-eb3efeb50e8a") : secret "image-registry-tls" not found
Oct 30 21:27:59.266038 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:59.265861    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 30 21:27:59.266038 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:59.265879    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/352b5b4c-c640-4300-abf3-aa70b08cbb4b-metrics-certs podName:352b5b4c-c640-4300-abf3-aa70b08cbb4b nodeName:}" failed. No retries permitted until 2026-10-30 21:28:31.265872725 +0000 UTC m=+66.146935669 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/352b5b4c-c640-4300-abf3-aa70b08cbb4b-metrics-certs") pod "network-metrics-daemon-vwxrd" (UID: "352b5b4c-c640-4300-abf3-aa70b08cbb4b") : secret "metrics-daemon-secret" not found
Oct 30 21:27:59.266038 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:59.265932    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9b4f9524-db75-4206-aa76-04400d95706f-service-ca-bundle podName:9b4f9524-db75-4206-aa76-04400d95706f nodeName:}" failed. No retries permitted until 2026-10-30 21:28:03.265926161 +0000 UTC m=+38.146989105 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9b4f9524-db75-4206-aa76-04400d95706f-service-ca-bundle") pod "router-default-5bbfd9465-x28lh" (UID: "9b4f9524-db75-4206-aa76-04400d95706f") : configmap references non-existent config key: service-ca.crt
Oct 30 21:27:59.366101 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:59.366058    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f171661-33a2-4631-81c7-bc10a5db791f-original-pull-secret\") pod \"global-pull-secret-syncer-csbxh\" (UID: \"5f171661-33a2-4631-81c7-bc10a5db791f\") " pod="kube-system/global-pull-secret-syncer-csbxh"
Oct 30 21:27:59.366294 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:59.366133    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08498890-fb63-4aea-9cf7-a2ade5397df2-metrics-tls\") pod \"dns-default-sbhch\" (UID: \"08498890-fb63-4aea-9cf7-a2ade5397df2\") " pod="openshift-dns/dns-default-sbhch"
Oct 30 21:27:59.366294 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:59.366224    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/94ee3757-5440-42e1-8617-2a68744ef3f4-registry-tls\") pod \"image-registry-86f87fb58b-5clbv\" (UID: \"94ee3757-5440-42e1-8617-2a68744ef3f4\") " pod="openshift-image-registry/image-registry-86f87fb58b-5clbv"
Oct 30 21:27:59.366294 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:59.366260    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9f54b2c6-b1f9-42da-abdc-275fc8827b71-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-9qppl\" (UID: \"9f54b2c6-b1f9-42da-abdc-275fc8827b71\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-9qppl"
Oct 30 21:27:59.366294 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:59.366288    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ms8rv\" (UniqueName: \"kubernetes.io/projected/3913f7d4-86cb-4e86-af5a-debd8b859252-kube-api-access-ms8rv\") pod \"network-check-target-4cvlt\" (UID: \"3913f7d4-86cb-4e86-af5a-debd8b859252\") " pod="openshift-network-diagnostics/network-check-target-4cvlt"
Oct 30 21:27:59.366978 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:59.366935    2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 21:27:59.366978 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:59.366976    2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 21:27:59.367135 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:59.367000    2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-86f87fb58b-5clbv: secret "image-registry-tls" not found
Oct 30 21:27:59.367135 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:59.367007    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08498890-fb63-4aea-9cf7-a2ade5397df2-metrics-tls podName:08498890-fb63-4aea-9cf7-a2ade5397df2 nodeName:}" failed. No retries permitted until 2026-10-30 21:28:03.366985105 +0000 UTC m=+38.248048040 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/08498890-fb63-4aea-9cf7-a2ade5397df2-metrics-tls") pod "dns-default-sbhch" (UID: "08498890-fb63-4aea-9cf7-a2ade5397df2") : secret "dns-default-metrics-tls" not found
Oct 30 21:27:59.367135 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:59.367056    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/94ee3757-5440-42e1-8617-2a68744ef3f4-registry-tls podName:94ee3757-5440-42e1-8617-2a68744ef3f4 nodeName:}" failed. No retries permitted until 2026-10-30 21:28:03.367035869 +0000 UTC m=+38.248098805 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/94ee3757-5440-42e1-8617-2a68744ef3f4-registry-tls") pod "image-registry-86f87fb58b-5clbv" (UID: "94ee3757-5440-42e1-8617-2a68744ef3f4") : secret "image-registry-tls" not found
Oct 30 21:27:59.367135 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:59.367129    2573 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 21:27:59.367340 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:27:59.367168    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f54b2c6-b1f9-42da-abdc-275fc8827b71-samples-operator-tls podName:9f54b2c6-b1f9-42da-abdc-275fc8827b71 nodeName:}" failed. No retries permitted until 2026-10-30 21:28:03.367155297 +0000 UTC m=+38.248218231 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9f54b2c6-b1f9-42da-abdc-275fc8827b71-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-9qppl" (UID: "9f54b2c6-b1f9-42da-abdc-275fc8827b71") : secret "samples-operator-tls" not found
Oct 30 21:27:59.378251 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:59.378226    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ms8rv\" (UniqueName: \"kubernetes.io/projected/3913f7d4-86cb-4e86-af5a-debd8b859252-kube-api-access-ms8rv\") pod \"network-check-target-4cvlt\" (UID: \"3913f7d4-86cb-4e86-af5a-debd8b859252\") " pod="openshift-network-diagnostics/network-check-target-4cvlt"
Oct 30 21:27:59.379200 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:59.379082    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f171661-33a2-4631-81c7-bc10a5db791f-original-pull-secret\") pod \"global-pull-secret-syncer-csbxh\" (UID: \"5f171661-33a2-4631-81c7-bc10a5db791f\") " pod="kube-system/global-pull-secret-syncer-csbxh"
Oct 30 21:27:59.383520 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:59.383495    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4cvlt"
Oct 30 21:27:59.463307 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:59.463272    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-csbxh"
Oct 30 21:27:59.737296 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:59.736370    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4cvlt"]
Oct 30 21:27:59.755560 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:59.755520    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3913f7d4_86cb_4e86_af5a_debd8b859252.slice/crio-7832534c8c7e80d5ba345d1e86c0d75998d77b27c1f4fba9dba1561978a6523b WatchSource:0}: Error finding container 7832534c8c7e80d5ba345d1e86c0d75998d77b27c1f4fba9dba1561978a6523b: Status 404 returned error can't find the container with id 7832534c8c7e80d5ba345d1e86c0d75998d77b27c1f4fba9dba1561978a6523b
Oct 30 21:27:59.794426 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:27:59.794283    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-csbxh"]
Oct 30 21:27:59.800406 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:27:59.800342    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f171661_33a2_4631_81c7_bc10a5db791f.slice/crio-bb1ff751df6656db7e8fef3bfd7a5baad5f96b46c870437d27222dd412553064 WatchSource:0}: Error finding container bb1ff751df6656db7e8fef3bfd7a5baad5f96b46c870437d27222dd412553064: Status 404 returned error can't find the container with id bb1ff751df6656db7e8fef3bfd7a5baad5f96b46c870437d27222dd412553064
Oct 30 21:28:00.010262 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:00.010187    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4cvlt" event={"ID":"3913f7d4-86cb-4e86-af5a-debd8b859252","Type":"ContainerStarted","Data":"7832534c8c7e80d5ba345d1e86c0d75998d77b27c1f4fba9dba1561978a6523b"}
Oct 30 21:28:00.012736 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:00.012627    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-csbxh" event={"ID":"5f171661-33a2-4631-81c7-bc10a5db791f","Type":"ContainerStarted","Data":"bb1ff751df6656db7e8fef3bfd7a5baad5f96b46c870437d27222dd412553064"}
Oct 30 21:28:01.018502 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:01.018466    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-9p8mb" event={"ID":"8bb24a0d-3af0-4a05-bba4-4f9630b04bf3","Type":"ContainerStarted","Data":"4b323db09cec9d37c1aea20e981dcafa4bed4684d8d34a1c16d709bb371445dc"}
Oct 30 21:28:01.040207 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:01.040143    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-9p8mb" podStartSLOduration=329.425210967 podStartE2EDuration="5m34.04012311s" totalImagesPullingTime="4.614912143s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:22:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:28:01.038457823 +0000 UTC m=+35.919520777" watchObservedRunningTime="2026-10-30 21:28:01.04012311 +0000 UTC m=+35.921186090"
Oct 30 21:28:01.824503 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:01.823836    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-l7nbh"]
Oct 30 21:28:01.852328 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:01.852301    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-gqdlr"]
Oct 30 21:28:01.852719 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:01.852692    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-l7nbh"
Oct 30 21:28:01.862854 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:01.862811    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Oct 30 21:28:01.863343 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:01.863236    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Oct 30 21:28:01.863343 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:01.863322    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-lz8sp\""
Oct 30 21:28:01.884554 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:01.884512    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-l7nbh"]
Oct 30 21:28:01.884687 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:01.884563    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-gqdlr"]
Oct 30 21:28:01.884687 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:01.884666    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-gqdlr"
Oct 30 21:28:01.996245 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:01.995952    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhrs4\" (UniqueName: \"kubernetes.io/projected/faca011a-4ff1-4de0-9af1-ed9b0f253aeb-kube-api-access-dhrs4\") pod \"migrator-66df5d496b-l7nbh\" (UID: \"faca011a-4ff1-4de0-9af1-ed9b0f253aeb\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-l7nbh"
Oct 30 21:28:01.996245 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:01.996084    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5qjc\" (UniqueName: \"kubernetes.io/projected/06a0877a-2fce-405f-b97c-091d88e19543-kube-api-access-l5qjc\") pod \"migrator-66df5d496b-gqdlr\" (UID: \"06a0877a-2fce-405f-b97c-091d88e19543\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-gqdlr"
Oct 30 21:28:02.025404 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:02.025306    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-7cgk9" event={"ID":"d6ca21b9-20a4-4887-9d08-19d9c491a09e","Type":"ContainerStarted","Data":"6c092a58968d74d97f4617cfb9ba42b31f18f329717a98a66041a041254a53f1"}
Oct 30 21:28:02.029176 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:02.029153    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-sklxd_67292bdc-7fe1-449d-af75-2baa90c16ea9/console-operator/0.log"
Oct 30 21:28:02.029323 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:02.029190    2573 generic.go:356] "Generic (PLEG): container finished" podID="67292bdc-7fe1-449d-af75-2baa90c16ea9" containerID="3d55f0ba701d905d70aaf978cffcb6e44fbede460643f48174f19eae7d69f3c1" exitCode=255
Oct 30 21:28:02.029323 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:02.029263    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-sklxd" event={"ID":"67292bdc-7fe1-449d-af75-2baa90c16ea9","Type":"ContainerDied","Data":"3d55f0ba701d905d70aaf978cffcb6e44fbede460643f48174f19eae7d69f3c1"}
Oct 30 21:28:02.029706 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:02.029683    2573 scope.go:122] "RemoveContainer" containerID="3d55f0ba701d905d70aaf978cffcb6e44fbede460643f48174f19eae7d69f3c1"
Oct 30 21:28:02.033070 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:02.033046    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-9r5sb" event={"ID":"8873f9b9-2ac5-409c-9299-fa024e00189c","Type":"ContainerStarted","Data":"e332f7c86412ccecce5e050bcc3cbe8ebb64334a0ce1c1d94cca07e381a2cb5f"}
Oct 30 21:28:02.037570 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:02.037549    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hzvrj" event={"ID":"3175ff47-1038-4139-9c0a-87062bb38dc2","Type":"ContainerStarted","Data":"bce06fd00f11e52ff83d82085706fbf11e1939533ba65869a4c9701b164e4015"}
Oct 30 21:28:02.057607 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:02.050390    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-operator-7c4986d449-7cgk9" podStartSLOduration=330.366748613 podStartE2EDuration="5m35.050373686s" totalImagesPullingTime="4.683625073s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:22:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:28:02.049468076 +0000 UTC m=+36.930531032" watchObservedRunningTime="2026-10-30 21:28:02.050373686 +0000 UTC m=+36.931436633"
Oct 30 21:28:02.072918 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:02.072756    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-9r5sb" podStartSLOduration=330.010781659 podStartE2EDuration="5m35.072738658s" totalImagesPullingTime="5.061956999s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:22:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:28:02.069960987 +0000 UTC m=+36.951023943" watchObservedRunningTime="2026-10-30 21:28:02.072738658 +0000 UTC m=+36.953801617"
Oct 30 21:28:02.096961 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:02.096934    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhrs4\" (UniqueName: \"kubernetes.io/projected/faca011a-4ff1-4de0-9af1-ed9b0f253aeb-kube-api-access-dhrs4\") pod \"migrator-66df5d496b-l7nbh\" (UID: \"faca011a-4ff1-4de0-9af1-ed9b0f253aeb\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-l7nbh"
Oct 30 21:28:02.099840 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:02.099819    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5qjc\" (UniqueName: \"kubernetes.io/projected/06a0877a-2fce-405f-b97c-091d88e19543-kube-api-access-l5qjc\") pod \"migrator-66df5d496b-gqdlr\" (UID: \"06a0877a-2fce-405f-b97c-091d88e19543\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-gqdlr"
Oct 30 21:28:02.104635 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:02.099682    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-hzvrj" podStartSLOduration=1.6429299259999999 podStartE2EDuration="7.099664866s" totalImagesPullingTime="5.45673494s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:27:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:28:02.099263782 +0000 UTC m=+36.980326738" watchObservedRunningTime="2026-10-30 21:28:02.099664866 +0000 UTC m=+36.980727825"
Oct 30 21:28:02.116926 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:02.116902    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhrs4\" (UniqueName: \"kubernetes.io/projected/faca011a-4ff1-4de0-9af1-ed9b0f253aeb-kube-api-access-dhrs4\") pod \"migrator-66df5d496b-l7nbh\" (UID: \"faca011a-4ff1-4de0-9af1-ed9b0f253aeb\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-l7nbh"
Oct 30 21:28:02.122449 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:02.122421    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5qjc\" (UniqueName: \"kubernetes.io/projected/06a0877a-2fce-405f-b97c-091d88e19543-kube-api-access-l5qjc\") pod \"migrator-66df5d496b-gqdlr\" (UID: \"06a0877a-2fce-405f-b97c-091d88e19543\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-gqdlr"
Oct 30 21:28:02.196262 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:02.196053    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-l7nbh"
Oct 30 21:28:02.219982 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:02.219632    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-gqdlr"
Oct 30 21:28:02.351145 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:02.351068    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-z2dd4"]
Oct 30 21:28:02.371468 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:02.371417    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-542ht-hjp6z"]
Oct 30 21:28:02.378088 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:02.374011    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-z2dd4"
Oct 30 21:28:02.378669 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:02.378601    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 21:28:02.378799 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:02.378672    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 21:28:02.378799 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:02.378688    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-2wf4q\""
Oct 30 21:28:02.389674 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:02.389243    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-z2dd4"]
Oct 30 21:28:02.389674 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:02.389299    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-542ht-hjp6z"]
Oct 30 21:28:02.389674 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:02.389404    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-542ht-hjp6z"
Oct 30 21:28:02.403813 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:02.403195    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnx5t\" (UniqueName: \"kubernetes.io/projected/fd34a723-6c9e-48f1-ba7d-924d542b2bc7-kube-api-access-fnx5t\") pod \"periodic-gathering-542ht-hjp6z\" (UID: \"fd34a723-6c9e-48f1-ba7d-924d542b2bc7\") " pod="openshift-insights/periodic-gathering-542ht-hjp6z"
Oct 30 21:28:02.403813 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:02.403267    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0bbc8570-4947-455a-b7f8-8e9fa8e023b3-crio-socket\") pod \"insights-runtime-extractor-z2dd4\" (UID: \"0bbc8570-4947-455a-b7f8-8e9fa8e023b3\") " pod="openshift-insights/insights-runtime-extractor-z2dd4"
Oct 30 21:28:02.403813 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:02.403293    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd34a723-6c9e-48f1-ba7d-924d542b2bc7-trusted-ca-bundle\") pod \"periodic-gathering-542ht-hjp6z\" (UID: \"fd34a723-6c9e-48f1-ba7d-924d542b2bc7\") " pod="openshift-insights/periodic-gathering-542ht-hjp6z"
Oct 30 21:28:02.403813 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:02.403323    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd34a723-6c9e-48f1-ba7d-924d542b2bc7-service-ca-bundle\") pod \"periodic-gathering-542ht-hjp6z\" (UID: \"fd34a723-6c9e-48f1-ba7d-924d542b2bc7\") " pod="openshift-insights/periodic-gathering-542ht-hjp6z"
Oct 30 21:28:02.403813 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:02.403462    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0bbc8570-4947-455a-b7f8-8e9fa8e023b3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-z2dd4\" (UID: \"0bbc8570-4947-455a-b7f8-8e9fa8e023b3\") " pod="openshift-insights/insights-runtime-extractor-z2dd4"
Oct 30 21:28:02.403813 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:02.403501    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qf6w\" (UniqueName: \"kubernetes.io/projected/0bbc8570-4947-455a-b7f8-8e9fa8e023b3-kube-api-access-9qf6w\") pod \"insights-runtime-extractor-z2dd4\" (UID: \"0bbc8570-4947-455a-b7f8-8e9fa8e023b3\") " pod="openshift-insights/insights-runtime-extractor-z2dd4"
Oct 30 21:28:02.403813 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:02.403529    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/fd34a723-6c9e-48f1-ba7d-924d542b2bc7-archives-path\") pod \"periodic-gathering-542ht-hjp6z\" (UID: \"fd34a723-6c9e-48f1-ba7d-924d542b2bc7\") " pod="openshift-insights/periodic-gathering-542ht-hjp6z"
Oct 30 21:28:02.403813 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:02.403556    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0bbc8570-4947-455a-b7f8-8e9fa8e023b3-data-volume\") pod \"insights-runtime-extractor-z2dd4\" (UID: \"0bbc8570-4947-455a-b7f8-8e9fa8e023b3\") " pod="openshift-insights/insights-runtime-extractor-z2dd4"
Oct 30 21:28:02.403813 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:02.403599    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0bbc8570-4947-455a-b7f8-8e9fa8e023b3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z2dd4\" (UID: \"0bbc8570-4947-455a-b7f8-8e9fa8e023b3\") " pod="openshift-insights/insights-runtime-extractor-z2dd4"
Oct 30 21:28:02.505107 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:02.504960    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd34a723-6c9e-48f1-ba7d-924d542b2bc7-trusted-ca-bundle\") pod \"periodic-gathering-542ht-hjp6z\" (UID: \"fd34a723-6c9e-48f1-ba7d-924d542b2bc7\") " pod="openshift-insights/periodic-gathering-542ht-hjp6z"
Oct 30 21:28:02.505107 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:02.504998    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd34a723-6c9e-48f1-ba7d-924d542b2bc7-service-ca-bundle\") pod \"periodic-gathering-542ht-hjp6z\" (UID: \"fd34a723-6c9e-48f1-ba7d-924d542b2bc7\") " pod="openshift-insights/periodic-gathering-542ht-hjp6z"
Oct 30 21:28:02.505107 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:02.505085    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0bbc8570-4947-455a-b7f8-8e9fa8e023b3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-z2dd4\" (UID: \"0bbc8570-4947-455a-b7f8-8e9fa8e023b3\") " pod="openshift-insights/insights-runtime-extractor-z2dd4"
Oct 30 21:28:02.505107 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:02.505106    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qf6w\" (UniqueName: \"kubernetes.io/projected/0bbc8570-4947-455a-b7f8-8e9fa8e023b3-kube-api-access-9qf6w\") pod \"insights-runtime-extractor-z2dd4\" (UID: \"0bbc8570-4947-455a-b7f8-8e9fa8e023b3\") " pod="openshift-insights/insights-runtime-extractor-z2dd4"
Oct 30 21:28:02.505432 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:02.505125    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/fd34a723-6c9e-48f1-ba7d-924d542b2bc7-archives-path\") pod \"periodic-gathering-542ht-hjp6z\" (UID: \"fd34a723-6c9e-48f1-ba7d-924d542b2bc7\") " pod="openshift-insights/periodic-gathering-542ht-hjp6z"
Oct 30 21:28:02.505432 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:02.505140    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0bbc8570-4947-455a-b7f8-8e9fa8e023b3-data-volume\") pod \"insights-runtime-extractor-z2dd4\" (UID: \"0bbc8570-4947-455a-b7f8-8e9fa8e023b3\") " pod="openshift-insights/insights-runtime-extractor-z2dd4"
Oct 30 21:28:02.505432 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:02.505163    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0bbc8570-4947-455a-b7f8-8e9fa8e023b3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z2dd4\" (UID: \"0bbc8570-4947-455a-b7f8-8e9fa8e023b3\") " pod="openshift-insights/insights-runtime-extractor-z2dd4"
Oct 30 21:28:02.505432 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:02.505212    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnx5t\" (UniqueName: \"kubernetes.io/projected/fd34a723-6c9e-48f1-ba7d-924d542b2bc7-kube-api-access-fnx5t\") pod \"periodic-gathering-542ht-hjp6z\" (UID: \"fd34a723-6c9e-48f1-ba7d-924d542b2bc7\") " pod="openshift-insights/periodic-gathering-542ht-hjp6z"
Oct 30 21:28:02.505432 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:02.505244    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0bbc8570-4947-455a-b7f8-8e9fa8e023b3-crio-socket\") pod \"insights-runtime-extractor-z2dd4\" (UID: \"0bbc8570-4947-455a-b7f8-8e9fa8e023b3\") " pod="openshift-insights/insights-runtime-extractor-z2dd4"
Oct 30 21:28:02.505432 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:02.505320    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0bbc8570-4947-455a-b7f8-8e9fa8e023b3-crio-socket\") pod \"insights-runtime-extractor-z2dd4\" (UID: \"0bbc8570-4947-455a-b7f8-8e9fa8e023b3\") " pod="openshift-insights/insights-runtime-extractor-z2dd4"
Oct 30 21:28:02.505965 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:02.505865    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/fd34a723-6c9e-48f1-ba7d-924d542b2bc7-archives-path\") pod \"periodic-gathering-542ht-hjp6z\" (UID: \"fd34a723-6c9e-48f1-ba7d-924d542b2bc7\") " pod="openshift-insights/periodic-gathering-542ht-hjp6z"
Oct 30 21:28:02.506881 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:02.506246    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd34a723-6c9e-48f1-ba7d-924d542b2bc7-trusted-ca-bundle\") pod \"periodic-gathering-542ht-hjp6z\" (UID: \"fd34a723-6c9e-48f1-ba7d-924d542b2bc7\") " pod="openshift-insights/periodic-gathering-542ht-hjp6z"
Oct 30 21:28:02.506881 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:02.506387    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd34a723-6c9e-48f1-ba7d-924d542b2bc7-service-ca-bundle\") pod \"periodic-gathering-542ht-hjp6z\" (UID: \"fd34a723-6c9e-48f1-ba7d-924d542b2bc7\") " pod="openshift-insights/periodic-gathering-542ht-hjp6z"
Oct 30 21:28:02.506881 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:28:02.506460    2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 21:28:02.506881 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:28:02.506523    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0bbc8570-4947-455a-b7f8-8e9fa8e023b3-kube-rbac-proxy-cm podName:0bbc8570-4947-455a-b7f8-8e9fa8e023b3 nodeName:}" failed. No retries permitted until 2026-10-30 21:28:03.006501179 +0000 UTC m=+37.887564124 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0bbc8570-4947-455a-b7f8-8e9fa8e023b3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-z2dd4" (UID: "0bbc8570-4947-455a-b7f8-8e9fa8e023b3") : configmap "kube-rbac-proxy" not found
Oct 30 21:28:02.506881 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:02.506542    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0bbc8570-4947-455a-b7f8-8e9fa8e023b3-data-volume\") pod \"insights-runtime-extractor-z2dd4\" (UID: \"0bbc8570-4947-455a-b7f8-8e9fa8e023b3\") " pod="openshift-insights/insights-runtime-extractor-z2dd4"
Oct 30 21:28:02.506881 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:28:02.506628    2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 21:28:02.506881 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:28:02.506669    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0bbc8570-4947-455a-b7f8-8e9fa8e023b3-insights-runtime-extractor-tls podName:0bbc8570-4947-455a-b7f8-8e9fa8e023b3 nodeName:}" failed. No retries permitted until 2026-10-30 21:28:03.006654932 +0000 UTC m=+37.887717867 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0bbc8570-4947-455a-b7f8-8e9fa8e023b3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-z2dd4" (UID: "0bbc8570-4947-455a-b7f8-8e9fa8e023b3") : secret "insights-runtime-extractor-tls" not found
Oct 30 21:28:02.522647 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:02.522590    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnx5t\" (UniqueName: \"kubernetes.io/projected/fd34a723-6c9e-48f1-ba7d-924d542b2bc7-kube-api-access-fnx5t\") pod \"periodic-gathering-542ht-hjp6z\" (UID: \"fd34a723-6c9e-48f1-ba7d-924d542b2bc7\") " pod="openshift-insights/periodic-gathering-542ht-hjp6z"
Oct 30 21:28:02.528494 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:02.528461    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qf6w\" (UniqueName: \"kubernetes.io/projected/0bbc8570-4947-455a-b7f8-8e9fa8e023b3-kube-api-access-9qf6w\") pod \"insights-runtime-extractor-z2dd4\" (UID: \"0bbc8570-4947-455a-b7f8-8e9fa8e023b3\") " pod="openshift-insights/insights-runtime-extractor-z2dd4"
Oct 30 21:28:02.715735 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:02.715707    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-l7nbh"]
Oct 30 21:28:02.715912 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:28:02.715889    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfaca011a_4ff1_4de0_9af1_ed9b0f253aeb.slice/crio-a9272b555796f4e5bfa1e2b12cda99544643a87b628fa11b2b151adf30413d4c WatchSource:0}: Error finding container a9272b555796f4e5bfa1e2b12cda99544643a87b628fa11b2b151adf30413d4c: Status 404 returned error can't find the container with id a9272b555796f4e5bfa1e2b12cda99544643a87b628fa11b2b151adf30413d4c
Oct 30 21:28:02.717403 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:02.717380    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-gqdlr"]
Oct 30 21:28:02.723012 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:28:02.722988    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06a0877a_2fce_405f_b97c_091d88e19543.slice/crio-25200764cd35dbf351705da7ea4eb6473438cd86a25b56a86438f0becd6fa64c WatchSource:0}: Error finding container 25200764cd35dbf351705da7ea4eb6473438cd86a25b56a86438f0becd6fa64c: Status 404 returned error can't find the container with id 25200764cd35dbf351705da7ea4eb6473438cd86a25b56a86438f0becd6fa64c
Oct 30 21:28:02.814201 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:02.813984    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-542ht-hjp6z"
Oct 30 21:28:03.011240 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:03.011194    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0bbc8570-4947-455a-b7f8-8e9fa8e023b3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-z2dd4\" (UID: \"0bbc8570-4947-455a-b7f8-8e9fa8e023b3\") " pod="openshift-insights/insights-runtime-extractor-z2dd4"
Oct 30 21:28:03.011434 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:03.011289    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0bbc8570-4947-455a-b7f8-8e9fa8e023b3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z2dd4\" (UID: \"0bbc8570-4947-455a-b7f8-8e9fa8e023b3\") " pod="openshift-insights/insights-runtime-extractor-z2dd4"
Oct 30 21:28:03.011434 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:28:03.011324    2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 21:28:03.011434 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:28:03.011422    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0bbc8570-4947-455a-b7f8-8e9fa8e023b3-kube-rbac-proxy-cm podName:0bbc8570-4947-455a-b7f8-8e9fa8e023b3 nodeName:}" failed. No retries permitted until 2026-10-30 21:28:04.011405399 +0000 UTC m=+38.892468351 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0bbc8570-4947-455a-b7f8-8e9fa8e023b3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-z2dd4" (UID: "0bbc8570-4947-455a-b7f8-8e9fa8e023b3") : configmap "kube-rbac-proxy" not found
Oct 30 21:28:03.011434 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:28:03.011423    2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 21:28:03.011659 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:28:03.011487    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0bbc8570-4947-455a-b7f8-8e9fa8e023b3-insights-runtime-extractor-tls podName:0bbc8570-4947-455a-b7f8-8e9fa8e023b3 nodeName:}" failed. No retries permitted until 2026-10-30 21:28:04.011468918 +0000 UTC m=+38.892531872 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0bbc8570-4947-455a-b7f8-8e9fa8e023b3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-z2dd4" (UID: "0bbc8570-4947-455a-b7f8-8e9fa8e023b3") : secret "insights-runtime-extractor-tls" not found
Oct 30 21:28:03.045717 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:03.044405    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-wpm2q"]
Oct 30 21:28:03.049631 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:03.049606    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-sklxd_67292bdc-7fe1-449d-af75-2baa90c16ea9/console-operator/1.log"
Oct 30 21:28:03.050547 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:03.050527    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-sklxd_67292bdc-7fe1-449d-af75-2baa90c16ea9/console-operator/0.log"
Oct 30 21:28:03.050673 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:03.050569    2573 generic.go:356] "Generic (PLEG): container finished" podID="67292bdc-7fe1-449d-af75-2baa90c16ea9" containerID="f667ffa124949edf8581923668ce1139c1ad553b96aeaa80c8bc06474a49998a" exitCode=255
Oct 30 21:28:03.051000 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:03.050982    2573 scope.go:122] "RemoveContainer" containerID="f667ffa124949edf8581923668ce1139c1ad553b96aeaa80c8bc06474a49998a"
Oct 30 21:28:03.051450 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:28:03.051219    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-sklxd_openshift-console-operator(67292bdc-7fe1-449d-af75-2baa90c16ea9)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-sklxd" podUID="67292bdc-7fe1-449d-af75-2baa90c16ea9"
Oct 30 21:28:03.067926 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:03.067889    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-l7nbh" event={"ID":"faca011a-4ff1-4de0-9af1-ed9b0f253aeb","Type":"ContainerStarted","Data":"a9272b555796f4e5bfa1e2b12cda99544643a87b628fa11b2b151adf30413d4c"}
Oct 30 21:28:03.068089 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:03.067942    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-wpm2q"]
Oct 30 21:28:03.068089 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:03.067971    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-gqdlr" event={"ID":"06a0877a-2fce-405f-b97c-091d88e19543","Type":"ContainerStarted","Data":"25200764cd35dbf351705da7ea4eb6473438cd86a25b56a86438f0becd6fa64c"}
Oct 30 21:28:03.068089 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:03.067991    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-sklxd" event={"ID":"67292bdc-7fe1-449d-af75-2baa90c16ea9","Type":"ContainerDied","Data":"f667ffa124949edf8581923668ce1139c1ad553b96aeaa80c8bc06474a49998a"}
Oct 30 21:28:03.068089 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:03.068017    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-tdmc8" event={"ID":"bba1a087-6e04-4db8-ba82-7260a88e2843","Type":"ContainerStarted","Data":"4735e00c57f55f83832d4d7d17148bab324c9e8e1ac08610c99cdf81b2ea6163"}
Oct 30 21:28:03.068089 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:03.068048    2573 scope.go:122] "RemoveContainer" containerID="3d55f0ba701d905d70aaf978cffcb6e44fbede460643f48174f19eae7d69f3c1"
Oct 30 21:28:03.068355 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:03.068117    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-5fffc97bdf-wpm2q"
Oct 30 21:28:03.072968 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:03.072950    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Oct 30 21:28:03.073070 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:03.073023    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Oct 30 21:28:03.073136 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:03.073082    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Oct 30 21:28:03.073136 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:03.073128    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Oct 30 21:28:03.073241 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:03.073175    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-56zgn\""
Oct 30 21:28:03.114017 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:03.111419    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-tdmc8" podStartSLOduration=323.985804737 podStartE2EDuration="5m30.111400218s" totalImagesPullingTime="6.125595481s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:22:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:28:03.111100159 +0000 UTC m=+37.992163112" watchObservedRunningTime="2026-10-30 21:28:03.111400218 +0000 UTC m=+37.992463176"
Oct 30 21:28:03.114017 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:03.112420    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8f74s\" (UniqueName: \"kubernetes.io/projected/11b44bf1-b3b1-4214-b7b4-4e508c618cd1-kube-api-access-8f74s\") pod \"service-ca-5fffc97bdf-wpm2q\" (UID: \"11b44bf1-b3b1-4214-b7b4-4e508c618cd1\") " pod="openshift-service-ca/service-ca-5fffc97bdf-wpm2q"
Oct 30 21:28:03.114017 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:03.112591    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/11b44bf1-b3b1-4214-b7b4-4e508c618cd1-signing-key\") pod \"service-ca-5fffc97bdf-wpm2q\" (UID: \"11b44bf1-b3b1-4214-b7b4-4e508c618cd1\") " pod="openshift-service-ca/service-ca-5fffc97bdf-wpm2q"
Oct 30 21:28:03.114017 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:03.112629    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/11b44bf1-b3b1-4214-b7b4-4e508c618cd1-signing-cabundle\") pod \"service-ca-5fffc97bdf-wpm2q\" (UID: \"11b44bf1-b3b1-4214-b7b4-4e508c618cd1\") " pod="openshift-service-ca/service-ca-5fffc97bdf-wpm2q"
Oct 30 21:28:03.176012 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:03.175969    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-542ht-hjp6z"]
Oct 30 21:28:03.185506 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:28:03.185478    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd34a723_6c9e_48f1_ba7d_924d542b2bc7.slice/crio-1dca56b0734ec47a566bcaede4782a3c941f7018b609beabda556ecedc01fe47 WatchSource:0}: Error finding container 1dca56b0734ec47a566bcaede4782a3c941f7018b609beabda556ecedc01fe47: Status 404 returned error can't find the container with id 1dca56b0734ec47a566bcaede4782a3c941f7018b609beabda556ecedc01fe47
Oct 30 21:28:03.213428 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:03.213405    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/11b44bf1-b3b1-4214-b7b4-4e508c618cd1-signing-cabundle\") pod \"service-ca-5fffc97bdf-wpm2q\" (UID: \"11b44bf1-b3b1-4214-b7b4-4e508c618cd1\") " pod="openshift-service-ca/service-ca-5fffc97bdf-wpm2q"
Oct 30 21:28:03.213608 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:03.213513    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8f74s\" (UniqueName: \"kubernetes.io/projected/11b44bf1-b3b1-4214-b7b4-4e508c618cd1-kube-api-access-8f74s\") pod \"service-ca-5fffc97bdf-wpm2q\" (UID: \"11b44bf1-b3b1-4214-b7b4-4e508c618cd1\") " pod="openshift-service-ca/service-ca-5fffc97bdf-wpm2q"
Oct 30 21:28:03.213690 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:03.213669    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/11b44bf1-b3b1-4214-b7b4-4e508c618cd1-signing-key\") pod \"service-ca-5fffc97bdf-wpm2q\" (UID: \"11b44bf1-b3b1-4214-b7b4-4e508c618cd1\") " pod="openshift-service-ca/service-ca-5fffc97bdf-wpm2q"
Oct 30 21:28:03.214156 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:03.214130    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/11b44bf1-b3b1-4214-b7b4-4e508c618cd1-signing-cabundle\") pod \"service-ca-5fffc97bdf-wpm2q\" (UID: \"11b44bf1-b3b1-4214-b7b4-4e508c618cd1\") " pod="openshift-service-ca/service-ca-5fffc97bdf-wpm2q"
Oct 30 21:28:03.216132 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:03.216101    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/11b44bf1-b3b1-4214-b7b4-4e508c618cd1-signing-key\") pod \"service-ca-5fffc97bdf-wpm2q\" (UID: \"11b44bf1-b3b1-4214-b7b4-4e508c618cd1\") " pod="openshift-service-ca/service-ca-5fffc97bdf-wpm2q"
Oct 30 21:28:03.227602 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:03.227553    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8f74s\" (UniqueName: \"kubernetes.io/projected/11b44bf1-b3b1-4214-b7b4-4e508c618cd1-kube-api-access-8f74s\") pod \"service-ca-5fffc97bdf-wpm2q\" (UID: \"11b44bf1-b3b1-4214-b7b4-4e508c618cd1\") " pod="openshift-service-ca/service-ca-5fffc97bdf-wpm2q"
Oct 30 21:28:03.314879 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:03.314829    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b4f9524-db75-4206-aa76-04400d95706f-service-ca-bundle\") pod \"router-default-5bbfd9465-x28lh\" (UID: \"9b4f9524-db75-4206-aa76-04400d95706f\") " pod="openshift-ingress/router-default-5bbfd9465-x28lh"
Oct 30 21:28:03.315056 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:03.314892    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/753d4639-09e6-4850-9f4d-ed39b0406439-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-lwm89\" (UID: \"753d4639-09e6-4850-9f4d-ed39b0406439\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-lwm89"
Oct 30 21:28:03.315056 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:03.314943    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b4f9524-db75-4206-aa76-04400d95706f-metrics-certs\") pod \"router-default-5bbfd9465-x28lh\" (UID: \"9b4f9524-db75-4206-aa76-04400d95706f\") " pod="openshift-ingress/router-default-5bbfd9465-x28lh"
Oct 30 21:28:03.315056 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:03.314967    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/90937bc9-0051-407d-9c82-014f0a9c65d4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-kdrrd\" (UID: \"90937bc9-0051-407d-9c82-014f0a9c65d4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-kdrrd"
Oct 30 21:28:03.315056 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:03.314984    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/039cf439-d12d-450a-80c1-eb3efeb50e8a-registry-tls\") pod \"image-registry-76679b6db7-ltmt6\" (UID: \"039cf439-d12d-450a-80c1-eb3efeb50e8a\") " pod="openshift-image-registry/image-registry-76679b6db7-ltmt6"
Oct 30 21:28:03.315246 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:28:03.315139    2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 21:28:03.315246 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:28:03.315149    2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76679b6db7-ltmt6: secret "image-registry-tls" not found
Oct 30 21:28:03.315246 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:28:03.315196    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/039cf439-d12d-450a-80c1-eb3efeb50e8a-registry-tls podName:039cf439-d12d-450a-80c1-eb3efeb50e8a nodeName:}" failed. No retries permitted until 2026-10-30 21:28:11.315178622 +0000 UTC m=+46.196241567 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/039cf439-d12d-450a-80c1-eb3efeb50e8a-registry-tls") pod "image-registry-76679b6db7-ltmt6" (UID: "039cf439-d12d-450a-80c1-eb3efeb50e8a") : secret "image-registry-tls" not found
Oct 30 21:28:03.315582 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:28:03.315565    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9b4f9524-db75-4206-aa76-04400d95706f-service-ca-bundle podName:9b4f9524-db75-4206-aa76-04400d95706f nodeName:}" failed. No retries permitted until 2026-10-30 21:28:11.315548917 +0000 UTC m=+46.196611851 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9b4f9524-db75-4206-aa76-04400d95706f-service-ca-bundle") pod "router-default-5bbfd9465-x28lh" (UID: "9b4f9524-db75-4206-aa76-04400d95706f") : configmap references non-existent config key: service-ca.crt
Oct 30 21:28:03.315652 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:28:03.315640    2573 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 21:28:03.315701 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:28:03.315677    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/753d4639-09e6-4850-9f4d-ed39b0406439-networking-console-plugin-cert podName:753d4639-09e6-4850-9f4d-ed39b0406439 nodeName:}" failed. No retries permitted until 2026-10-30 21:28:11.315665608 +0000 UTC m=+46.196728541 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/753d4639-09e6-4850-9f4d-ed39b0406439-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-lwm89" (UID: "753d4639-09e6-4850-9f4d-ed39b0406439") : secret "networking-console-plugin-cert" not found
Oct 30 21:28:03.315752 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:28:03.315727    2573 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 21:28:03.315801 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:28:03.315756    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b4f9524-db75-4206-aa76-04400d95706f-metrics-certs podName:9b4f9524-db75-4206-aa76-04400d95706f nodeName:}" failed. No retries permitted until 2026-10-30 21:28:11.315746893 +0000 UTC m=+46.196809839 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9b4f9524-db75-4206-aa76-04400d95706f-metrics-certs") pod "router-default-5bbfd9465-x28lh" (UID: "9b4f9524-db75-4206-aa76-04400d95706f") : secret "router-metrics-certs-default" not found
Oct 30 21:28:03.315855 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:28:03.315799    2573 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 30 21:28:03.315855 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:28:03.315821    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90937bc9-0051-407d-9c82-014f0a9c65d4-cluster-monitoring-operator-tls podName:90937bc9-0051-407d-9c82-014f0a9c65d4 nodeName:}" failed. No retries permitted until 2026-10-30 21:28:11.315815587 +0000 UTC m=+46.196878530 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/90937bc9-0051-407d-9c82-014f0a9c65d4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-kdrrd" (UID: "90937bc9-0051-407d-9c82-014f0a9c65d4") : secret "cluster-monitoring-operator-tls" not found
Oct 30 21:28:03.393724 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:03.393447    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-5fffc97bdf-wpm2q"
Oct 30 21:28:03.416040 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:03.416008    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08498890-fb63-4aea-9cf7-a2ade5397df2-metrics-tls\") pod \"dns-default-sbhch\" (UID: \"08498890-fb63-4aea-9cf7-a2ade5397df2\") " pod="openshift-dns/dns-default-sbhch"
Oct 30 21:28:03.416218 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:03.416199    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/94ee3757-5440-42e1-8617-2a68744ef3f4-registry-tls\") pod \"image-registry-86f87fb58b-5clbv\" (UID: \"94ee3757-5440-42e1-8617-2a68744ef3f4\") " pod="openshift-image-registry/image-registry-86f87fb58b-5clbv"
Oct 30 21:28:03.416281 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:28:03.416224    2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 21:28:03.416281 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:03.416242    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9f54b2c6-b1f9-42da-abdc-275fc8827b71-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-9qppl\" (UID: \"9f54b2c6-b1f9-42da-abdc-275fc8827b71\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-9qppl"
Oct 30 21:28:03.416421 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:28:03.416292    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08498890-fb63-4aea-9cf7-a2ade5397df2-metrics-tls podName:08498890-fb63-4aea-9cf7-a2ade5397df2 nodeName:}" failed. No retries permitted until 2026-10-30 21:28:11.416269041 +0000 UTC m=+46.297331976 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/08498890-fb63-4aea-9cf7-a2ade5397df2-metrics-tls") pod "dns-default-sbhch" (UID: "08498890-fb63-4aea-9cf7-a2ade5397df2") : secret "dns-default-metrics-tls" not found
Oct 30 21:28:03.416421 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:28:03.416344    2573 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 21:28:03.416421 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:28:03.416405    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f54b2c6-b1f9-42da-abdc-275fc8827b71-samples-operator-tls podName:9f54b2c6-b1f9-42da-abdc-275fc8827b71 nodeName:}" failed. No retries permitted until 2026-10-30 21:28:11.416391832 +0000 UTC m=+46.297454766 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9f54b2c6-b1f9-42da-abdc-275fc8827b71-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-9qppl" (UID: "9f54b2c6-b1f9-42da-abdc-275fc8827b71") : secret "samples-operator-tls" not found
Oct 30 21:28:03.416584 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:28:03.416464    2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 21:28:03.416584 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:28:03.416474    2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-86f87fb58b-5clbv: secret "image-registry-tls" not found
Oct 30 21:28:03.416584 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:28:03.416519    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/94ee3757-5440-42e1-8617-2a68744ef3f4-registry-tls podName:94ee3757-5440-42e1-8617-2a68744ef3f4 nodeName:}" failed. No retries permitted until 2026-10-30 21:28:11.416508843 +0000 UTC m=+46.297571796 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/94ee3757-5440-42e1-8617-2a68744ef3f4-registry-tls") pod "image-registry-86f87fb58b-5clbv" (UID: "94ee3757-5440-42e1-8617-2a68744ef3f4") : secret "image-registry-tls" not found
Oct 30 21:28:03.718420 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:03.718345    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-wpm2q"]
Oct 30 21:28:03.723461 ip-10-0-129-195 kubenswrapper[2573]: W1030 21:28:03.723425    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11b44bf1_b3b1_4214_b7b4_4e508c618cd1.slice/crio-5eceb5374b97fe1e5b74255fdff647f9a544bdccc3e50a91d984a973c5a74612 WatchSource:0}: Error finding container 5eceb5374b97fe1e5b74255fdff647f9a544bdccc3e50a91d984a973c5a74612: Status 404 returned error can't find the container with id 5eceb5374b97fe1e5b74255fdff647f9a544bdccc3e50a91d984a973c5a74612
Oct 30 21:28:04.024697 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:04.023916    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0bbc8570-4947-455a-b7f8-8e9fa8e023b3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-z2dd4\" (UID: \"0bbc8570-4947-455a-b7f8-8e9fa8e023b3\") " pod="openshift-insights/insights-runtime-extractor-z2dd4"
Oct 30 21:28:04.024697 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:04.023985    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0bbc8570-4947-455a-b7f8-8e9fa8e023b3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z2dd4\" (UID: \"0bbc8570-4947-455a-b7f8-8e9fa8e023b3\") " pod="openshift-insights/insights-runtime-extractor-z2dd4"
Oct 30 21:28:04.024697 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:28:04.024188    2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 21:28:04.024697 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:28:04.024253    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0bbc8570-4947-455a-b7f8-8e9fa8e023b3-insights-runtime-extractor-tls podName:0bbc8570-4947-455a-b7f8-8e9fa8e023b3 nodeName:}" failed. No retries permitted until 2026-10-30 21:28:06.024235633 +0000 UTC m=+40.905298634 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0bbc8570-4947-455a-b7f8-8e9fa8e023b3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-z2dd4" (UID: "0bbc8570-4947-455a-b7f8-8e9fa8e023b3") : secret "insights-runtime-extractor-tls" not found
Oct 30 21:28:04.024697 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:28:04.024621    2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 21:28:04.024697 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:28:04.024661    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0bbc8570-4947-455a-b7f8-8e9fa8e023b3-kube-rbac-proxy-cm podName:0bbc8570-4947-455a-b7f8-8e9fa8e023b3 nodeName:}" failed. No retries permitted until 2026-10-30 21:28:06.024650664 +0000 UTC m=+40.905713595 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0bbc8570-4947-455a-b7f8-8e9fa8e023b3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-z2dd4" (UID: "0bbc8570-4947-455a-b7f8-8e9fa8e023b3") : configmap "kube-rbac-proxy" not found
Oct 30 21:28:04.063507 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:04.063467    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-5fffc97bdf-wpm2q" event={"ID":"11b44bf1-b3b1-4214-b7b4-4e508c618cd1","Type":"ContainerStarted","Data":"66993872d527790c98fcdbbb9078287fcbbabb569584624362c4b66d9cad4975"}
Oct 30 21:28:04.063995 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:04.063525    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-5fffc97bdf-wpm2q" event={"ID":"11b44bf1-b3b1-4214-b7b4-4e508c618cd1","Type":"ContainerStarted","Data":"5eceb5374b97fe1e5b74255fdff647f9a544bdccc3e50a91d984a973c5a74612"}
Oct 30 21:28:04.065789 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:04.065682    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-sklxd_67292bdc-7fe1-449d-af75-2baa90c16ea9/console-operator/1.log"
Oct 30 21:28:04.066050 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:04.066026    2573 scope.go:122] "RemoveContainer" containerID="f667ffa124949edf8581923668ce1139c1ad553b96aeaa80c8bc06474a49998a"
Oct 30 21:28:04.066261 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:28:04.066241    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-sklxd_openshift-console-operator(67292bdc-7fe1-449d-af75-2baa90c16ea9)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-sklxd" podUID="67292bdc-7fe1-449d-af75-2baa90c16ea9"
Oct 30 21:28:04.067804 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:04.067779    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4cvlt" event={"ID":"3913f7d4-86cb-4e86-af5a-debd8b859252","Type":"ContainerStarted","Data":"a68dcdcf60df8510803e843857ae80f698e736dc76177827d1eccb123e04d07b"}
Oct 30 21:28:04.068466 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:04.067897    2573 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-4cvlt"
Oct 30 21:28:04.069695 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:04.069670    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-542ht-hjp6z" event={"ID":"fd34a723-6c9e-48f1-ba7d-924d542b2bc7","Type":"ContainerStarted","Data":"891019481c213734fb14c99190ebe423c215bdaca160040e5852097370c6f0c2"}
Oct 30 21:28:04.069796 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:04.069711    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-542ht-hjp6z" event={"ID":"fd34a723-6c9e-48f1-ba7d-924d542b2bc7","Type":"ContainerStarted","Data":"1dca56b0734ec47a566bcaede4782a3c941f7018b609beabda556ecedc01fe47"}
Oct 30 21:28:04.081943 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:04.081889    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-service-ca/service-ca-5fffc97bdf-wpm2q" podStartSLOduration=1.081874371 podStartE2EDuration="1.081874371s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:28:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:28:04.081613182 +0000 UTC m=+38.962676135" watchObservedRunningTime="2026-10-30 21:28:04.081874371 +0000 UTC m=+38.962937325"
Oct 30 21:28:04.102871 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:04.102706    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-542ht-hjp6z" podStartSLOduration=2.102687738 podStartE2EDuration="2.102687738s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:28:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:28:04.10129477 +0000 UTC m=+38.982357725" watchObservedRunningTime="2026-10-30 21:28:04.102687738 +0000 UTC m=+38.983750692"
Oct 30 21:28:05.076815 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:05.076610    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-l7nbh" event={"ID":"faca011a-4ff1-4de0-9af1-ed9b0f253aeb","Type":"ContainerStarted","Data":"ad9b2ba840fbdcb2cc1912b1e7c9fb31f5131479dd61591a549bb123994ba542"}
Oct 30 21:28:05.079489 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:05.079460    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-gqdlr" event={"ID":"06a0877a-2fce-405f-b97c-091d88e19543","Type":"ContainerStarted","Data":"1a798d2f997b1926d33757f20539f6e3218ebaa7263eb766b48d9395b3308dfb"}
Oct 30 21:28:05.096474 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:05.096371    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-4cvlt" podStartSLOduration=35.288834959 podStartE2EDuration="39.096340713s" totalImagesPullingTime="3.807505754s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:27:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:28:04.181241313 +0000 UTC m=+39.062304268" watchObservedRunningTime="2026-10-30 21:28:05.096340713 +0000 UTC m=+39.977403669"
Oct 30 21:28:05.096630 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:05.096500    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-l7nbh" podStartSLOduration=2.027377074 podStartE2EDuration="4.096493845s" totalImagesPullingTime="2.069116771s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:28:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:28:05.096154301 +0000 UTC m=+39.977217252" watchObservedRunningTime="2026-10-30 21:28:05.096493845 +0000 UTC m=+39.977556799"
Oct 30 21:28:05.116026 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:05.115448    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-gqdlr" podStartSLOduration=2.2655658880000002 podStartE2EDuration="4.115435482s" totalImagesPullingTime="1.849869594s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:28:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:28:05.11417763 +0000 UTC m=+39.995240582" watchObservedRunningTime="2026-10-30 21:28:05.115435482 +0000 UTC m=+39.996498435"
Oct 30 21:28:05.347192 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:05.347123    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-97tsr_baa8d0c0-d477-4f30-a0f6-375fc84cc448/dns-node-resolver/0.log"
Oct 30 21:28:05.894150 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:05.894105    2573 kubelet.go:2727] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7cd56b8dbc-sklxd"
Oct 30 21:28:05.894314 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:05.894186    2573 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-sklxd"
Oct 30 21:28:05.894655 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:05.894640    2573 scope.go:122] "RemoveContainer" containerID="f667ffa124949edf8581923668ce1139c1ad553b96aeaa80c8bc06474a49998a"
Oct 30 21:28:05.894897 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:28:05.894870    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-sklxd_openshift-console-operator(67292bdc-7fe1-449d-af75-2baa90c16ea9)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-sklxd" podUID="67292bdc-7fe1-449d-af75-2baa90c16ea9"
Oct 30 21:28:06.048347 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:06.048313    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0bbc8570-4947-455a-b7f8-8e9fa8e023b3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-z2dd4\" (UID: \"0bbc8570-4947-455a-b7f8-8e9fa8e023b3\") " pod="openshift-insights/insights-runtime-extractor-z2dd4"
Oct 30 21:28:06.048536 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:06.048384    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0bbc8570-4947-455a-b7f8-8e9fa8e023b3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z2dd4\" (UID: \"0bbc8570-4947-455a-b7f8-8e9fa8e023b3\") " pod="openshift-insights/insights-runtime-extractor-z2dd4"
Oct 30 21:28:06.048536 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:28:06.048447    2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 21:28:06.048611 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:28:06.048552    2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 21:28:06.048611 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:28:06.048557    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0bbc8570-4947-455a-b7f8-8e9fa8e023b3-kube-rbac-proxy-cm podName:0bbc8570-4947-455a-b7f8-8e9fa8e023b3 nodeName:}" failed. No retries permitted until 2026-10-30 21:28:10.048536059 +0000 UTC m=+44.929598993 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0bbc8570-4947-455a-b7f8-8e9fa8e023b3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-z2dd4" (UID: "0bbc8570-4947-455a-b7f8-8e9fa8e023b3") : configmap "kube-rbac-proxy" not found
Oct 30 21:28:06.048708 ip-10-0-129-195 kubenswrapper[2573]: E1030 21:28:06.048628    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0bbc8570-4947-455a-b7f8-8e9fa8e023b3-insights-runtime-extractor-tls podName:0bbc8570-4947-455a-b7f8-8e9fa8e023b3 nodeName:}" failed. No retries permitted until 2026-10-30 21:28:10.048610421 +0000 UTC m=+44.929673355 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0bbc8570-4947-455a-b7f8-8e9fa8e023b3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-z2dd4" (UID: "0bbc8570-4947-455a-b7f8-8e9fa8e023b3") : secret "insights-runtime-extractor-tls" not found
Oct 30 21:28:06.739398 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:06.739348    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-jgns8_48efa9dc-3c20-4f81-86f3-c87ad697772c/node-ca/0.log"
Oct 30 21:28:07.091090 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:07.090990    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-csbxh" event={"ID":"5f171661-33a2-4631-81c7-bc10a5db791f","Type":"ContainerStarted","Data":"3d7689e1b5fcb511c1d15c5df12de6a4312605d48d8f84809aafc1fb17c9464b"}
Oct 30 21:28:07.110215 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:07.110158    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-csbxh" podStartSLOduration=34.100870438 podStartE2EDuration="41.110145762s" totalImagesPullingTime="7.009275324s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:27:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:28:07.109570436 +0000 UTC m=+41.990633392" watchObservedRunningTime="2026-10-30 21:28:07.110145762 +0000 UTC m=+41.991208714"
Oct 30 21:28:07.128478 ip-10-0-129-195 kubenswrapper[2573]: I1030 21:28:07.128402    2573 dynamic_cafile_content.go:175] "Shutting down controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 21:28:07.128588 ip-10-0-129-195 systemd[1]: Stopping Kubernetes Kubelet...
Oct 30 21:28:07.138830 ip-10-0-129-195 systemd[1]: kubelet.service: Deactivated successfully.
Oct 30 21:28:07.139001 ip-10-0-129-195 systemd[1]: Stopped Kubernetes Kubelet.
Oct 30 21:28:07.139054 ip-10-0-129-195 systemd[1]: kubelet.service: Consumed 3.260s CPU time.
Oct 30 21:28:07.150989 ip-10-0-129-195 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 21:28:07.257756 ip-10-0-129-195 kubenswrapper[5032]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:28:07.257756 ip-10-0-129-195 kubenswrapper[5032]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 21:28:07.257756 ip-10-0-129-195 kubenswrapper[5032]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:28:07.257756 ip-10-0-129-195 kubenswrapper[5032]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:28:07.259945 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.259847    5032 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:28:07.259945 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.259944    5032 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:28:07.259945 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.259948    5032 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:28:07.260044 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.259951    5032 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:28:07.260044 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.259954    5032 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:28:07.260044 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.259958    5032 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:28:07.260044 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.259960    5032 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:28:07.260044 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.259963    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:28:07.260044 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.259976    5032 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:28:07.260044 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.259979    5032 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:28:07.260044 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.259982    5032 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:28:07.260044 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.259985    5032 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:28:07.260044 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.259987    5032 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:28:07.260044 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.259990    5032 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:28:07.260044 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.259992    5032 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:28:07.260044 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.259995    5032 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:28:07.260044 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.259997    5032 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:28:07.260044 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260000    5032 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:28:07.260044 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260004    5032 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:28:07.260044 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260007    5032 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:28:07.260044 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260009    5032 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:28:07.260044 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260012    5032 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:28:07.260044 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260015    5032 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:28:07.260553 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260017    5032 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:28:07.260553 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260020    5032 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:28:07.260553 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260022    5032 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:28:07.260553 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260025    5032 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:28:07.260553 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260027    5032 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:28:07.260553 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260030    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:28:07.260553 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260032    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:28:07.260553 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260035    5032 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:28:07.260553 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260038    5032 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:28:07.260553 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260040    5032 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:28:07.260553 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260043    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:28:07.260553 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260045    5032 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:28:07.260553 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260049    5032 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:28:07.260553 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260052    5032 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:28:07.260553 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260054    5032 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:28:07.260553 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260057    5032 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:28:07.260553 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260060    5032 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:28:07.260553 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260063    5032 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:28:07.260553 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260065    5032 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:28:07.260553 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260068    5032 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:28:07.261024 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260070    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:28:07.261024 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260073    5032 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:28:07.261024 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260076    5032 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:28:07.261024 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260080    5032 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:28:07.261024 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260085    5032 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:28:07.261024 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260087    5032 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:28:07.261024 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260090    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:28:07.261024 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260093    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:28:07.261024 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260096    5032 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:28:07.261024 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260098    5032 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:28:07.261024 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260101    5032 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:28:07.261024 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260103    5032 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:28:07.261024 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260106    5032 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:28:07.261024 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260109    5032 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:28:07.261024 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260112    5032 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:28:07.261024 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260114    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:28:07.261024 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260117    5032 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:28:07.261024 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260119    5032 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:28:07.261024 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260122    5032 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:28:07.261501 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260124    5032 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:28:07.261501 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260128    5032 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:28:07.261501 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260131    5032 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:28:07.261501 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260134    5032 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:28:07.261501 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260137    5032 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:28:07.261501 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260139    5032 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:28:07.261501 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260142    5032 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:28:07.261501 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260144    5032 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:28:07.261501 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260147    5032 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:28:07.261501 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260149    5032 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:28:07.261501 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260153    5032 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:28:07.261501 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260156    5032 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:28:07.261501 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260159    5032 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:28:07.261501 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260162    5032 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:28:07.261501 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260165    5032 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:28:07.261501 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260168    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:28:07.261501 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260170    5032 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:28:07.261501 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260173    5032 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:28:07.261501 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260175    5032 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:28:07.261501 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260178    5032 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:28:07.261984 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260181    5032 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:28:07.261984 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260183    5032 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:28:07.261984 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260187    5032 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:28:07.261984 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260191    5032 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:28:07.261984 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260194    5032 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:28:07.261984 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260196    5032 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:28:07.261984 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260199    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:28:07.261984 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260202    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:28:07.261984 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260204    5032 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:28:07.261984 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260207    5032 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:28:07.261984 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260209    5032 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:28:07.261984 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260212    5032 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:28:07.261984 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260215    5032 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:28:07.261984 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260219    5032 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:28:07.261984 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260221    5032 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:28:07.261984 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260224    5032 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:28:07.261984 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260226    5032 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:28:07.261984 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260229    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:28:07.261984 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260232    5032 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:28:07.262455 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260234    5032 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:28:07.262455 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260237    5032 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:28:07.262455 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260239    5032 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:28:07.262455 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260244    5032 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:28:07.262455 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.260247    5032 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:28:07.262455 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261081    5032 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:28:07.262455 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261089    5032 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:28:07.262455 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261092    5032 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:28:07.262455 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261095    5032 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:28:07.262455 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261098    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:28:07.262455 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261101    5032 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:28:07.262455 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261103    5032 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:28:07.262455 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261105    5032 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:28:07.262455 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261108    5032 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:28:07.262455 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261111    5032 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:28:07.262455 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261113    5032 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:28:07.262455 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261117    5032 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:28:07.262455 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261121    5032 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:28:07.262455 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261125    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:28:07.262941 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261128    5032 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:28:07.262941 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261131    5032 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:28:07.262941 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261134    5032 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:28:07.262941 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261137    5032 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:28:07.262941 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261139    5032 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:28:07.262941 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261142    5032 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:28:07.262941 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261145    5032 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:28:07.262941 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261148    5032 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:28:07.262941 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261150    5032 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:28:07.262941 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261153    5032 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:28:07.262941 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261156    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:28:07.262941 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261159    5032 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:28:07.262941 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261161    5032 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:28:07.262941 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261164    5032 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:28:07.262941 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261166    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:28:07.262941 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261169    5032 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:28:07.262941 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261172    5032 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:28:07.262941 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261174    5032 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:28:07.262941 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261183    5032 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:28:07.262941 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261185    5032 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:28:07.263431 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261189    5032 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:28:07.263431 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261191    5032 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:28:07.263431 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261194    5032 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:28:07.263431 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261196    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:28:07.263431 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261199    5032 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:28:07.263431 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261201    5032 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:28:07.263431 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261204    5032 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:28:07.263431 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261206    5032 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:28:07.263431 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261209    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:28:07.263431 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261211    5032 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:28:07.263431 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261215    5032 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:28:07.263431 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261218    5032 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:28:07.263431 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261221    5032 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:28:07.263431 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261223    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:28:07.263431 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261226    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:28:07.263431 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261229    5032 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:28:07.263431 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261232    5032 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:28:07.263431 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261234    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:28:07.263431 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261238    5032 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:28:07.263936 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261240    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:28:07.263936 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261243    5032 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:28:07.263936 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261245    5032 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:28:07.263936 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261248    5032 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:28:07.263936 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261250    5032 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:28:07.263936 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261253    5032 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:28:07.263936 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261255    5032 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:28:07.263936 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261258    5032 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:28:07.263936 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261260    5032 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:28:07.263936 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261263    5032 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:28:07.263936 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261265    5032 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:28:07.263936 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261268    5032 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:28:07.263936 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261272    5032 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:28:07.263936 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261274    5032 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:28:07.263936 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261277    5032 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:28:07.263936 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261279    5032 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:28:07.263936 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261282    5032 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:28:07.263936 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261284    5032 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:28:07.263936 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261287    5032 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:28:07.263936 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261290    5032 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:28:07.264452 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261292    5032 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:28:07.264452 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261294    5032 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:28:07.264452 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261297    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:28:07.264452 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261300    5032 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:28:07.264452 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261303    5032 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:28:07.264452 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261305    5032 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:28:07.264452 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261308    5032 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:28:07.264452 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261310    5032 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:28:07.264452 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261313    5032 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:28:07.264452 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261316    5032 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:28:07.264452 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261319    5032 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:28:07.264452 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261322    5032 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:28:07.264452 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261324    5032 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:28:07.264452 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261327    5032 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:28:07.264452 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261330    5032 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:28:07.264452 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261332    5032 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:28:07.264452 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261335    5032 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:28:07.264452 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261337    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:28:07.264452 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261340    5032 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:28:07.264452 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261342    5032 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:28:07.264930 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261345    5032 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:28:07.264930 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261347    5032 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:28:07.264930 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261350    5032 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:28:07.264930 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261352    5032 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:28:07.264930 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261355    5032 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:28:07.264930 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261375    5032 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:28:07.264930 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261380    5032 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:28:07.264930 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261384    5032 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:28:07.264930 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261388    5032 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:28:07.264930 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261392    5032 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:28:07.264930 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261396    5032 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:28:07.264930 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261404    5032 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:28:07.264930 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261410    5032 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:28:07.264930 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261904    5032 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:28:07.264930 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261909    5032 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:28:07.264930 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261912    5032 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:28:07.264930 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261915    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:28:07.264930 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261918    5032 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:28:07.264930 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261921    5032 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:28:07.265430 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261923    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:28:07.265430 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261926    5032 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:28:07.265430 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261929    5032 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:28:07.265430 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261932    5032 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:28:07.265430 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261934    5032 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:28:07.265430 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261937    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:28:07.265430 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261940    5032 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:28:07.265430 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261943    5032 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:28:07.265430 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261945    5032 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:28:07.265430 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261948    5032 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:28:07.265430 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261950    5032 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:28:07.265430 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261953    5032 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:28:07.265430 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261955    5032 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:28:07.265430 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261958    5032 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:28:07.265430 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261961    5032 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:28:07.265430 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261964    5032 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:28:07.265430 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261966    5032 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:28:07.265430 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261969    5032 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:28:07.265430 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261972    5032 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:28:07.265904 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261975    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:28:07.265904 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261977    5032 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:28:07.265904 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261980    5032 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:28:07.265904 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261983    5032 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:28:07.265904 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261986    5032 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:28:07.265904 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261988    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:28:07.265904 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261991    5032 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:28:07.265904 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261994    5032 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:28:07.265904 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.261997    5032 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:28:07.265904 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262000    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:28:07.265904 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262002    5032 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:28:07.265904 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262005    5032 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:28:07.265904 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262007    5032 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:28:07.265904 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262010    5032 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:28:07.265904 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262013    5032 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:28:07.265904 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262016    5032 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:28:07.265904 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262019    5032 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:28:07.265904 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262021    5032 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:28:07.265904 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262024    5032 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:28:07.265904 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262026    5032 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:28:07.266749 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262029    5032 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:28:07.266749 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262031    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:28:07.266749 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262034    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:28:07.266749 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262036    5032 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:28:07.266749 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262039    5032 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:28:07.266749 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262043    5032 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:28:07.266749 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262046    5032 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:28:07.266749 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262050    5032 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:28:07.266749 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262053    5032 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:28:07.266749 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262056    5032 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:28:07.266749 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262059    5032 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:28:07.266749 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262061    5032 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:28:07.266749 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262064    5032 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:28:07.266749 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262068    5032 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:28:07.266749 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262070    5032 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:28:07.266749 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262073    5032 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:28:07.266749 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262076    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:28:07.266749 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262078    5032 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:28:07.266749 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262081    5032 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:28:07.267600 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262084    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:28:07.267600 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262086    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:28:07.267600 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262089    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:28:07.267600 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262092    5032 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:28:07.267600 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262096    5032 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:28:07.267600 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262099    5032 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:28:07.267600 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262103    5032 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:28:07.267600 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262106    5032 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:28:07.267600 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262112    5032 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:28:07.267600 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262114    5032 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:28:07.267600 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262117    5032 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:28:07.267600 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262120    5032 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:28:07.267600 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262122    5032 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:28:07.267600 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262125    5032 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:28:07.267600 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262127    5032 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:28:07.267600 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262130    5032 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:28:07.267600 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262133    5032 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:28:07.267600 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262135    5032 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:28:07.267600 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262138    5032 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:28:07.268501 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262140    5032 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:28:07.268501 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262143    5032 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:28:07.268501 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262146    5032 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:28:07.268501 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262149    5032 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:28:07.268501 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262151    5032 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:28:07.268501 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262154    5032 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:28:07.268501 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262156    5032 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:28:07.268501 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262160    5032 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:28:07.268501 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262162    5032 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:28:07.268501 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262165    5032 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:28:07.268501 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262167    5032 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:28:07.268501 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262170    5032 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:28:07.268501 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262173    5032 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:28:07.268501 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262176    5032 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:28:07.268501 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262178    5032 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:28:07.268501 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262181    5032 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:28:07.268501 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262183    5032 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:28:07.268501 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262187    5032 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:28:07.268501 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262189    5032 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:28:07.268501 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262192    5032 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:28:07.269370 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262194    5032 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:28:07.269370 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262200    5032 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:28:07.269370 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.262203    5032 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:28:07.269370 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262306    5032 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 21:28:07.269370 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262314    5032 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 21:28:07.269370 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262321    5032 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 21:28:07.269370 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262325    5032 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 21:28:07.269370 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262330    5032 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 21:28:07.269370 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262333    5032 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 21:28:07.269370 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262338    5032 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 21:28:07.269370 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262343    5032 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 21:28:07.269370 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262346    5032 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 21:28:07.269370 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262349    5032 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 21:28:07.269370 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262353    5032 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 21:28:07.269370 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262370    5032 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 21:28:07.269370 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262376    5032 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 21:28:07.269370 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262381    5032 flags.go:64] FLAG: --cgroup-root=""
Oct 30 21:28:07.269370 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262385    5032 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 21:28:07.269370 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262391    5032 flags.go:64] FLAG: --client-ca-file=""
Oct 30 21:28:07.269370 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262395    5032 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 21:28:07.269370 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262401    5032 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 21:28:07.269370 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262406    5032 flags.go:64] FLAG: --cluster-domain=""
Oct 30 21:28:07.269370 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262410    5032 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 21:28:07.269370 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262413    5032 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 21:28:07.270400 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262416    5032 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 21:28:07.270400 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262420    5032 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 21:28:07.270400 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262425    5032 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 21:28:07.270400 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262428    5032 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 21:28:07.270400 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262431    5032 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 21:28:07.270400 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262434    5032 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 21:28:07.270400 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262438    5032 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 21:28:07.270400 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262441    5032 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 21:28:07.270400 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262444    5032 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 21:28:07.270400 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262449    5032 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 21:28:07.270400 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262452    5032 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 21:28:07.270400 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262457    5032 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 21:28:07.270400 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262460    5032 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 21:28:07.270400 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262463    5032 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 21:28:07.270400 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262466    5032 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 21:28:07.270400 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262470    5032 flags.go:64] FLAG: --enable-server="true"
Oct 30 21:28:07.270400 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262473    5032 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 21:28:07.270400 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262479    5032 flags.go:64] FLAG: --event-burst="100"
Oct 30 21:28:07.270400 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262482    5032 flags.go:64] FLAG: --event-qps="50"
Oct 30 21:28:07.270400 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262485    5032 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 21:28:07.270400 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262489    5032 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 21:28:07.270400 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262492    5032 flags.go:64] FLAG: --eviction-hard=""
Oct 30 21:28:07.270400 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262496    5032 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 21:28:07.270400 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262499    5032 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 21:28:07.271269 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262502    5032 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 21:28:07.271269 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262505    5032 flags.go:64] FLAG: --eviction-soft=""
Oct 30 21:28:07.271269 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262509    5032 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 21:28:07.271269 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262512    5032 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 21:28:07.271269 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262515    5032 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 21:28:07.271269 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262519    5032 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 21:28:07.271269 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262522    5032 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 21:28:07.271269 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262525    5032 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 21:28:07.271269 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262528    5032 flags.go:64] FLAG: --feature-gates=""
Oct 30 21:28:07.271269 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262532    5032 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 21:28:07.271269 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262535    5032 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 21:28:07.271269 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262538    5032 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 21:28:07.271269 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262541    5032 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 21:28:07.271269 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262545    5032 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 21:28:07.271269 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262548    5032 flags.go:64] FLAG: --help="false"
Oct 30 21:28:07.271269 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262551    5032 flags.go:64] FLAG: --hostname-override="ip-10-0-129-195.us-east-2.compute.internal"
Oct 30 21:28:07.271269 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262554    5032 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 21:28:07.271269 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262559    5032 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 21:28:07.271269 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262562    5032 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 21:28:07.271269 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262566    5032 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 21:28:07.271269 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262569    5032 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 21:28:07.271269 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262572    5032 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 21:28:07.271269 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262576    5032 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 21:28:07.272276 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262579    5032 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 21:28:07.272276 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262582    5032 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 21:28:07.272276 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262586    5032 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 21:28:07.272276 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262589    5032 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 21:28:07.272276 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262592    5032 flags.go:64] FLAG: --kube-reserved=""
Oct 30 21:28:07.272276 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262595    5032 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 21:28:07.272276 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262598    5032 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 21:28:07.272276 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262601    5032 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 21:28:07.272276 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262604    5032 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 21:28:07.272276 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262607    5032 flags.go:64] FLAG: --lock-file=""
Oct 30 21:28:07.272276 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262610    5032 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 21:28:07.272276 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262613    5032 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 21:28:07.272276 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262616    5032 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 21:28:07.272276 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262622    5032 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 21:28:07.272276 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262625    5032 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 21:28:07.272276 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262629    5032 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 21:28:07.272276 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262632    5032 flags.go:64] FLAG: --logging-format="text"
Oct 30 21:28:07.272276 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262635    5032 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 21:28:07.272276 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262638    5032 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 21:28:07.272276 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262641    5032 flags.go:64] FLAG: --manifest-url=""
Oct 30 21:28:07.272276 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262644    5032 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 21:28:07.272276 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262648    5032 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 21:28:07.272276 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262651    5032 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 21:28:07.272276 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262655    5032 flags.go:64] FLAG: --max-pods="110"
Oct 30 21:28:07.272276 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262658    5032 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 21:28:07.273442 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262661    5032 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 21:28:07.273442 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262666    5032 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 21:28:07.273442 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262669    5032 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 21:28:07.273442 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262672    5032 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 21:28:07.273442 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262675    5032 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 21:28:07.273442 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262678    5032 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 21:28:07.273442 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262686    5032 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 21:28:07.273442 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262689    5032 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 21:28:07.273442 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262692    5032 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 21:28:07.273442 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262695    5032 flags.go:64] FLAG: --pod-cidr=""
Oct 30 21:28:07.273442 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262698    5032 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 21:28:07.273442 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262701    5032 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 21:28:07.273442 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262704    5032 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 21:28:07.273442 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262707    5032 flags.go:64] FLAG: --port="10250"
Oct 30 21:28:07.273442 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262710    5032 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 21:28:07.273442 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262713    5032 flags.go:64] FLAG: --provider-id="aws:///us-east-2a/i-07ceafd550d2fa8d9"
Oct 30 21:28:07.273442 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262716    5032 flags.go:64] FLAG: --qos-reserved=""
Oct 30 21:28:07.273442 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262719    5032 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 21:28:07.273442 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262723    5032 flags.go:64] FLAG: --register-node="true"
Oct 30 21:28:07.273442 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262726    5032 flags.go:64] FLAG: --register-with-taints=""
Oct 30 21:28:07.273442 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262730    5032 flags.go:64] FLAG: --registry-burst="10"
Oct 30 21:28:07.273442 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262733    5032 flags.go:64] FLAG: --registry-qps="5"
Oct 30 21:28:07.273442 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262735    5032 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 21:28:07.273442 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262739    5032 flags.go:64] FLAG: --reserved-memory=""
Oct 30 21:28:07.273442 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262743    5032 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 21:28:07.273442 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262746    5032 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 21:28:07.274547 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262749    5032 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 21:28:07.274547 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262752    5032 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 21:28:07.274547 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262755    5032 flags.go:64] FLAG: --runonce="false"
Oct 30 21:28:07.274547 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262758    5032 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 21:28:07.274547 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262762    5032 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 21:28:07.274547 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262765    5032 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 21:28:07.274547 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262771    5032 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 21:28:07.274547 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262776    5032 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 21:28:07.274547 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262780    5032 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 21:28:07.274547 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262783    5032 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 21:28:07.274547 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262786    5032 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 21:28:07.274547 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262789    5032 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 21:28:07.274547 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262793    5032 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 21:28:07.274547 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262796    5032 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 21:28:07.274547 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262799    5032 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 21:28:07.274547 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262802    5032 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 21:28:07.274547 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262805    5032 flags.go:64] FLAG: --system-cgroups=""
Oct 30 21:28:07.274547 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262808    5032 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 21:28:07.274547 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262819    5032 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 21:28:07.274547 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262822    5032 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 21:28:07.274547 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262825    5032 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 21:28:07.274547 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262829    5032 flags.go:64] FLAG: --tls-min-version=""
Oct 30 21:28:07.274547 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262832    5032 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 21:28:07.274547 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262835    5032 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 21:28:07.274547 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262838    5032 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 21:28:07.275638 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262841    5032 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 21:28:07.275638 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262844    5032 flags.go:64] FLAG: --v="2"
Oct 30 21:28:07.275638 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262853    5032 flags.go:64] FLAG: --version="false"
Oct 30 21:28:07.275638 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262858    5032 flags.go:64] FLAG: --vmodule=""
Oct 30 21:28:07.275638 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262862    5032 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 21:28:07.275638 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.262866    5032 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 21:28:07.275638 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263032    5032 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:28:07.275638 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263037    5032 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:28:07.275638 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263040    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:28:07.275638 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263042    5032 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:28:07.275638 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263046    5032 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:28:07.275638 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263049    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:28:07.275638 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263051    5032 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:28:07.275638 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263054    5032 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:28:07.275638 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263058    5032 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:28:07.275638 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263061    5032 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:28:07.275638 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263064    5032 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:28:07.275638 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263066    5032 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:28:07.275638 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263069    5032 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:28:07.275638 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263072    5032 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:28:07.275638 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263074    5032 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:28:07.276556 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263077    5032 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:28:07.276556 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263080    5032 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:28:07.276556 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263083    5032 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:28:07.276556 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263085    5032 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:28:07.276556 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263088    5032 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:28:07.276556 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263090    5032 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:28:07.276556 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263093    5032 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:28:07.276556 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263095    5032 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:28:07.276556 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263098    5032 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:28:07.276556 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263100    5032 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:28:07.276556 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263103    5032 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:28:07.276556 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263105    5032 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:28:07.276556 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263108    5032 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:28:07.276556 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263111    5032 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:28:07.276556 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263113    5032 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:28:07.276556 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263116    5032 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:28:07.276556 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263119    5032 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:28:07.276556 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263122    5032 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:28:07.276556 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263124    5032 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:28:07.277275 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263127    5032 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:28:07.277275 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263130    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:28:07.277275 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263132    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:28:07.277275 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263135    5032 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:28:07.277275 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263137    5032 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:28:07.277275 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263140    5032 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:28:07.277275 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263144    5032 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:28:07.277275 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263147    5032 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:28:07.277275 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263150    5032 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:28:07.277275 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263153    5032 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:28:07.277275 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263156    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:28:07.277275 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263158    5032 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:28:07.277275 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263162    5032 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:28:07.277275 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263166    5032 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:28:07.277275 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263169    5032 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:28:07.277275 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263172    5032 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:28:07.277275 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263175    5032 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:28:07.277275 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263177    5032 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:28:07.277275 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263180    5032 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:28:07.278082 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263182    5032 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:28:07.278082 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263185    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:28:07.278082 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263188    5032 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:28:07.278082 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263190    5032 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:28:07.278082 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263193    5032 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:28:07.278082 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263195    5032 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:28:07.278082 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263198    5032 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:28:07.278082 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263200    5032 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:28:07.278082 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263203    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:28:07.278082 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263206    5032 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:28:07.278082 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263208    5032 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:28:07.278082 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263211    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:28:07.278082 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263214    5032 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:28:07.278082 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263217    5032 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:28:07.278082 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263220    5032 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:28:07.278082 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263222    5032 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:28:07.278082 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263225    5032 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:28:07.278082 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263229    5032 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:28:07.278082 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263232    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:28:07.278650 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263236    5032 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:28:07.278650 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263238    5032 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:28:07.278650 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263241    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:28:07.278650 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263243    5032 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:28:07.278650 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263246    5032 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:28:07.278650 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263249    5032 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:28:07.278650 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263252    5032 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:28:07.278650 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263254    5032 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:28:07.278650 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263257    5032 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:28:07.278650 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263259    5032 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:28:07.278650 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263264    5032 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:28:07.278650 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263267    5032 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:28:07.278650 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263269    5032 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:28:07.278650 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263272    5032 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:28:07.278650 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263275    5032 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:28:07.278650 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263278    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:28:07.278650 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263280    5032 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:28:07.278650 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263283    5032 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:28:07.278650 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263285    5032 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:28:07.278650 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263288    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:28:07.279378 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263291    5032 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:28:07.279378 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263293    5032 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:28:07.279378 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263299    5032 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:28:07.279378 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263301    5032 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:28:07.279378 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263304    5032 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:28:07.279378 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263306    5032 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:28:07.279378 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263309    5032 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:28:07.279378 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263312    5032 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:28:07.279378 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263315    5032 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:28:07.279378 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263317    5032 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:28:07.279378 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263320    5032 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:28:07.279378 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263323    5032 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:28:07.279378 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263326    5032 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:28:07.279378 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263329    5032 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:28:07.279378 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.263355    5032 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:28:07.279378 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263411    5032 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:28:07.280108 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263417    5032 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:28:07.280108 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263422    5032 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:28:07.280108 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263425    5032 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:28:07.280108 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263428    5032 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:28:07.280108 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263431    5032 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:28:07.280108 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263433    5032 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:28:07.280108 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263436    5032 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:28:07.280108 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263439    5032 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:28:07.280108 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263442    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:28:07.280108 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263445    5032 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:28:07.280108 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263448    5032 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:28:07.280108 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263450    5032 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:28:07.280108 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263453    5032 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:28:07.280108 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263456    5032 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:28:07.280108 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263458    5032 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:28:07.280108 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263461    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:28:07.280108 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263464    5032 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:28:07.280108 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263466    5032 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:28:07.280108 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263470    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:28:07.280699 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263473    5032 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:28:07.280699 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263476    5032 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:28:07.280699 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263479    5032 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:28:07.280699 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263481    5032 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:28:07.280699 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263484    5032 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:28:07.280699 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263487    5032 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:28:07.280699 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263490    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:28:07.280699 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263493    5032 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:28:07.280699 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263495    5032 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:28:07.280699 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263498    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:28:07.280699 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263501    5032 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:28:07.280699 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263504    5032 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:28:07.280699 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263507    5032 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:28:07.280699 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263509    5032 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:28:07.280699 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263512    5032 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:28:07.280699 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263514    5032 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:28:07.280699 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263517    5032 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:28:07.280699 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263520    5032 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:28:07.280699 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263523    5032 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:28:07.281205 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263526    5032 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:28:07.281205 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263529    5032 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:28:07.281205 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263531    5032 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:28:07.281205 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263534    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:28:07.281205 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263536    5032 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:28:07.281205 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263539    5032 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:28:07.281205 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263541    5032 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:28:07.281205 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263544    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:28:07.281205 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263547    5032 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:28:07.281205 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263549    5032 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:28:07.281205 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263552    5032 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:28:07.281205 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263554    5032 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:28:07.281205 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263558    5032 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:28:07.281205 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263561    5032 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:28:07.281205 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263563    5032 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:28:07.281205 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263566    5032 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:28:07.281205 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263568    5032 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:28:07.281205 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263571    5032 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:28:07.281205 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263574    5032 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:28:07.281205 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263576    5032 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:28:07.281727 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263579    5032 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:28:07.281727 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263582    5032 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:28:07.281727 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263584    5032 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:28:07.281727 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263587    5032 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:28:07.281727 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263589    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:28:07.281727 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263592    5032 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:28:07.281727 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263595    5032 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:28:07.281727 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263597    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:28:07.281727 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263600    5032 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:28:07.281727 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263602    5032 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:28:07.281727 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263605    5032 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:28:07.281727 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263607    5032 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:28:07.281727 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263611    5032 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:28:07.281727 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263613    5032 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:28:07.281727 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263616    5032 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:28:07.281727 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263618    5032 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:28:07.281727 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263621    5032 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:28:07.281727 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263623    5032 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:28:07.281727 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263626    5032 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:28:07.281727 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263628    5032 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:28:07.282195 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263631    5032 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:28:07.282195 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263633    5032 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:28:07.282195 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263636    5032 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:28:07.282195 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263638    5032 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:28:07.282195 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263642    5032 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:28:07.282195 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263645    5032 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:28:07.282195 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263647    5032 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:28:07.282195 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263650    5032 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:28:07.282195 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263652    5032 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:28:07.282195 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263655    5032 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:28:07.282195 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263657    5032 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:28:07.282195 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263660    5032 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:28:07.282195 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263663    5032 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:28:07.282195 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263666    5032 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:28:07.282195 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263669    5032 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:28:07.282195 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263671    5032 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:28:07.282195 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263674    5032 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:28:07.282195 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263676    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:28:07.282195 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263679    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:28:07.282662 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263682    5032 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:28:07.282662 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263684    5032 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:28:07.282662 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263687    5032 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:28:07.282662 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263690    5032 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:28:07.282662 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263692    5032 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:28:07.282662 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263695    5032 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:28:07.282662 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263698    5032 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:28:07.282662 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.263701    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:28:07.282662 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.268962    5032 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 21:28:07.282662 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.268983    5032 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 21:28:07.282662 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.269009    5032 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 21:28:07.282662 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.269016    5032 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 21:28:07.282662 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269111    5032 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:28:07.282662 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269119    5032 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:28:07.282662 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269125    5032 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:28:07.282662 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269130    5032 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:28:07.282662 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269135    5032 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:28:07.282662 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269139    5032 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:28:07.282662 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269144    5032 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:28:07.283122 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269148    5032 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:28:07.283122 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269154    5032 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:28:07.283122 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269161    5032 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:28:07.283122 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269166    5032 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:28:07.283122 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269171    5032 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:28:07.283122 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269175    5032 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:28:07.283122 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269194    5032 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:28:07.283122 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269199    5032 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:28:07.283122 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269203    5032 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:28:07.283122 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269207    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:28:07.283122 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269212    5032 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:28:07.283122 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269217    5032 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:28:07.283122 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269221    5032 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:28:07.283122 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269226    5032 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:28:07.283122 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269230    5032 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:28:07.283122 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269234    5032 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:28:07.283122 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269239    5032 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:28:07.283122 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269243    5032 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:28:07.283122 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269247    5032 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:28:07.283720 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269252    5032 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:28:07.283720 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269256    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:28:07.283720 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269260    5032 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:28:07.283720 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269274    5032 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:28:07.283720 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269279    5032 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:28:07.283720 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269284    5032 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:28:07.283720 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269288    5032 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:28:07.283720 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269292    5032 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:28:07.283720 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269297    5032 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:28:07.283720 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269301    5032 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:28:07.283720 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269305    5032 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:28:07.283720 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269310    5032 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:28:07.283720 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269314    5032 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:28:07.283720 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269318    5032 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:28:07.283720 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269322    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:28:07.283720 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269327    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:28:07.283720 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269331    5032 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:28:07.283720 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269335    5032 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:28:07.283720 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269340    5032 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:28:07.283720 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269344    5032 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:28:07.284198 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269348    5032 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:28:07.284198 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269352    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:28:07.284198 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269372    5032 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:28:07.284198 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269377    5032 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:28:07.284198 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269381    5032 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:28:07.284198 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269384    5032 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:28:07.284198 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269388    5032 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:28:07.284198 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269392    5032 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:28:07.284198 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269405    5032 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:28:07.284198 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269410    5032 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:28:07.284198 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269414    5032 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:28:07.284198 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269418    5032 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:28:07.284198 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269422    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:28:07.284198 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269426    5032 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:28:07.284198 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269430    5032 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:28:07.284198 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269435    5032 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:28:07.284198 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269447    5032 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:28:07.284198 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269451    5032 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:28:07.284198 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269455    5032 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:28:07.284764 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269460    5032 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:28:07.284764 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269464    5032 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:28:07.284764 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269469    5032 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:28:07.284764 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269473    5032 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:28:07.284764 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269477    5032 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:28:07.284764 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269482    5032 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:28:07.284764 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269486    5032 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:28:07.284764 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269490    5032 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:28:07.284764 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269494    5032 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:28:07.284764 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269501    5032 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:28:07.284764 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269508    5032 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:28:07.284764 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269514    5032 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:28:07.284764 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269518    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:28:07.284764 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269522    5032 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:28:07.284764 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269527    5032 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:28:07.284764 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269532    5032 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:28:07.284764 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269536    5032 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:28:07.284764 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269540    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:28:07.284764 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269544    5032 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:28:07.285214 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269549    5032 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:28:07.285214 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269553    5032 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:28:07.285214 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269558    5032 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:28:07.285214 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269562    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:28:07.285214 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269566    5032 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:28:07.285214 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269571    5032 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:28:07.285214 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269576    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:28:07.285214 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269580    5032 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:28:07.285214 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269584    5032 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:28:07.285214 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269588    5032 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:28:07.285214 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269592    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:28:07.285214 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269605    5032 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:28:07.285214 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269610    5032 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:28:07.285214 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269614    5032 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:28:07.285214 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269619    5032 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:28:07.285214 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269623    5032 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:28:07.285214 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269627    5032 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:28:07.285214 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269631    5032 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:28:07.285214 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269635    5032 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:28:07.285214 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269639    5032 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:28:07.285714 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269644    5032 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:28:07.285714 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.269649    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:28:07.285714 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.269695    5032 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:28:07.285714 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.270509    5032 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:28:07.285714 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.270760    5032 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:28:07.285714 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.270766    5032 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:28:07.285714 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.270771    5032 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:28:07.285714 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.270776    5032 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:28:07.285714 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.270781    5032 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:28:07.285714 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.270786    5032 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:28:07.285714 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.270791    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:28:07.285714 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.270802    5032 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:28:07.285714 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.270807    5032 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:28:07.285714 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.270812    5032 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:28:07.285714 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.270816    5032 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:28:07.285714 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.270821    5032 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:28:07.286100 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.270826    5032 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:28:07.286100 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.270830    5032 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:28:07.286100 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.270835    5032 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:28:07.286100 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.270840    5032 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:28:07.286100 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.270844    5032 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:28:07.286100 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.270849    5032 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:28:07.286100 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.270854    5032 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:28:07.286100 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.270858    5032 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:28:07.286100 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.270899    5032 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:28:07.286100 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.270904    5032 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:28:07.286100 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.270909    5032 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:28:07.286100 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.270913    5032 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:28:07.286100 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.270918    5032 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:28:07.286100 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.270924    5032 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:28:07.286100 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.270928    5032 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:28:07.286100 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.270933    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:28:07.286100 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.270938    5032 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:28:07.286100 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.270942    5032 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:28:07.286100 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.270946    5032 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:28:07.286100 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.270951    5032 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:28:07.286676 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.270960    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:28:07.286676 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.270965    5032 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:28:07.286676 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.270972    5032 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:28:07.286676 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.270980    5032 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:28:07.286676 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.270986    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:28:07.286676 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.270992    5032 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:28:07.286676 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.270997    5032 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:28:07.286676 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271002    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:28:07.286676 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271007    5032 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:28:07.286676 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271011    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:28:07.286676 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271016    5032 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:28:07.286676 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271020    5032 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:28:07.286676 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271025    5032 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:28:07.286676 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271035    5032 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:28:07.286676 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271040    5032 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:28:07.286676 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271045    5032 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:28:07.286676 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271049    5032 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:28:07.286676 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271054    5032 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:28:07.286676 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271059    5032 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:28:07.287160 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271063    5032 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:28:07.287160 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271068    5032 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:28:07.287160 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271075    5032 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:28:07.287160 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271083    5032 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:28:07.287160 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271088    5032 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:28:07.287160 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271094    5032 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:28:07.287160 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271104    5032 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:28:07.287160 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271109    5032 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:28:07.287160 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271115    5032 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:28:07.287160 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271120    5032 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:28:07.287160 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271125    5032 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:28:07.287160 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271130    5032 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:28:07.287160 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271135    5032 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:28:07.287160 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271143    5032 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:28:07.287160 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271147    5032 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:28:07.287160 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271152    5032 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:28:07.287160 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271158    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:28:07.287160 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271170    5032 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:28:07.287160 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271287    5032 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:28:07.287160 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271294    5032 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:28:07.287732 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271298    5032 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:28:07.287732 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271302    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:28:07.287732 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271307    5032 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:28:07.287732 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271311    5032 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:28:07.287732 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271316    5032 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:28:07.287732 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271320    5032 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:28:07.287732 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271325    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:28:07.287732 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271329    5032 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:28:07.287732 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271333    5032 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:28:07.287732 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271337    5032 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:28:07.287732 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271342    5032 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:28:07.287732 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271346    5032 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:28:07.287732 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271351    5032 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:28:07.287732 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271355    5032 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:28:07.287732 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271381    5032 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:28:07.287732 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271387    5032 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:28:07.287732 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271392    5032 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:28:07.287732 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271396    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:28:07.287732 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271401    5032 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:28:07.287732 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271405    5032 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:28:07.288222 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271409    5032 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:28:07.288222 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271414    5032 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:28:07.288222 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271419    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:28:07.288222 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271423    5032 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:28:07.288222 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271428    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:28:07.288222 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271432    5032 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:28:07.288222 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271436    5032 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:28:07.288222 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271441    5032 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:28:07.288222 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271445    5032 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:28:07.288222 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271449    5032 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:28:07.288222 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271456    5032 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:28:07.288222 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271461    5032 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:28:07.288222 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271465    5032 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:28:07.288222 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271470    5032 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:28:07.288222 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.271519    5032 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:28:07.288655 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271589    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:28:07.288655 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271598    5032 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:28:07.288655 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271605    5032 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:28:07.288655 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271610    5032 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:28:07.288655 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271615    5032 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:28:07.288655 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271620    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:28:07.288655 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271625    5032 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:28:07.288655 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271629    5032 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:28:07.288655 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271633    5032 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:28:07.288655 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271638    5032 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:28:07.288655 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271642    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:28:07.288655 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271646    5032 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:28:07.288655 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271651    5032 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:28:07.288655 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271655    5032 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:28:07.288655 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271659    5032 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:28:07.288655 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271664    5032 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:28:07.288655 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271669    5032 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:28:07.288655 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271674    5032 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:28:07.288655 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271678    5032 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:28:07.288655 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271682    5032 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:28:07.289141 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271687    5032 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:28:07.289141 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271691    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:28:07.289141 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271695    5032 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:28:07.289141 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271699    5032 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:28:07.289141 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271704    5032 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:28:07.289141 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271709    5032 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:28:07.289141 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271715    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:28:07.289141 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271719    5032 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:28:07.289141 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271725    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:28:07.289141 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271730    5032 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:28:07.289141 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271734    5032 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:28:07.289141 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271739    5032 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:28:07.289141 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271743    5032 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:28:07.289141 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271747    5032 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:28:07.289141 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271752    5032 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:28:07.289141 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271756    5032 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:28:07.289141 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271764    5032 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:28:07.289141 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271769    5032 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:28:07.289141 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271779    5032 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:28:07.289601 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271784    5032 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:28:07.289601 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271788    5032 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:28:07.289601 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271792    5032 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:28:07.289601 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271798    5032 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:28:07.289601 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271803    5032 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:28:07.289601 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271808    5032 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:28:07.289601 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271812    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:28:07.289601 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271816    5032 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:28:07.289601 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271821    5032 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:28:07.289601 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271825    5032 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:28:07.289601 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271829    5032 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:28:07.289601 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271834    5032 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:28:07.289601 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271838    5032 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:28:07.289601 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271843    5032 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:28:07.289601 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271847    5032 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:28:07.289601 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271851    5032 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:28:07.289601 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271856    5032 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:28:07.289601 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271860    5032 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:28:07.289601 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271864    5032 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:28:07.289601 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271868    5032 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:28:07.290111 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271873    5032 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:28:07.290111 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271879    5032 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:28:07.290111 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271884    5032 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:28:07.290111 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271888    5032 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:28:07.290111 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271893    5032 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:28:07.290111 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271897    5032 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:28:07.290111 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271902    5032 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:28:07.290111 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271906    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:28:07.290111 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271910    5032 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:28:07.290111 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271921    5032 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:28:07.290111 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271926    5032 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:28:07.290111 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271930    5032 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:28:07.290111 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271934    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:28:07.290111 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271939    5032 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:28:07.290111 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271947    5032 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:28:07.290111 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271955    5032 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:28:07.290111 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271960    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:28:07.290111 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271965    5032 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:28:07.290817 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271969    5032 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:28:07.290817 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271974    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:28:07.290817 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271978    5032 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:28:07.290817 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271983    5032 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:28:07.290817 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271987    5032 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:28:07.290817 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271991    5032 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:28:07.290817 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.271996    5032 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:28:07.290817 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.272000    5032 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:28:07.290817 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.272004    5032 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:28:07.290817 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.272008    5032 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:28:07.290817 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.272013    5032 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:28:07.290817 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.272017    5032 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:28:07.290817 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.272022    5032 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:28:07.290817 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.272026    5032 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:28:07.290817 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.272031    5032 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:28:07.290817 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.272036    5032 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:28:07.290817 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.272043    5032 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:28:07.290817 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.272048    5032 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:28:07.290817 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.272053    5032 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:28:07.291531 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.272058    5032 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:28:07.291531 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.272062    5032 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:28:07.291531 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.272069    5032 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:28:07.291531 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.272073    5032 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:28:07.291531 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.272080    5032 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:28:07.291531 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.272084    5032 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:28:07.291531 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.272088    5032 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:28:07.291531 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.272093    5032 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:28:07.291531 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.272097    5032 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:28:07.291531 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:07.272102    5032 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:28:07.291531 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.272188    5032 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 21:28:07.291531 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.273868    5032 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Oct 30 21:28:07.291531 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.273957    5032 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Oct 30 21:28:07.291531 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.274225    5032 server.go:1014] "Starting client certificate rotation"
Oct 30 21:28:07.291531 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.274351    5032 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 21:28:07.291531 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.274419    5032 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 21:22:25 +0000 UTC" deadline="2028-06-20 09:30:49.084404363 +0000 UTC"
Oct 30 21:28:07.291531 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.274446    5032 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14364h2m41.809960478s"
Oct 30 21:28:07.292062 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.275035    5032 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 21:28:07.292062 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.276887    5032 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 21:28:07.292062 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.278334    5032 log.go:25] "Validated CRI v1 runtime API"
Oct 30 21:28:07.292062 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.280489    5032 log.go:25] "Validated CRI v1 image API"
Oct 30 21:28:07.292062 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.281179    5032 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 21:28:07.292062 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.283219    5032 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 e60aa7d7-0091-4b1c-9a9d-101906f592a1:/dev/nvme0n1p4 e9e2424f-0447-4325-ae80-d80a1da18197:/dev/nvme0n1p3]
Oct 30 21:28:07.292266 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.283241    5032 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/01a6be223caaa40916fbe86ec4ffed50b3b9de9f3f8b0241a8c368c01fa597a7/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/01a6be223caaa40916fbe86ec4ffed50b3b9de9f3f8b0241a8c368c01fa597a7/userdata/shm major:0 minor:236 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/04cbe888f55f0793a5c9408b6f252956dba81d6385e86f4366157ebd66e9a3e6/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/04cbe888f55f0793a5c9408b6f252956dba81d6385e86f4366157ebd66e9a3e6/userdata/shm major:0 minor:72 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/0bbe505a0a4b7594e8e39d86bde05cbb8a3dd7cb5cb29b5128cb8d3b9cd43991/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/0bbe505a0a4b7594e8e39d86bde05cbb8a3dd7cb5cb29b5128cb8d3b9cd43991/userdata/shm major:0 minor:244 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/0bc1321b91785136139aed9902de94cdf1a2c849b8813d68d5d7a26b1d104efb/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/0bc1321b91785136139aed9902de94cdf1a2c849b8813d68d5d7a26b1d104efb/userdata/shm major:0 minor:240 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/0f738ceac995d4100163621b468e95def6099e0a6b0e9758ea5f3b383bb595f4/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/0f738ceac995d4100163621b468e95def6099e0a6b0e9758ea5f3b383bb595f4/userdata/shm major:0 minor:238 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1567ec928ebf4029842c9202f9fbc1ca86870cc5b45a9cf2214aa801ce31a73f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1567ec928ebf4029842c9202f9fbc1ca86870cc5b45a9cf2214aa801ce31a73f/userdata/shm major:0 minor:88 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1c916fbc0fbaa7a6b041fa9c07924c5020e909cd2b5516de2fae05b8d63f51a4/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1c916fbc0fbaa7a6b041fa9c07924c5020e909cd2b5516de2fae05b8d63f51a4/userdata/shm major:0 minor:242 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1dca56b0734ec47a566bcaede4782a3c941f7018b609beabda556ecedc01fe47/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1dca56b0734ec47a566bcaede4782a3c941f7018b609beabda556ecedc01fe47/userdata/shm major:0 minor:344 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/25200764cd35dbf351705da7ea4eb6473438cd86a25b56a86438f0becd6fa64c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/25200764cd35dbf351705da7ea4eb6473438cd86a25b56a86438f0becd6fa64c/userdata/shm major:0 minor:318 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/28d6df74a321bc7402b0991709090e293ce1a21aea1c20e6a53e533ecaaf619e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/28d6df74a321bc7402b0991709090e293ce1a21aea1c20e6a53e533ecaaf619e/userdata/shm major:0 minor:89 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/57dbd87c6c23e36cb2934dbe29990855b3d7f1a4fedbd560cc649beff9706264/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/57dbd87c6c23e36cb2934dbe29990855b3d7f1a4fedbd560cc649beff9706264/userdata/shm major:0 minor:96 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/5eceb5374b97fe1e5b74255fdff647f9a544bdccc3e50a91d984a973c5a74612/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/5eceb5374b97fe1e5b74255fdff647f9a544bdccc3e50a91d984a973c5a74612/userdata/shm major:0 minor:351 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/5f2f9bf4612d88a6d984149720b0481036f148988b145cddf801e00076f200b1/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/5f2f9bf4612d88a6d984149720b0481036f148988b145cddf801e00076f200b1/userdata/shm major:0 minor:77 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/674d20fe97c246ae471b27f055ab576608b0a4cad5fabb0dca5b5ee7a31bfc48/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/674d20fe97c246ae471b27f055ab576608b0a4cad5fabb0dca5b5ee7a31bfc48/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/75526b7530220d62f568ed172aa859dcd9a25086d66f706ee52dc359e8bdc322/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/75526b7530220d62f568ed172aa859dcd9a25086d66f706ee52dc359e8bdc322/userdata/shm major:0 minor:223 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/7832534c8c7e80d5ba345d1e86c0d75998d77b27c1f4fba9dba1561978a6523b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/7832534c8c7e80d5ba345d1e86c0d75998d77b27c1f4fba9dba1561978a6523b/userdata/shm major:0 minor:294 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/868d0e77ebffb05386c7b1e5376b26ba106aa3b18521844b1d33ca992dd6d4cc/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/868d0e77ebffb05386c7b1e5376b26ba106aa3b18521844b1d33ca992dd6d4cc/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/914ffc5c554a5c4122f2c41a837189ca7dc51ff6991d0eb637f078079903684a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/914ffc5c554a5c4122f2c41a837189ca7dc51ff6991d0eb637f078079903684a/userdata/shm major:0 minor:232 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a9272b555796f4e5bfa1e2b12cda99544643a87b628fa11b2b151adf30413d4c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a9272b555796f4e5bfa1e2b12cda99544643a87b628fa11b2b151adf30413d4c/userdata/shm major:0 minor:323 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b9c929b2079c2178d1e0dbb3a0cd5c7965226441feeb7576f7b91f47724b22ff/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b9c929b2079c2178d1e0dbb3a0cd5c7965226441feeb7576f7b91f47724b22ff/userdata/shm major:0 minor:49 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/bb1ff751df6656db7e8fef3bfd7a5baad5f96b46c870437d27222dd412553064/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/bb1ff751df6656db7e8fef3bfd7a5baad5f96b46c870437d27222dd412553064/userdata/shm major:0 minor:296 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d122ee4dad1efd8d9cfa8ca3bf428e14dcdb5a02d16d145d71957b4666f0361f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d122ee4dad1efd8d9cfa8ca3bf428e14dcdb5a02d16d145d71957b4666f0361f/userdata/shm major:0 minor:76 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ef6c6d4940b60a96a28dda299962451ccbe7af14eef1e415eaf6be3ef995c61d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ef6c6d4940b60a96a28dda299962451ccbe7af14eef1e415eaf6be3ef995c61d/userdata/shm major:0 minor:234 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f0e4195569bff5c7940a6afbb3d373bff17293a533a3c98574dcfcdb118b3a57/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f0e4195569bff5c7940a6afbb3d373bff17293a533a3c98574dcfcdb118b3a57/userdata/shm major:0 minor:84 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f2508aca20577c2f409c061e53ac97f8b3464b791954ba175ee8f9ef6d4c76f8/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f2508aca20577c2f409c061e53ac97f8b3464b791954ba175ee8f9ef6d4c76f8/userdata/shm major:0 minor:230 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f555f9b5cf699ae142da23c63edc76a2380d32f62058b986654cf82116d1c660/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f555f9b5cf699ae142da23c63edc76a2380d32f62058b986654cf82116d1c660/userdata/shm major:0 minor:245 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/039cf439-d12d-450a-80c1-eb3efeb50e8a/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/039cf439-d12d-450a-80c1-eb3efeb50e8a/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:207 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/039cf439-d12d-450a-80c1-eb3efeb50e8a/volumes/kubernetes.io~projected/kube-api-access-zrrdm:{mountpoint:/var/lib/kubelet/pods/039cf439-d12d-450a-80c1-eb3efeb50e8a/volumes/kubernetes.io~projected/kube-api-access-zrrdm major:0 minor:209 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/039cf439-d12d-450a-80c1-eb3efeb50e8a/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/039cf439-d12d-450a-80c1-eb3efeb50e8a/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:205 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/039cf439-d12d-450a-80c1-eb3efeb50e8a/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/039cf439-d12d-450a-80c1-eb3efeb50e8a/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/06a0877a-2fce-405f-b97c-091d88e19543/volumes/kubernetes.io~projected/kube-api-access-l5qjc:{mountpoint:/var/lib/kubelet/pods/06a0877a-2fce-405f-b97c-091d88e19543/volumes/kubernetes.io~projected/kube-api-access-l5qjc major:0 minor:334 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/08498890-fb63-4aea-9cf7-a2ade5397df2/volumes/kubernetes.io~projected/kube-api-access-hdf4s:{mountpoint:/var/lib/kubelet/pods/08498890-fb63-4aea-9cf7-a2ade5397df2/volumes/kubernetes.io~projected/kube-api-access-hdf4s major:0 minor:229 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0bbc8570-4947-455a-b7f8-8e9fa8e023b3/volumes/kubernetes.io~projected/kube-api-access-9qf6w:{mountpoint:/var/lib/kubelet/pods/0bbc8570-4947-455a-b7f8-8e9fa8e023b3/volumes/kubernetes.io~projected/kube-api-access-9qf6w major:0 minor:335 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0d83c70b-fab8-4872-969f-11f74c0ddea0/volumes/kubernetes.io~projected/kube-api-access-9wp72:{mountpoint:/var/lib/kubelet/pods/0d83c70b-fab8-4872-969f-11f74c0ddea0/volumes/kubernetes.io~projected/kube-api-access-9wp72 major:0 minor:210 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/11b44bf1-b3b1-4214-b7b4-4e508c618cd1/volumes/kubernetes.io~projected/kube-api-access-8f74s:{mountpoint:/var/lib/kubelet/pods/11b44bf1-b3b1-4214-b7b4-4e508c618cd1/volumes/kubernetes.io~projected/kube-api-access-8f74s major:0 minor:352 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/11b44bf1-b3b1-4214-b7b4-4e508c618cd1/volumes/kubernetes.io~secret/signing-key:{mountpoint:/var/lib/kubelet/pods/11b44bf1-b3b1-4214-b7b4-4e508c618cd1/volumes/kubernetes.io~secret/signing-key major:0 minor:350 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3175ff47-1038-4139-9c0a-87062bb38dc2/volumes/kubernetes.io~projected/kube-api-access-zfphc:{mountpoint:/var/lib/kubelet/pods/3175ff47-1038-4139-9c0a-87062bb38dc2/volumes/kubernetes.io~projected/kube-api-access-zfphc major:0 minor:227 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3175ff47-1038-4139-9c0a-87062bb38dc2/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/3175ff47-1038-4139-9c0a-87062bb38dc2/volumes/kubernetes.io~secret/cert major:0 minor:220 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/352b5b4c-c640-4300-abf3-aa70b08cbb4b/volumes/kubernetes.io~projected/kube-api-access-96bzs:{mountpoint:/var/lib/kubelet/pods/352b5b4c-c640-4300-abf3-aa70b08cbb4b/volumes/kubernetes.io~projected/kube-api-access-96bzs major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3913f7d4-86cb-4e86-af5a-debd8b859252/volumes/kubernetes.io~projected/kube-api-access-ms8rv:{mountpoint:/var/lib/kubelet/pods/3913f7d4-86cb-4e86-af5a-debd8b859252/volumes/kubernetes.io~projected/kube-api-access-ms8rv major:0 minor:288 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3c835b41-25cb-4f0c-8e7c-4ac9e2fee318/volumes/kubernetes.io~projected/kube-api-access-pls28:{mountpoint:/var/lib/kubelet/pods/3c835b41-25cb-4f0c-8e7c-4ac9e2fee318/volumes/kubernetes.io~projected/kube-api-access-pls28 major:0 minor:211 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/48efa9dc-3c20-4f81-86f3-c87ad697772c/volumes/kubernetes.io~projected/kube-api-access-vtmzr:{mountpoint:/var/lib/kubelet/pods/48efa9dc-3c20-4f81-86f3-c87ad697772c/volumes/kubernetes.io~projected/kube-api-access-vtmzr major:0 minor:65 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5d90d533-ebca-46db-ab59-733837db23e6/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/5d90d533-ebca-46db-ab59-733837db23e6/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5d90d533-ebca-46db-ab59-733837db23e6/volumes/kubernetes.io~projected/kube-api-access-rf7bl:{mountpoint:/var/lib/kubelet/pods/5d90d533-ebca-46db-ab59-733837db23e6/volumes/kubernetes.io~projected/kube-api-access-rf7bl major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5d90d533-ebca-46db-ab59-733837db23e6/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/5d90d533-ebca-46db-ab59-733837db23e6/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5e93253e-a238-4426-a584-b9de8e931cf7/volumes/kubernetes.io~projected/kube-api-access-2trdz:{mountpoint:/var/lib/kubelet/pods/5e93253e-a238-4426-a584-b9de8e931cf7/volumes/kubernetes.io~projected/kube-api-access-2trdz major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5f171661-33a2-4631-81c7-bc10a5db791f/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/5f171661-33a2-4631-81c7-bc10a5db791f/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:293 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/67292bdc-7fe1-449d-af75-2baa90c16ea9/volumes/kubernetes.io~projected/kube-api-access-m8km8:{mountpoint:/var/lib/kubelet/pods/67292bdc-7fe1-449d-af75-2baa90c16ea9/volumes/kubernetes.io~projected/kube-api-access-m8km8 major:0 minor:219 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/67292bdc-7fe1-449d-af75-2baa90c16ea9/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/67292bdc-7fe1-449d-af75-2baa90c16ea9/volumes/kubernetes.io~secret/serving-cert major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7deac9ed-2d3a-48c3-bcf8-566ffdef0869/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/7deac9ed-2d3a-48c3-bcf8-566ffdef0869/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7deac9ed-2d3a-48c3-bcf8-566ffdef0869/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/7deac9ed-2d3a-48c3-bcf8-566ffdef0869/volumes/kubernetes.io~empty-dir/tmp major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7deac9ed-2d3a-48c3-bcf8-566ffdef0869/volumes/kubernetes.io~projected/kube-api-access-dv6wd:{mountpoint:/var/lib/kubelet/pods/7deac9ed-2d3a-48c3-bcf8-566ffdef0869/volumes/kubernetes.io~projected/kube-api-access-dv6wd major:0 minor:64 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/84a732a8-045d-4a8b-bb54-859c8d46f08d/volumes/kubernetes.io~projected/kube-api-access-w5tf5:{mountpoint:/var/lib/kubelet/pods/84a732a8-045d-4a8b-bb54-859c8d46f08d/volumes/kubernetes.io~projected/kube-api-access-w5tf5 major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8873f9b9-2ac5-409c-9299-fa024e00189c/volumes/kubernetes.io~projected/kube-api-access-jsdkt:{mountpoint:/var/lib/kubelet/pods/8873f9b9-2ac5-409c-9299-fa024e00189c/volumes/kubernetes.io~projected/kube-api-access-jsdkt major:0 minor:218 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8873f9b9-2ac5-409c-9299-fa024e00189c/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/8873f9b9-2ac5-409c-9299-fa024e00189c/volumes/kubernetes.io~secret/serving-cert major:0 minor:206 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8bb24a0d-3af0-4a05-bba4-4f9630b04bf3/volumes/kubernetes.io~projected/kube-api-access-spppx:{mountpoint:/var/lib/kubelet/pods/8bb24a0d-3af0-4a05-bba4-4f9630b04bf3/volumes/kubernetes.io~projected/kube-api-access-spppx major:0 minor:215 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8bb24a0d-3af0-4a05-bba4-4f9630b04bf3/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/8bb24a0d-3af0-4a05-bba4-4f9630b04bf3/volumes/kubernetes.io~secret/serving-cert major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/90937bc9-0051-407d-9c82-014f0a9c65d4/volumes/kubernetes.io~projected/kube-api-access-56qdr:{mountpoint:/var/lib/kubelet/pods/90937bc9-0051-407d-9c82-014f0a9c65d4/volumes/kubernetes.io~projected/kube-api-access-56qdr major:0 minor:213 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/94ee3757-5440-42e1-8617-2a68744ef3f4/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/94ee3757-5440-42e1-8617-2a68744ef3f4/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:226 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/94ee3757-5440-42e1-8617-2a68744ef3f4/volumes/kubernetes.io~projected/kube-api-access-t6lpv:{mountpoint:/var/lib/kubelet/pods/94ee3757-5440-42e1-8617-2a68744ef3f4/volumes/kubernetes.io~projected/kube-api-access-t6lpv major:0 minor:228 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/94ee3757-5440-42e1-8617-2a68744ef3f4/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/94ee3757-5440-42e1-8617-2a68744ef3f4/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:221 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/94ee3757-5440-42e1-8617-2a68744ef3f4/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/94ee3757-5440-42e1-8617-2a68744ef3f4/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:222 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9b4f9524-db75-4206-aa76-04400d95706f/volumes/kubernetes.io~projected/kube-api-access-wfrc4:{mountpoint:/var/lib/kubelet/pods/9b4f9524-db75-4206-aa76-04400d95706f/volumes/kubernetes.io~projected/kube-api-access-wfrc4 major:0 minor:212 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9b4f9524-db75-4206-aa76-04400d95706f/volumes/kubernetes.io~secret/default-certificate:{mountpoint:/var/lib/kubelet/pods/9b4f9524-db75-4206-aa76-04400d95706f/volumes/kubernetes.io~secret/default-certificate major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9b4f9524-db75-4206-aa76-04400d95706f/volumes/kubernetes.io~secret/stats-auth:{mountpoint:/var/lib/kubelet/pods/9b4f9524-db75-4206-aa76-04400d95706f/volumes/kubernetes.io~secret/stats-auth major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9f54b2c6-b1f9-42da-abdc-275fc8827b71/volumes/kubernetes.io~projected/kube-api-access-jqwd9:{mountpoint:/var/lib/kubelet/pods/9f54b2c6-b1f9-42da-abdc-275fc8827b71/volumes/kubernetes.io~projected/kube-api-access-jqwd9 major:0 minor:225 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/baa8d0c0-d477-4f30-a0f6-375fc84cc448/volumes/kubernetes.io~projected/kube-api-access-klwsc:{mountpoint:/var/lib/kubelet/pods/baa8d0c0-d477-4f30-a0f6-375fc84cc448/volumes/kubernetes.io~projected/kube-api-access-klwsc major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bba1a087-6e04-4db8-ba82-7260a88e2843/volumes/kubernetes.io~projected/kube-api-access-qcgbg:{mountpoint:/var/lib/kubelet/pods/bba1a087-6e04-4db8-ba82-7260a88e2843/volumes/kubernetes.io~projected/kube-api-access-qcgbg major:0 minor:208 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cf90e1fc-bac9-4c33-8dc4-dbca18974399/volumes/kubernetes.io~projected/kube-api-access-ct4zm:{mountpoint:/var/lib/kubelet/pods/cf90e1fc-bac9-4c33-8dc4-dbca18974399/volumes/kubernetes.io~projected/kube-api-access-ct4zm major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d6ca21b9-20a4-4887-9d08-19d9c491a09e/volumes/kubernetes.io~projected/kube-api-access-pklf4:{mountpoint:/var/lib/kubelet/pods/d6ca21b9-20a4-4887-9d08-19d9c491a09e/volumes/kubernetes.io~projected/kube-api-access-pklf4 major:0 minor:216 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d6ca21b9-20a4-4887-9d08-19d9c491a09e/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/d6ca21b9-20a4-4887-9d08-19d9c491a09e/volumes/kubernetes.io~secret/serving-cert major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e9d2f04b-bb7d-422f-b497-5fa61580f95a/volumes/kubernetes.io~projected/kube-api-access-bzmzn:{mountpoint:/var/lib/kubelet/pods/e9d2f04b-bb7d-422f-b497-5fa61580f95a/volumes/kubernetes.io~projected/kube-api-access-bzmzn major:0 minor:214 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ee6de179-7e35-46a5-86e1-c7c31428c149/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/ee6de179-7e35-46a5-86e1-c7c31428c149/volumes/kubernetes.io~secret/agent-certs major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f6750ff3-d2db-42b7-9c85-7896c9ba67c9/volumes/kubernetes.io~projected/kube-api-access-qhvst:{mountpoint:/var/lib/kubelet/pods/f6750ff3-d2db-42b7-9c85-7896c9ba67c9/volumes/kubernetes.io~projected/kube-api-access-qhvst major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/faca011a-4ff1-4de0-9af1-ed9b0f253aeb/volumes/kubernetes.io~projected/kube-api-access-dhrs4:{mountpoint:/var/lib/kubelet/pods/faca011a-4ff1-4de0-9af1-ed9b0f253aeb/volumes/kubernetes.io~projected/kube-api-access-dhrs4 major:0 minor:333 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fb314739-c678-4da8-9d54-147dc0612ee2/volumes/kubernetes.io~projected/kube-api-access-ffq8c:{mountpoint:/var/lib/kubelet/pods/fb314739-c678-4da8-9d54-147dc0612ee2/volumes/kubernetes.io~projected/kube-api-access-ffq8c major:0 minor:217 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fd34a723-6c9e-48f1-ba7d-924d542b2bc7/volumes/kubernetes.io~projected/kube-api-access-fnx5t:{mountpoint:/var/lib/kubelet/pods/fd34a723-6c9e-48f1-ba7d-924d542b2bc7/volumes/kubernetes.io~projected/kube-api-access-fnx5t major:0 minor:330 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/035ff0f39b96a9aa189d59bb0b7e91e35e5d2d403a3365f79d6c4e9b6eeb06dc/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/24f12434bea94c736908cd4e2415ed986a40639719f63303cbad44405b8c22c0/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/fcc1b109661c845c0a860cf228813f783341df4f7296b6e4f8e5fd5619b20170/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-106:{mountpoint:/var/lib/containers/storage/overlay/a10f6dbf8da0b1729efb650d11bed98415e2fd28ac4ae9a39bf545f734e6d0e2/merged major:0 minor:106 fsType:overlay blockSize:0} overlay_0-111:{mountpoint:/var/lib/containers/storage/overlay/9e4953982abe677429db39586716b77494f04974d248d374ba0920beb9908ec4/merged major:0 minor:111 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/f53e8c2b5176451c11341c2d5f53be9143c6e0b31e4649b4e5a159dba681ebb5/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/1f4bb2b61bae70ab91d9f82d4cb92d8943a887c839e93fd0fdfe9d57bd1be12b/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/ffc863ed3ed2873f0240d0e0acf08dac605dcee417a36b0b5efc6e432ffb2df5/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/40420f30a663ad33930e49828a7bd6684317c9524cae5395959a24893e8b9018/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-137:{mountpoint:/var/lib/containers/storage/overlay/73c89e7e7f2eaebe48b3e3c3ec07f656d2ecb71572542e8fab88daa2d31a6e55/merged major:0 minor:137 fsType:overlay blockSize:0} overlay_0-139:{mountpoint:/var/lib/containers/storage/overlay/a7a86d3bef5be73439bf95ea65629b3a1fc6e525c23041762328c8d3eed8a9e1/merged major:0 minor:139 fsType:overlay blockSize:0} overlay_0-143:{mountpoint:/var/lib/containers/storage/overlay/d8bcf4f8974d242ec623235bdecc5c720192cdef693151d8b52a36645016a732/merged major:0 minor:143 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/eec41ed8067e17ec8ab250045419f8e8a490242e067b5088293636850d350fef/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/5957b5efb31facc129ca6abd529c109d9fdafac5d4b77f0b6ecd456a7ed02dc1/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/615b804a6c907d6129cac6434545eba2bf1240eac9c434e70f1bd3feafebc50e/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/46f1f30954b6df8db09ea6aafed742764562ec82011100218c1eb7e9bb3eee09/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-174:{mountpoint:/var/lib/containers/storage/overlay/83dc77fa52b96968c0fde56cfeaef9bf6ab0fa547827264505d3d0dc978b1edf/merged major:0 minor:174 fsType:overlay blockSize:0} overlay_0-179:{mountpoint:/var/lib/containers/storage/overlay/c2f46af9bbdf71fc6523c1d92d4870fce3e8a75aa4a9e127c1e4f00304791b64/merged major:0 minor:179 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/1dc24644fc3ca46f58d06488a6d75daec737ea96496ffffb728bc0083283bbb1/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/c095b98ed705b540f8c8018ff25928a2e9cb981ceb6249ba14f51685dbdfca41/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/db49fca5022482cd32d497fb45361e8cc25a6680e8fb560dd9f91e541d0885fa/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-248:{mountpoint:/var/lib/containers/storage/overlay/a427a68e6fb6ed0ebdcf99d9b05142830f7c8c099e4bbb64eb6940f753f9db8f/merged major:0 minor:248 fsType:overlay blockSize:0} overlay_0-250:{mountpoint:/var/lib/containers/storage/overlay/f9408999e97a18483436122ce863057b75ec81a6dcc12c73151fb473da18e1d1/merged major:0 minor:250 fsType:overlay blockSize:0} overlay_0-252:{mountpoint:/var/lib/containers/storage/overlay/9d0ed6c8715375d81390270bf34b28fd6ee7c7ad5659e4e16f8c013611b92603/merged major:0 minor:252 fsType:overlay blockSize:0} overlay_0-254:{mountpoint:/var/lib/containers/storage/overlay/55a339bdb465b81c2b733bc629163d1b3de87e3a62ac58e6b1bec20c16f50c3f/merged major:0 minor:254 fsType:overlay blockSize:0} overlay_0-256:{mountpoint:/var/lib/containers/storage/overlay/c8a6a4d7407988987641fe8844b594bfdda79b960814ecca55569eec8d08031c/merged major:0 minor:256 fsType:overlay blockSize:0} overlay_0-258:{mountpoint:/var/lib/containers/storage/overlay/c8f8c3767c012b7bbcdd6014e2d5a5e984088b6fae2b203ddcf13e3479545378/merged major:0 minor:258 fsType:overlay blockSize:0} overlay_0-260:{mountpoint:/var/lib/containers/storage/overlay/02a983c9e9677d4934733dcd6edbc04582fdc11158695fe3baa2f3d12fb02c85/merged major:0 minor:260 fsType:overlay blockSize:0} overlay_0-262:{mountpoint:/var/lib/containers/storage/overlay/220a69ea4d0c4845fbe6367e50643c099f5dacbc231d0ff6e0992519d4259d47/merged major:0 minor:262 fsType:overlay blockSize:0} overlay_0-264:{mountpoint:/var/lib/containers/storage/overlay/4e081f8db64abb7b2e57a43aa307b99808befde321d4baf4577a1265f7c108f3/merged major:0 minor:264 fsType:overlay blockSize:0} overlay_0-265:{mountpoint:/var/lib/containers/storage/overlay/df6f296d0f212a0c48482e2f5edad50944d82883973199d5bebb87fcc793d349/merged major:0 minor:265 fsType:overlay blockSize:0} overlay_0-274:{mountpoint:/var/lib/containers/storage/overlay/0f03ce5d7186d4effb077ec98880365c753c76bdfa0b8f5f41acf331344a9a17/merged major:0 minor:274 fsType:overlay blockSize:0} overlay_0-276:{mountpoint:/var/lib/containers/storage/overlay/e3197330896dc5058505f1e01267f7b71dad6eb80fa0ae8cc30e121b34400327/merged major:0 minor:276 fsType:overlay blockSize:0} overlay_0-278:{mountpoint:/var/lib/containers/storage/overlay/8d0d96f959eacbd4f4a3ddfb73843d1b2e44ed55bfbf2b7cbc25c9500133e92a/merged major:0 minor:278 fsType:overlay blockSize:0} overlay_0-280:{mountpoint:/var/lib/containers/storage/overlay/1de24dee5069c57974168491c514ebb5831dfcf25008878108a4709c2e502956/merged major:0 minor:280 fsType:overlay blockSize:0} overlay_0-298:{mountpoint:/var/lib/containers/storage/overlay/6e6c0c80dc93dd8c57e24aadaa5a1dbf699124757b0bd78f71f927031f28e2a0/merged major:0 minor:298 fsType:overlay blockSize:0} overlay_0-300:{mountpoint:/var/lib/containers/storage/overlay/e3acb305ae322f94a42bacd84b29da87e6dc6e000873f12692e4794795ce17bf/merged major:0 minor:300 fsType:overlay blockSize:0} overlay_0-302:{mountpoint:/var/lib/containers/storage/overlay/793850bfb9a9e02f9f3f73fa4e2a2ed3ba3c6e73feb70c7d63479db17f3041fb/merged major:0 minor:302 fsType:overlay blockSize:0} overlay_0-304:{mountpoint:/var/lib/containers/storage/overlay/1633d2acc70557951d5e239f399473b1d5f6b272427a9b4c7d1766acfac9d3fa/merged major:0 minor:304 fsType:overlay blockSize:0} overlay_0-310:{mountpoint:/var/lib/containers/storage/overlay/954ebe075badffed44ffd53d04360db3c6b651e56f2bb8244fa031124e6436d4/merged major:0 minor:310 fsType:overlay blockSize:0} overlay_0-315:{mountpoint:/var/lib/containers/storage/overlay/d3bc06f528c8ce1f2a3c8fe8863584b58b134e58ddbf51e9685d321066e52bbc/merged major:0 minor:315 fsType:overlay blockSize:0} overlay_0-331:{mountpoint:/var/lib/containers/storage/overlay/78ad12ad6f7145f0970a941d7270d597df9147fcb6aabe089e15f0c731c3c5ff/merged major:0 minor:331 fsType:overlay blockSize:0} overlay_0-336:{mountpoint:/var/lib/containers/storage/overlay/e2b83fa3dcc35523af6b90d7070342ba862bbba2f09471d2213bb4a7e69cc1b9/merged major:0 minor:336 fsType:overlay blockSize:0} overlay_0-342:{mountpoint:/var/lib/containers/storage/overlay/04a67c8e40d429f3b3df0c3e3f6aa459fc77422a378f03437c6be6afc4f9365d/merged major:0 minor:342 fsType:overlay blockSize:0} overlay_0-346:{mountpoint:/var/lib/containers/storage/overlay/9c35372a866fdd747f92b4d9cf99071eb3b2e8ace4c3315f84f2d041fd317d25/merged major:0 minor:346 fsType:overlay blockSize:0} overlay_0-348:{mountpoint:/var/lib/containers/storage/overlay/9b3953cc0e639d6ff1beac7a413e4958759187c2cb99c1586910a2798ac71893/merged major:0 minor:348 fsType:overlay blockSize:0} overlay_0-358:{mountpoint:/var/lib/containers/storage/overlay/1d1ae5dbd8c05cfbdb01e6d5e44c2b59c2ba74c6a1b53bbd76c93fefa01d65aa/merged major:0 minor:358 fsType:overlay blockSize:0} overlay_0-360:{mountpoint:/var/lib/containers/storage/overlay/c82981345e4472f0e6166e45dc4d391a646e5f305f8ba5cd4e58893e583a295d/merged major:0 minor:360 fsType:overlay blockSize:0} overlay_0-366:{mountpoint:/var/lib/containers/storage/overlay/ec9f0786a1a6211bef4ed4e3813b07ea59c78596725c369f910c748dc0683ffb/merged major:0 minor:366 fsType:overlay blockSize:0} overlay_0-368:{mountpoint:/var/lib/containers/storage/overlay/e44841bf5a7d05638e1b4a48552ed81a60e127d21f8872a0c72716a5d57ac39e/merged major:0 minor:368 fsType:overlay blockSize:0} overlay_0-374:{mountpoint:/var/lib/containers/storage/overlay/dd518137d0ad9bf44c3eb8530ce76abd6fc643f9d64896abd6afce5f2fcf28db/merged major:0 minor:374 fsType:overlay blockSize:0} overlay_0-380:{mountpoint:/var/lib/containers/storage/overlay/65de75ab88933277af3c1c2b4c44d86212a87e1dbe342624c3255a2fb60a3bcb/merged major:0 minor:380 fsType:overlay blockSize:0} overlay_0-50:{mountpoint:/var/lib/containers/storage/overlay/d9d3c12fa68d04496ebeed0b96496220b15e30f75bde2fdaec64ab07358fd957/merged major:0 minor:50 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/9f91847dbfea7f6e22d8c4ccd81b2eedaf0d76a347205b15a7d06242af4f9cb6/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/a41efafae22d65f148411f9cf418543721ce73f58984316896ba2f1dd598100d/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/8a9d92bb27e22afa954b8ab4aa8e3cfff57bb9c55f90b9a2401046cdcdd147ac/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-79:{mountpoint:/var/lib/containers/storage/overlay/8be21e4201f40793df3c62e1ed325252fc93320d77b9c76b5f76bd490ae9d354/merged major:0 minor:79 fsType:overlay blockSize:0} overlay_0-82:{mountpoint:/var/lib/containers/storage/overlay/5cebc39160a717bda43434259d461954be64527f8e3c1c2576e4e1defe01620a/merged major:0 minor:82 fsType:overlay blockSize:0} overlay_0-86:{mountpoint:/var/lib/containers/storage/overlay/13a3ee6d1780c148cd94fee37639634be9e3553a219330f757942dbca1ac7162/merged major:0 minor:86 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/ba47ae2d6b0b7e8c1a568d9619622ba7b9a7ba11c88c3ea232f1548e6a112420/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-94:{mountpoint:/var/lib/containers/storage/overlay/a09a2107e0860b80a2233063aa0b7888d0bdd57fd7b468d44d5735fe8861e28c/merged major:0 minor:94 fsType:overlay blockSize:0} overlay_0-97:{mountpoint:/var/lib/containers/storage/overlay/017ba6b781fbcedc845c00d1f8e99f49fdf960e3eaa6932ed2b6964fb7853058/merged major:0 minor:97 fsType:overlay blockSize:0}]
Oct 30 21:28:07.292999 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.292601    5032 manager.go:217] Machine: {Timestamp:2026-10-30 21:28:07.292105051 +0000 UTC m=+0.093917507 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3103348 MemoryCapacity:33164337152 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2a4a41525858d008977e6d6a0fcd46 SystemUUID:ec2a4a41-5258-58d0-0897-7e6d6a0fcd46 BootID:fc49fec5-ea94-4d22-8da1-172f96334599 Filesystems:[{Device:overlay_0-174 DeviceMajor:0 DeviceMinor:174 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/04cbe888f55f0793a5c9408b6f252956dba81d6385e86f4366157ebd66e9a3e6/userdata/shm DeviceMajor:0 DeviceMinor:72 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/84a732a8-045d-4a8b-bb54-859c8d46f08d/volumes/kubernetes.io~projected/kube-api-access-w5tf5 DeviceMajor:0 DeviceMinor:55 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/94ee3757-5440-42e1-8617-2a68744ef3f4/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:222 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/0bbe505a0a4b7594e8e39d86bde05cbb8a3dd7cb5cb29b5128cb8d3b9cd43991/userdata/shm DeviceMajor:0 DeviceMinor:244 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b9c929b2079c2178d1e0dbb3a0cd5c7965226441feeb7576f7b91f47724b22ff/userdata/shm DeviceMajor:0 DeviceMinor:49 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/ee6de179-7e35-46a5-86e1-c7c31428c149/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:61 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-265 DeviceMajor:0 DeviceMinor:265 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1dca56b0734ec47a566bcaede4782a3c941f7018b609beabda556ecedc01fe47/userdata/shm DeviceMajor:0 DeviceMinor:344 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/f6750ff3-d2db-42b7-9c85-7896c9ba67c9/volumes/kubernetes.io~projected/kube-api-access-qhvst DeviceMajor:0 DeviceMinor:59 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d122ee4dad1efd8d9cfa8ca3bf428e14dcdb5a02d16d145d71957b4666f0361f/userdata/shm DeviceMajor:0 DeviceMinor:76 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-82 DeviceMajor:0 DeviceMinor:82 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-139 DeviceMajor:0 DeviceMinor:139 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/bba1a087-6e04-4db8-ba82-7260a88e2843/volumes/kubernetes.io~projected/kube-api-access-qcgbg DeviceMajor:0 DeviceMinor:208 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/90937bc9-0051-407d-9c82-014f0a9c65d4/volumes/kubernetes.io~projected/kube-api-access-56qdr DeviceMajor:0 DeviceMinor:213 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/8873f9b9-2ac5-409c-9299-fa024e00189c/volumes/kubernetes.io~projected/kube-api-access-jsdkt DeviceMajor:0 DeviceMinor:218 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f555f9b5cf699ae142da23c63edc76a2380d32f62058b986654cf82116d1c660/userdata/shm DeviceMajor:0 DeviceMinor:245 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-254 DeviceMajor:0 DeviceMinor:254 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-256 DeviceMajor:0 DeviceMinor:256 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-258 DeviceMajor:0 DeviceMinor:258 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-264 DeviceMajor:0 DeviceMinor:264 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-302 DeviceMajor:0 DeviceMinor:302 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:/var/lib/kubelet/pods/0d83c70b-fab8-4872-969f-11f74c0ddea0/volumes/kubernetes.io~projected/kube-api-access-9wp72 DeviceMajor:0 DeviceMinor:210 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-252 DeviceMajor:0 DeviceMinor:252 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-304 DeviceMajor:0 DeviceMinor:304 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-366 DeviceMajor:0 DeviceMinor:366 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-368 DeviceMajor:0 DeviceMinor:368 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/5d90d533-ebca-46db-ab59-733837db23e6/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/9b4f9524-db75-4206-aa76-04400d95706f/volumes/kubernetes.io~secret/default-certificate DeviceMajor:0 DeviceMinor:197 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-298 DeviceMajor:0 DeviceMinor:298 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-310 DeviceMajor:0 DeviceMinor:310 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/11b44bf1-b3b1-4214-b7b4-4e508c618cd1/volumes/kubernetes.io~secret/signing-key DeviceMajor:0 DeviceMinor:350 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1567ec928ebf4029842c9202f9fbc1ca86870cc5b45a9cf2214aa801ce31a73f/userdata/shm DeviceMajor:0 DeviceMinor:88 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-137 DeviceMajor:0 DeviceMinor:137 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3175ff47-1038-4139-9c0a-87062bb38dc2/volumes/kubernetes.io~projected/kube-api-access-zfphc DeviceMajor:0 DeviceMinor:227 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/0f738ceac995d4100163621b468e95def6099e0a6b0e9758ea5f3b383bb595f4/userdata/shm DeviceMajor:0 DeviceMinor:238 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-86 DeviceMajor:0 DeviceMinor:86 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8873f9b9-2ac5-409c-9299-fa024e00189c/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:206 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/8bb24a0d-3af0-4a05-bba4-4f9630b04bf3/volumes/kubernetes.io~projected/kube-api-access-spppx DeviceMajor:0 DeviceMinor:215 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/fb314739-c678-4da8-9d54-147dc0612ee2/volumes/kubernetes.io~projected/kube-api-access-ffq8c DeviceMajor:0 DeviceMinor:217 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/5e93253e-a238-4426-a584-b9de8e931cf7/volumes/kubernetes.io~projected/kube-api-access-2trdz DeviceMajor:0 DeviceMinor:56 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ef6c6d4940b60a96a28dda299962451ccbe7af14eef1e415eaf6be3ef995c61d/userdata/shm DeviceMajor:0 DeviceMinor:234 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-276 DeviceMajor:0 DeviceMinor:276 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-336 DeviceMajor:0 DeviceMinor:336 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-346 DeviceMajor:0 DeviceMinor:346 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/11b44bf1-b3b1-4214-b7b4-4e508c618cd1/volumes/kubernetes.io~projected/kube-api-access-8f74s DeviceMajor:0 DeviceMinor:352 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/7deac9ed-2d3a-48c3-bcf8-566ffdef0869/volumes/kubernetes.io~projected/kube-api-access-dv6wd DeviceMajor:0 DeviceMinor:64 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-111 DeviceMajor:0 DeviceMinor:111 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/039cf439-d12d-450a-80c1-eb3efeb50e8a/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:205 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/3175ff47-1038-4139-9c0a-87062bb38dc2/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:220 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/94ee3757-5440-42e1-8617-2a68744ef3f4/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:221 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/914ffc5c554a5c4122f2c41a837189ca7dc51ff6991d0eb637f078079903684a/userdata/shm DeviceMajor:0 DeviceMinor:232 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/5f171661-33a2-4631-81c7-bc10a5db791f/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:293 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/01a6be223caaa40916fbe86ec4ffed50b3b9de9f3f8b0241a8c368c01fa597a7/userdata/shm DeviceMajor:0 DeviceMinor:236 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-280 DeviceMajor:0 DeviceMinor:280 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/06a0877a-2fce-405f-b97c-091d88e19543/volumes/kubernetes.io~projected/kube-api-access-l5qjc DeviceMajor:0 DeviceMinor:334 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/var/lib/kubelet/pods/baa8d0c0-d477-4f30-a0f6-375fc84cc448/volumes/kubernetes.io~projected/kube-api-access-klwsc DeviceMajor:0 DeviceMinor:66 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-179 DeviceMajor:0 DeviceMinor:179 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-300 DeviceMajor:0 DeviceMinor:300 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-331 DeviceMajor:0 DeviceMinor:331 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/674d20fe97c246ae471b27f055ab576608b0a4cad5fabb0dca5b5ee7a31bfc48/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/039cf439-d12d-450a-80c1-eb3efeb50e8a/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:207 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-315 DeviceMajor:0 DeviceMinor:315 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/5f2f9bf4612d88a6d984149720b0481036f148988b145cddf801e00076f200b1/userdata/shm DeviceMajor:0 DeviceMinor:77 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d6ca21b9-20a4-4887-9d08-19d9c491a09e/volumes/kubernetes.io~projected/kube-api-access-pklf4 DeviceMajor:0 DeviceMinor:216 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-248 DeviceMajor:0 DeviceMinor:248 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-374 DeviceMajor:0 DeviceMinor:374 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/039cf439-d12d-450a-80c1-eb3efeb50e8a/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:204 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/75526b7530220d62f568ed172aa859dcd9a25086d66f706ee52dc359e8bdc322/userdata/shm DeviceMajor:0 DeviceMinor:223 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/94ee3757-5440-42e1-8617-2a68744ef3f4/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:226 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/94ee3757-5440-42e1-8617-2a68744ef3f4/volumes/kubernetes.io~projected/kube-api-access-t6lpv DeviceMajor:0 DeviceMinor:228 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/48efa9dc-3c20-4f81-86f3-c87ad697772c/volumes/kubernetes.io~projected/kube-api-access-vtmzr DeviceMajor:0 DeviceMinor:65 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/5d90d533-ebca-46db-ab59-733837db23e6/volumes/kubernetes.io~projected/kube-api-access-rf7bl DeviceMajor:0 DeviceMinor:67 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/0bc1321b91785136139aed9902de94cdf1a2c849b8813d68d5d7a26b1d104efb/userdata/shm DeviceMajor:0 DeviceMinor:240 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/cf90e1fc-bac9-4c33-8dc4-dbca18974399/volumes/kubernetes.io~projected/kube-api-access-ct4zm DeviceMajor:0 DeviceMinor:58 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/28d6df74a321bc7402b0991709090e293ce1a21aea1c20e6a53e533ecaaf619e/userdata/shm DeviceMajor:0 DeviceMinor:89 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-274 DeviceMajor:0 DeviceMinor:274 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/08498890-fb63-4aea-9cf7-a2ade5397df2/volumes/kubernetes.io~projected/kube-api-access-hdf4s DeviceMajor:0 DeviceMinor:229 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-260 DeviceMajor:0 DeviceMinor:260 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/faca011a-4ff1-4de0-9af1-ed9b0f253aeb/volumes/kubernetes.io~projected/kube-api-access-dhrs4 DeviceMajor:0 DeviceMinor:333 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-50 DeviceMajor:0 DeviceMinor:50 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d6ca21b9-20a4-4887-9d08-19d9c491a09e/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:203 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/5eceb5374b97fe1e5b74255fdff647f9a544bdccc3e50a91d984a973c5a74612/userdata/shm DeviceMajor:0 DeviceMinor:351 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-380 DeviceMajor:0 DeviceMinor:380 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-79 DeviceMajor:0 DeviceMinor:79 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/67292bdc-7fe1-449d-af75-2baa90c16ea9/volumes/kubernetes.io~projected/kube-api-access-m8km8 DeviceMajor:0 DeviceMinor:219 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/3913f7d4-86cb-4e86-af5a-debd8b859252/volumes/kubernetes.io~projected/kube-api-access-ms8rv DeviceMajor:0 DeviceMinor:288 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/352b5b4c-c640-4300-abf3-aa70b08cbb4b/volumes/kubernetes.io~projected/kube-api-access-96bzs DeviceMajor:0 DeviceMinor:57 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/7deac9ed-2d3a-48c3-bcf8-566ffdef0869/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:62 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/5d90d533-ebca-46db-ab59-733837db23e6/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:63 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/9f54b2c6-b1f9-42da-abdc-275fc8827b71/volumes/kubernetes.io~projected/kube-api-access-jqwd9 DeviceMajor:0 DeviceMinor:225 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1c916fbc0fbaa7a6b041fa9c07924c5020e909cd2b5516de2fae05b8d63f51a4/userdata/shm DeviceMajor:0 DeviceMinor:242 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a9272b555796f4e5bfa1e2b12cda99544643a87b628fa11b2b151adf30413d4c/userdata/shm DeviceMajor:0 DeviceMinor:323 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/bb1ff751df6656db7e8fef3bfd7a5baad5f96b46c870437d27222dd412553064/userdata/shm DeviceMajor:0 DeviceMinor:296 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/fd34a723-6c9e-48f1-ba7d-924d542b2bc7/volumes/kubernetes.io~projected/kube-api-access-fnx5t DeviceMajor:0 DeviceMinor:330 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f0e4195569bff5c7940a6afbb3d373bff17293a533a3c98574dcfcdb118b3a57/userdata/shm DeviceMajor:0 DeviceMinor:84 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-94 DeviceMajor:0 DeviceMinor:94 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8bb24a0d-3af0-4a05-bba4-4f9630b04bf3/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:201 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/9b4f9524-db75-4206-aa76-04400d95706f/volumes/kubernetes.io~projected/kube-api-access-wfrc4 DeviceMajor:0 DeviceMinor:212 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/039cf439-d12d-450a-80c1-eb3efeb50e8a/volumes/kubernetes.io~projected/kube-api-access-zrrdm DeviceMajor:0 DeviceMinor:209 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/57dbd87c6c23e36cb2934dbe29990855b3d7f1a4fedbd560cc649beff9706264/userdata/shm DeviceMajor:0 DeviceMinor:96 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f2508aca20577c2f409c061e53ac97f8b3464b791954ba175ee8f9ef6d4c76f8/userdata/shm DeviceMajor:0 DeviceMinor:230 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-250 DeviceMajor:0 DeviceMinor:250 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/868d0e77ebffb05386c7b1e5376b26ba106aa3b18521844b1d33ca992dd6d4cc/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/3c835b41-25cb-4f0c-8e7c-4ac9e2fee318/volumes/kubernetes.io~projected/kube-api-access-pls28 DeviceMajor:0 DeviceMinor:211 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/0bbc8570-4947-455a-b7f8-8e9fa8e023b3/volumes/kubernetes.io~projected/kube-api-access-9qf6w DeviceMajor:0 DeviceMinor:335 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-348 DeviceMajor:0 DeviceMinor:348 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-358 DeviceMajor:0 DeviceMinor:358 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-262 DeviceMajor:0 DeviceMinor:262 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-278 DeviceMajor:0 DeviceMinor:278 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7deac9ed-2d3a-48c3-bcf8-566ffdef0869/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:60 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-106 DeviceMajor:0 DeviceMinor:106 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e9d2f04b-bb7d-422f-b497-5fa61580f95a/volumes/kubernetes.io~projected/kube-api-access-bzmzn DeviceMajor:0 DeviceMinor:214 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/25200764cd35dbf351705da7ea4eb6473438cd86a25b56a86438f0becd6fa64c/userdata/shm DeviceMajor:0 DeviceMinor:318 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-342 DeviceMajor:0 DeviceMinor:342 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-360 DeviceMajor:0 DeviceMinor:360 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-97 DeviceMajor:0 DeviceMinor:97 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/9b4f9524-db75-4206-aa76-04400d95706f/volumes/kubernetes.io~secret/stats-auth DeviceMajor:0 DeviceMinor:45 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/67292bdc-7fe1-449d-af75-2baa90c16ea9/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:202 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/7832534c8c7e80d5ba345d1e86c0d75998d77b27c1f4fba9dba1561978a6523b/userdata/shm DeviceMajor:0 DeviceMinor:294 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-143 DeviceMajor:0 DeviceMinor:143 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:01a6be223caaa40 MacAddress:92:04:7e:1e:7a:11 Speed:10000 Mtu:8901} {Name:0bbe505a0a4b759 MacAddress:fe:81:c6:23:d6:7a Speed:10000 Mtu:8901} {Name:0bc1321b9178513 MacAddress:e2:a3:42:9f:af:9d Speed:10000 Mtu:8901} {Name:0f738ceac995d41 MacAddress:16:c6:c6:54:c1:cf Speed:10000 Mtu:8901} {Name:1c916fbc0fbaa7a MacAddress:d2:09:dc:04:04:e3 Speed:10000 Mtu:8901} {Name:1dca56b0734ec47 MacAddress:5a:fa:16:d1:d8:fd Speed:10000 Mtu:8901} {Name:25200764cd35dbf MacAddress:5e:00:4f:e3:c5:10 Speed:10000 Mtu:8901} {Name:5eceb5374b97fe1 MacAddress:0e:6d:f0:18:c3:ee Speed:10000 Mtu:8901} {Name:75526b7530220d6 MacAddress:32:2f:2e:2d:1e:e4 Speed:10000 Mtu:8901} {Name:7832534c8c7e80d MacAddress:f6:22:43:99:65:d2 Speed:10000 Mtu:8901} {Name:914ffc5c554a5c4 MacAddress:2e:65:33:ad:7f:5e Speed:10000 Mtu:8901} {Name:a9272b555796f4e MacAddress:ba:09:6f:7c:60:ad Speed:10000 Mtu:8901} {Name:bb1ff751df6656d MacAddress:26:1a:e5:c8:89:c6 Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:ff:e8:3a:ae:1d Speed:0 Mtu:9001} {Name:br-int MacAddress:86:82:b9:5b:13:b9 Speed:0 Mtu:8901} {Name:ef6c6d4940b60a9 MacAddress:ee:c9:1c:25:62:e5 Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:ff:e8:3a:ae:1d Speed:0 Mtu:9001} {Name:f2508aca20577c2 MacAddress:b2:c9:09:75:d8:e8 Speed:10000 Mtu:8901} {Name:f555f9b5cf699ae MacAddress:da:42:41:e1:f8:fd Speed:10000 Mtu:8901} {Name:genev_sys_6081 MacAddress:36:4e:b7:5b:c6:3e Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:84:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:ca:ce:47:b9:ca:37 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164337152 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 21:28:07.293186 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.293003    5032 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 21:28:07.293186 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.293054    5032 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 21:28:07.293330 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.293310    5032 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 21:28:07.293539 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.293332    5032 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-195.us-east-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 21:28:07.293591 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.293543    5032 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 21:28:07.293591 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.293551    5032 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 21:28:07.293591 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.293563    5032 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 21:28:07.293591 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.293586    5032 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 21:28:07.293693 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.293606    5032 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 21:28:07.293777 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.293767    5032 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 21:28:07.293891 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.293882    5032 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 21:28:07.293933 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.293925    5032 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 21:28:07.293965 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.293936    5032 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 21:28:07.293965 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.293946    5032 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 21:28:07.293965 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.293955    5032 kubelet.go:405] "Adding apiserver pod source"
Oct 30 21:28:07.293965 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.293964    5032 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 21:28:07.294740 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.294720    5032 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 21:28:07.294740 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.294734    5032 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 21:28:07.295258 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.295242    5032 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 21:28:07.297560 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.297546    5032 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 21:28:07.297560 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.297559    5032 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 21:28:07.297669 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.297660    5032 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Oct 30 21:28:07.297859 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.297849    5032 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 21:28:07.297895 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.297878    5032 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 21:28:07.297982 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.297973    5032 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 21:28:07.298017 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.297989    5032 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 21:28:07.298017 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.298000    5032 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 21:28:07.298017 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.298007    5032 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 21:28:07.298017 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.298013    5032 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 21:28:07.298017 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.298018    5032 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 21:28:07.298147 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.298024    5032 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 21:28:07.298147 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.298029    5032 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 21:28:07.298147 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.298036    5032 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 21:28:07.298147 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.298042    5032 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 21:28:07.298147 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.298055    5032 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 21:28:07.298147 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.298066    5032 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 21:28:07.298147 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.298091    5032 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 21:28:07.298147 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.298097    5032 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 21:28:07.301662 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.301647    5032 server.go:1263] "Started kubelet"
Oct 30 21:28:07.302331 ip-10-0-129-195 systemd[1]: Started Kubernetes Kubelet.
Oct 30 21:28:07.303539 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.303503    5032 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 21:28:07.306968 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.302265    5032 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 21:28:07.306968 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.304755    5032 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 21:28:07.306968 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.305081    5032 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 21:28:07.306968 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.305537    5032 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 21:28:07.308124 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.308104    5032 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 21:28:07.310434 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.308386    5032 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 21:28:07.310434 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.308499    5032 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 21:28:07.310434 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.308514    5032 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 21:28:07.310434 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.308578    5032 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 21:28:07.310434 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.309015    5032 factory.go:55] Registering systemd factory
Oct 30 21:28:07.310434 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.309033    5032 factory.go:223] Registration of the systemd container factory successfully
Oct 30 21:28:07.310434 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.309253    5032 factory.go:153] Registering CRI-O factory
Oct 30 21:28:07.310434 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.309266    5032 factory.go:223] Registration of the crio container factory successfully
Oct 30 21:28:07.310434 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.309326    5032 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 21:28:07.310434 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.309347    5032 factory.go:103] Registering Raw factory
Oct 30 21:28:07.310434 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.309380    5032 manager.go:1199] Started watching for new ooms in manager
Oct 30 21:28:07.310434 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.309719    5032 manager.go:319] Starting recovery of all containers
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:28:07.310712    5032 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-195.us-east-2.compute.internal\" not found"
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.310994    5032 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 21:22:27 +0000 UTC" deadline="2028-06-01 10:11:38.222347772 +0000 UTC"
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.311036    5032 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13908h43m30.91131565s"
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.312539    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="11b44bf1-b3b1-4214-b7b4-4e508c618cd1" volumeName="kubernetes.io/secret/11b44bf1-b3b1-4214-b7b4-4e508c618cd1-signing-key" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.312577    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="352b5b4c-c640-4300-abf3-aa70b08cbb4b" volumeName="kubernetes.io/projected/352b5b4c-c640-4300-abf3-aa70b08cbb4b-kube-api-access-96bzs" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.312591    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8873f9b9-2ac5-409c-9299-fa024e00189c" volumeName="kubernetes.io/projected/8873f9b9-2ac5-409c-9299-fa024e00189c-kube-api-access-jsdkt" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.312621    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8873f9b9-2ac5-409c-9299-fa024e00189c" volumeName="kubernetes.io/secret/8873f9b9-2ac5-409c-9299-fa024e00189c-serving-cert" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.312634    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8bb24a0d-3af0-4a05-bba4-4f9630b04bf3" volumeName="kubernetes.io/secret/8bb24a0d-3af0-4a05-bba4-4f9630b04bf3-serving-cert" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.312647    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9b4f9524-db75-4206-aa76-04400d95706f" volumeName="kubernetes.io/secret/9b4f9524-db75-4206-aa76-04400d95706f-default-certificate" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.312659    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f6750ff3-d2db-42b7-9c85-7896c9ba67c9" volumeName="kubernetes.io/configmap/f6750ff3-d2db-42b7-9c85-7896c9ba67c9-cni-sysctl-allowlist" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.312671    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="48efa9dc-3c20-4f81-86f3-c87ad697772c" volumeName="kubernetes.io/projected/48efa9dc-3c20-4f81-86f3-c87ad697772c-kube-api-access-vtmzr" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.312687    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5e93253e-a238-4426-a584-b9de8e931cf7" volumeName="kubernetes.io/configmap/5e93253e-a238-4426-a584-b9de8e931cf7-multus-daemon-config" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.312700    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5f171661-33a2-4631-81c7-bc10a5db791f" volumeName="kubernetes.io/secret/5f171661-33a2-4631-81c7-bc10a5db791f-original-pull-secret" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.312713    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="67292bdc-7fe1-449d-af75-2baa90c16ea9" volumeName="kubernetes.io/configmap/67292bdc-7fe1-449d-af75-2baa90c16ea9-config" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.312726    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="84a732a8-045d-4a8b-bb54-859c8d46f08d" volumeName="kubernetes.io/configmap/84a732a8-045d-4a8b-bb54-859c8d46f08d-iptables-alerter-script" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.312739    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9f54b2c6-b1f9-42da-abdc-275fc8827b71" volumeName="kubernetes.io/projected/9f54b2c6-b1f9-42da-abdc-275fc8827b71-kube-api-access-jqwd9" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.312751    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ee6de179-7e35-46a5-86e1-c7c31428c149" volumeName="kubernetes.io/configmap/ee6de179-7e35-46a5-86e1-c7c31428c149-konnectivity-ca" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.312768    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="039cf439-d12d-450a-80c1-eb3efeb50e8a" volumeName="kubernetes.io/secret/039cf439-d12d-450a-80c1-eb3efeb50e8a-installation-pull-secrets" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.312780    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="753d4639-09e6-4850-9f4d-ed39b0406439" volumeName="kubernetes.io/configmap/753d4639-09e6-4850-9f4d-ed39b0406439-nginx-conf" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.312813    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="90937bc9-0051-407d-9c82-014f0a9c65d4" volumeName="kubernetes.io/configmap/90937bc9-0051-407d-9c82-014f0a9c65d4-telemetry-config" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.312841    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="90937bc9-0051-407d-9c82-014f0a9c65d4" volumeName="kubernetes.io/projected/90937bc9-0051-407d-9c82-014f0a9c65d4-kube-api-access-56qdr" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.312856    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="94ee3757-5440-42e1-8617-2a68744ef3f4" volumeName="kubernetes.io/configmap/94ee3757-5440-42e1-8617-2a68744ef3f4-registry-certificates" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.312868    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="94ee3757-5440-42e1-8617-2a68744ef3f4" volumeName="kubernetes.io/projected/94ee3757-5440-42e1-8617-2a68744ef3f4-bound-sa-token" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.312881    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="baa8d0c0-d477-4f30-a0f6-375fc84cc448" volumeName="kubernetes.io/projected/baa8d0c0-d477-4f30-a0f6-375fc84cc448-kube-api-access-klwsc" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.312893    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d6ca21b9-20a4-4887-9d08-19d9c491a09e" volumeName="kubernetes.io/secret/d6ca21b9-20a4-4887-9d08-19d9c491a09e-serving-cert" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.312906    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e9d2f04b-bb7d-422f-b497-5fa61580f95a" volumeName="kubernetes.io/projected/e9d2f04b-bb7d-422f-b497-5fa61580f95a-kube-api-access-bzmzn" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.312919    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f6750ff3-d2db-42b7-9c85-7896c9ba67c9" volumeName="kubernetes.io/configmap/f6750ff3-d2db-42b7-9c85-7896c9ba67c9-whereabouts-flatfile-configmap" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.312932    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5d90d533-ebca-46db-ab59-733837db23e6" volumeName="kubernetes.io/configmap/5d90d533-ebca-46db-ab59-733837db23e6-env-overrides" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.312945    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fd34a723-6c9e-48f1-ba7d-924d542b2bc7" volumeName="kubernetes.io/projected/fd34a723-6c9e-48f1-ba7d-924d542b2bc7-kube-api-access-fnx5t" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.312961    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3c835b41-25cb-4f0c-8e7c-4ac9e2fee318" volumeName="kubernetes.io/projected/3c835b41-25cb-4f0c-8e7c-4ac9e2fee318-kube-api-access-pls28" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.312986    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7deac9ed-2d3a-48c3-bcf8-566ffdef0869" volumeName="kubernetes.io/empty-dir/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-tmp" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.313030    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8bb24a0d-3af0-4a05-bba4-4f9630b04bf3" volumeName="kubernetes.io/projected/8bb24a0d-3af0-4a05-bba4-4f9630b04bf3-kube-api-access-spppx" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.313043    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9b4f9524-db75-4206-aa76-04400d95706f" volumeName="kubernetes.io/projected/9b4f9524-db75-4206-aa76-04400d95706f-kube-api-access-wfrc4" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.313056    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d6ca21b9-20a4-4887-9d08-19d9c491a09e" volumeName="kubernetes.io/configmap/d6ca21b9-20a4-4887-9d08-19d9c491a09e-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.313069    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d6ca21b9-20a4-4887-9d08-19d9c491a09e" volumeName="kubernetes.io/configmap/d6ca21b9-20a4-4887-9d08-19d9c491a09e-service-ca-bundle" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.313082    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f6750ff3-d2db-42b7-9c85-7896c9ba67c9" volumeName="kubernetes.io/configmap/f6750ff3-d2db-42b7-9c85-7896c9ba67c9-cni-binary-copy" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.313097    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="039cf439-d12d-450a-80c1-eb3efeb50e8a" volumeName="kubernetes.io/configmap/039cf439-d12d-450a-80c1-eb3efeb50e8a-registry-certificates" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.313111    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="08498890-fb63-4aea-9cf7-a2ade5397df2" volumeName="kubernetes.io/empty-dir/08498890-fb63-4aea-9cf7-a2ade5397df2-tmp-dir" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.313124    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="11b44bf1-b3b1-4214-b7b4-4e508c618cd1" volumeName="kubernetes.io/configmap/11b44bf1-b3b1-4214-b7b4-4e508c618cd1-signing-cabundle" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.313138    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="84a732a8-045d-4a8b-bb54-859c8d46f08d" volumeName="kubernetes.io/projected/84a732a8-045d-4a8b-bb54-859c8d46f08d-kube-api-access-w5tf5" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.313151    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="94ee3757-5440-42e1-8617-2a68744ef3f4" volumeName="kubernetes.io/secret/94ee3757-5440-42e1-8617-2a68744ef3f4-image-registry-private-configuration" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.313163    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fd34a723-6c9e-48f1-ba7d-924d542b2bc7" volumeName="kubernetes.io/configmap/fd34a723-6c9e-48f1-ba7d-924d542b2bc7-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.313175    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="48efa9dc-3c20-4f81-86f3-c87ad697772c" volumeName="kubernetes.io/configmap/48efa9dc-3c20-4f81-86f3-c87ad697772c-serviceca" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.313190    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="67292bdc-7fe1-449d-af75-2baa90c16ea9" volumeName="kubernetes.io/projected/67292bdc-7fe1-449d-af75-2baa90c16ea9-kube-api-access-m8km8" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.313203    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="94ee3757-5440-42e1-8617-2a68744ef3f4" volumeName="kubernetes.io/empty-dir/94ee3757-5440-42e1-8617-2a68744ef3f4-ca-trust-extracted" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.313217    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fd34a723-6c9e-48f1-ba7d-924d542b2bc7" volumeName="kubernetes.io/configmap/fd34a723-6c9e-48f1-ba7d-924d542b2bc7-service-ca-bundle" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.313230    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3175ff47-1038-4139-9c0a-87062bb38dc2" volumeName="kubernetes.io/projected/3175ff47-1038-4139-9c0a-87062bb38dc2-kube-api-access-zfphc" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.313243    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5d90d533-ebca-46db-ab59-733837db23e6" volumeName="kubernetes.io/configmap/5d90d533-ebca-46db-ab59-733837db23e6-ovnkube-script-lib" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.313258    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="039cf439-d12d-450a-80c1-eb3efeb50e8a" volumeName="kubernetes.io/secret/039cf439-d12d-450a-80c1-eb3efeb50e8a-image-registry-private-configuration" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.313272    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="08498890-fb63-4aea-9cf7-a2ade5397df2" volumeName="kubernetes.io/configmap/08498890-fb63-4aea-9cf7-a2ade5397df2-config-volume" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.313287    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5e93253e-a238-4426-a584-b9de8e931cf7" volumeName="kubernetes.io/configmap/5e93253e-a238-4426-a584-b9de8e931cf7-cni-binary-copy" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.313300    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fb314739-c678-4da8-9d54-147dc0612ee2" volumeName="kubernetes.io/projected/fb314739-c678-4da8-9d54-147dc0612ee2-kube-api-access-ffq8c" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.313315    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0d83c70b-fab8-4872-969f-11f74c0ddea0" volumeName="kubernetes.io/projected/0d83c70b-fab8-4872-969f-11f74c0ddea0-kube-api-access-9wp72" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.313328    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="11b44bf1-b3b1-4214-b7b4-4e508c618cd1" volumeName="kubernetes.io/projected/11b44bf1-b3b1-4214-b7b4-4e508c618cd1-kube-api-access-8f74s" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.313339    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3913f7d4-86cb-4e86-af5a-debd8b859252" volumeName="kubernetes.io/projected/3913f7d4-86cb-4e86-af5a-debd8b859252-kube-api-access-ms8rv" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.313352    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="67292bdc-7fe1-449d-af75-2baa90c16ea9" volumeName="kubernetes.io/secret/67292bdc-7fe1-449d-af75-2baa90c16ea9-serving-cert" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.313388    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8bb24a0d-3af0-4a05-bba4-4f9630b04bf3" volumeName="kubernetes.io/configmap/8bb24a0d-3af0-4a05-bba4-4f9630b04bf3-config" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.313399    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f6750ff3-d2db-42b7-9c85-7896c9ba67c9" volumeName="kubernetes.io/projected/f6750ff3-d2db-42b7-9c85-7896c9ba67c9-kube-api-access-qhvst" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.313412    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="039cf439-d12d-450a-80c1-eb3efeb50e8a" volumeName="kubernetes.io/configmap/039cf439-d12d-450a-80c1-eb3efeb50e8a-trusted-ca" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.313434    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="039cf439-d12d-450a-80c1-eb3efeb50e8a" volumeName="kubernetes.io/empty-dir/039cf439-d12d-450a-80c1-eb3efeb50e8a-ca-trust-extracted" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.313447    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="06a0877a-2fce-405f-b97c-091d88e19543" volumeName="kubernetes.io/projected/06a0877a-2fce-405f-b97c-091d88e19543-kube-api-access-l5qjc" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.313460    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5d90d533-ebca-46db-ab59-733837db23e6" volumeName="kubernetes.io/configmap/5d90d533-ebca-46db-ab59-733837db23e6-ovnkube-config" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.313472    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5d90d533-ebca-46db-ab59-733837db23e6" volumeName="kubernetes.io/projected/5d90d533-ebca-46db-ab59-733837db23e6-kube-api-access-rf7bl" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.313484    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fd34a723-6c9e-48f1-ba7d-924d542b2bc7" volumeName="kubernetes.io/empty-dir/fd34a723-6c9e-48f1-ba7d-924d542b2bc7-archives-path" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.313495    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="039cf439-d12d-450a-80c1-eb3efeb50e8a" volumeName="kubernetes.io/projected/039cf439-d12d-450a-80c1-eb3efeb50e8a-kube-api-access-zrrdm" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.313510    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3175ff47-1038-4139-9c0a-87062bb38dc2" volumeName="kubernetes.io/secret/3175ff47-1038-4139-9c0a-87062bb38dc2-cert" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.313522    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5e93253e-a238-4426-a584-b9de8e931cf7" volumeName="kubernetes.io/projected/5e93253e-a238-4426-a584-b9de8e931cf7-kube-api-access-2trdz" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.313535    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="baa8d0c0-d477-4f30-a0f6-375fc84cc448" volumeName="kubernetes.io/empty-dir/baa8d0c0-d477-4f30-a0f6-375fc84cc448-tmp-dir" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.313547    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d6ca21b9-20a4-4887-9d08-19d9c491a09e" volumeName="kubernetes.io/empty-dir/d6ca21b9-20a4-4887-9d08-19d9c491a09e-tmp" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.313558    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="94ee3757-5440-42e1-8617-2a68744ef3f4" volumeName="kubernetes.io/projected/94ee3757-5440-42e1-8617-2a68744ef3f4-kube-api-access-t6lpv" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.313568    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="94ee3757-5440-42e1-8617-2a68744ef3f4" volumeName="kubernetes.io/secret/94ee3757-5440-42e1-8617-2a68744ef3f4-installation-pull-secrets" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.313580    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9f54b2c6-b1f9-42da-abdc-275fc8827b71" volumeName="kubernetes.io/configmap/9f54b2c6-b1f9-42da-abdc-275fc8827b71-samples-operator-config" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.313591    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="faca011a-4ff1-4de0-9af1-ed9b0f253aeb" volumeName="kubernetes.io/projected/faca011a-4ff1-4de0-9af1-ed9b0f253aeb-kube-api-access-dhrs4" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.313604    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="039cf439-d12d-450a-80c1-eb3efeb50e8a" volumeName="kubernetes.io/projected/039cf439-d12d-450a-80c1-eb3efeb50e8a-bound-sa-token" seLinuxMountContext=""
Oct 30 21:28:07.313923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.313615    5032 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:28:07.317713 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.313615    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="08498890-fb63-4aea-9cf7-a2ade5397df2" volumeName="kubernetes.io/projected/08498890-fb63-4aea-9cf7-a2ade5397df2-kube-api-access-hdf4s" seLinuxMountContext=""
Oct 30 21:28:07.317713 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.314423    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0bbc8570-4947-455a-b7f8-8e9fa8e023b3" volumeName="kubernetes.io/empty-dir/0bbc8570-4947-455a-b7f8-8e9fa8e023b3-data-volume" seLinuxMountContext=""
Oct 30 21:28:07.317713 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.314445    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="67292bdc-7fe1-449d-af75-2baa90c16ea9" volumeName="kubernetes.io/configmap/67292bdc-7fe1-449d-af75-2baa90c16ea9-trusted-ca" seLinuxMountContext=""
Oct 30 21:28:07.317713 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.314459    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7deac9ed-2d3a-48c3-bcf8-566ffdef0869" volumeName="kubernetes.io/empty-dir/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-etc-tuned" seLinuxMountContext=""
Oct 30 21:28:07.317713 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.314472    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d6ca21b9-20a4-4887-9d08-19d9c491a09e" volumeName="kubernetes.io/empty-dir/d6ca21b9-20a4-4887-9d08-19d9c491a09e-snapshots" seLinuxMountContext=""
Oct 30 21:28:07.317713 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.314484    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d6ca21b9-20a4-4887-9d08-19d9c491a09e" volumeName="kubernetes.io/projected/d6ca21b9-20a4-4887-9d08-19d9c491a09e-kube-api-access-pklf4" seLinuxMountContext=""
Oct 30 21:28:07.317713 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.314496    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ee6de179-7e35-46a5-86e1-c7c31428c149" volumeName="kubernetes.io/secret/ee6de179-7e35-46a5-86e1-c7c31428c149-agent-certs" seLinuxMountContext=""
Oct 30 21:28:07.317713 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.314508    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0bbc8570-4947-455a-b7f8-8e9fa8e023b3" volumeName="kubernetes.io/projected/0bbc8570-4947-455a-b7f8-8e9fa8e023b3-kube-api-access-9qf6w" seLinuxMountContext=""
Oct 30 21:28:07.317713 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.314520    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5d90d533-ebca-46db-ab59-733837db23e6" volumeName="kubernetes.io/secret/5d90d533-ebca-46db-ab59-733837db23e6-ovn-node-metrics-cert" seLinuxMountContext=""
Oct 30 21:28:07.317713 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.314531    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7deac9ed-2d3a-48c3-bcf8-566ffdef0869" volumeName="kubernetes.io/projected/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-kube-api-access-dv6wd" seLinuxMountContext=""
Oct 30 21:28:07.317713 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.314542    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8873f9b9-2ac5-409c-9299-fa024e00189c" volumeName="kubernetes.io/configmap/8873f9b9-2ac5-409c-9299-fa024e00189c-config" seLinuxMountContext=""
Oct 30 21:28:07.317713 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.314553    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="94ee3757-5440-42e1-8617-2a68744ef3f4" volumeName="kubernetes.io/configmap/94ee3757-5440-42e1-8617-2a68744ef3f4-trusted-ca" seLinuxMountContext=""
Oct 30 21:28:07.317713 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.314564    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9b4f9524-db75-4206-aa76-04400d95706f" volumeName="kubernetes.io/secret/9b4f9524-db75-4206-aa76-04400d95706f-stats-auth" seLinuxMountContext=""
Oct 30 21:28:07.317713 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.314579    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bba1a087-6e04-4db8-ba82-7260a88e2843" volumeName="kubernetes.io/projected/bba1a087-6e04-4db8-ba82-7260a88e2843-kube-api-access-qcgbg" seLinuxMountContext=""
Oct 30 21:28:07.317713 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.314589    5032 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cf90e1fc-bac9-4c33-8dc4-dbca18974399" volumeName="kubernetes.io/projected/cf90e1fc-bac9-4c33-8dc4-dbca18974399-kube-api-access-ct4zm" seLinuxMountContext=""
Oct 30 21:28:07.317713 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.314601    5032 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 21:28:07.317713 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.314608    5032 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 21:28:07.317713 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.315127    5032 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:28:07.320076 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.319823    5032 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:28:07.331818 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.331791    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-sklxd_67292bdc-7fe1-449d-af75-2baa90c16ea9/console-operator/1.log"
Oct 30 21:28:07.331944 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.331911    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-sklxd_67292bdc-7fe1-449d-af75-2baa90c16ea9/console-operator/1.log"
Oct 30 21:28:07.333160 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.333140    5032 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Oct 30 21:28:07.336072 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.335955    5032 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 21:28:07.337352 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.337332    5032 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 21:28:07.337352 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.337351    5032 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 21:28:07.337500 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.337387    5032 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 21:28:07.337500 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:28:07.337430    5032 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Oct 30 21:28:07.341770 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.341703    5032 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:28:07.365149 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.365121    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-sklxd_67292bdc-7fe1-449d-af75-2baa90c16ea9/console-operator/1.log"
Oct 30 21:28:07.365301 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.365158    5032 generic.go:356] "Generic (PLEG): container finished" podID="67292bdc-7fe1-449d-af75-2baa90c16ea9" containerID="f667ffa124949edf8581923668ce1139c1ad553b96aeaa80c8bc06474a49998a" exitCode=255
Oct 30 21:28:07.371011 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.370987    5032 generic.go:356] "Generic (PLEG): container finished" podID="f6750ff3-d2db-42b7-9c85-7896c9ba67c9" containerID="4b1645c2c8087cf36011a87d81b2947223e8d7b98752f433167910385073a1b6" exitCode=0
Oct 30 21:28:07.371011 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.371007    5032 generic.go:356] "Generic (PLEG): container finished" podID="f6750ff3-d2db-42b7-9c85-7896c9ba67c9" containerID="e9e39c89fd50eacd050925cff6dd2516294bcbed1507a3777a28e1f700a36c6f" exitCode=0
Oct 30 21:28:07.371011 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.371013    5032 generic.go:356] "Generic (PLEG): container finished" podID="f6750ff3-d2db-42b7-9c85-7896c9ba67c9" containerID="601fac517d55b2bdddecc7d87a3b14947fc0797f8747034ff725d4044efa8c95" exitCode=0
Oct 30 21:28:07.371207 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.371019    5032 generic.go:356] "Generic (PLEG): container finished" podID="f6750ff3-d2db-42b7-9c85-7896c9ba67c9" containerID="ee78ec47141b73f993c9bea8c2dcf0df41883085d8accde67385072b698f82e7" exitCode=0
Oct 30 21:28:07.371207 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.371024    5032 generic.go:356] "Generic (PLEG): container finished" podID="f6750ff3-d2db-42b7-9c85-7896c9ba67c9" containerID="718d207492d450b1620381f78079cb900c0ea4c80634af5286e4a9875eb3af50" exitCode=0
Oct 30 21:28:07.371207 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.371029    5032 generic.go:356] "Generic (PLEG): container finished" podID="f6750ff3-d2db-42b7-9c85-7896c9ba67c9" containerID="1cc6b317cd9fddeb36db820d12c3cfb4540385856cdea18e9fa0cbb43d467e5b" exitCode=0
Oct 30 21:28:07.383977 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.383939    5032 generic.go:356] "Generic (PLEG): container finished" podID="7efe602e8ba1524abe391aabbb698457" containerID="a1435fa8c0ca5dc2d2b220c80044bade18bae4c3b25fd17682d861f4c413c59a" exitCode=0
Oct 30 21:28:07.415013 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.414986    5032 manager.go:324] Recovery completed
Oct 30 21:28:07.435492 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.435464    5032 cpu_manager.go:226] "Starting" policy="none"
Oct 30 21:28:07.435492 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.435482    5032 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 21:28:07.435666 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.435502    5032 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 21:28:07.435890 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.435622    5032 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Oct 30 21:28:07.436830 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.435887    5032 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Oct 30 21:28:07.436918 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.436836    5032 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Oct 30 21:28:07.436918 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.436847    5032 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Oct 30 21:28:07.436918 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.436856    5032 policy_none.go:50] "Start"
Oct 30 21:28:07.436918 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.436879    5032 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 21:28:07.436918 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.436898    5032 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 21:28:07.438151 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:28:07.438086    5032 kubelet.go:2545] "Skipping pod synchronization" err="container runtime status check may not have completed yet"
Oct 30 21:28:07.448207 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.448183    5032 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Oct 30 21:28:07.448207 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.448210    5032 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Oct 30 21:28:07.448354 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.448223    5032 policy_none.go:44] "Start"
Oct 30 21:28:07.453771 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.453737    5032 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 21:28:07.456494 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.456478    5032 manager.go:341] "Starting Device Plugin manager"
Oct 30 21:28:07.456592 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:28:07.456508    5032 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 21:28:07.456592 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.456520    5032 server.go:87] "Starting device plugin registration server"
Oct 30 21:28:07.456852 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.456838    5032 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 21:28:07.456916 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.456858    5032 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 21:28:07.456964 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.456941    5032 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 21:28:07.457064 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.457031    5032 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 21:28:07.457131 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.457076    5032 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 21:28:07.457131 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.457083    5032 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 21:28:07.457211 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.457131    5032 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T21:28:07.457059967Z","UUID":"01967858-0984-48c0-87df-df0ad1684570","Handler":null,"Name":"","Endpoint":""}
Oct 30 21:28:07.458846 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.458826    5032 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 21:28:07.458937 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.458866    5032 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 21:28:07.459206 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:28:07.459169    5032 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 21:28:07.557802 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.557757    5032 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:28:07.559318 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.559299    5032 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-195.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:28:07.559407 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.559337    5032 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-195.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:28:07.559407 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.559352    5032 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-195.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:28:07.559521 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.559425    5032 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-129-195.us-east-2.compute.internal"
Oct 30 21:28:07.572211 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.572180    5032 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-129-195.us-east-2.compute.internal"
Oct 30 21:28:07.572391 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.572279    5032 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-129-195.us-east-2.compute.internal"
Oct 30 21:28:07.639173 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.639064    5032 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-195.us-east-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-195.us-east-2.compute.internal"]
Oct 30 21:28:07.639814 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.639794    5032 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-195.us-east-2.compute.internal"
Oct 30 21:28:07.639891 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.639828    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-195.us-east-2.compute.internal" event={"ID":"7efe602e8ba1524abe391aabbb698457","Type":"ContainerStarted","Data":"b7f568d38ac5e35b04ee68347b079798ac640c11b6fa76d1e54e73b9d3905ba9"}
Oct 30 21:28:07.639891 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.639881    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-195.us-east-2.compute.internal" event={"ID":"7efe602e8ba1524abe391aabbb698457","Type":"ContainerDied","Data":"a1435fa8c0ca5dc2d2b220c80044bade18bae4c3b25fd17682d861f4c413c59a"}
Oct 30 21:28:07.639956 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.639898    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-195.us-east-2.compute.internal" event={"ID":"7efe602e8ba1524abe391aabbb698457","Type":"ContainerStarted","Data":"b9c929b2079c2178d1e0dbb3a0cd5c7965226441feeb7576f7b91f47724b22ff"}
Oct 30 21:28:07.639956 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.639910    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-195.us-east-2.compute.internal" event={"ID":"7c1602df5a81ac24aa1efba1059dbbe8","Type":"ContainerStarted","Data":"ef221f135be982cbcdfd9ff62f4c5f763aedc599e41c092987c770d1bebb1a93"}
Oct 30 21:28:07.639956 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.639922    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-195.us-east-2.compute.internal" event={"ID":"7c1602df5a81ac24aa1efba1059dbbe8","Type":"ContainerStarted","Data":"674d20fe97c246ae471b27f055ab576608b0a4cad5fabb0dca5b5ee7a31bfc48"}
Oct 30 21:28:07.640042 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.639966    5032 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-195.us-east-2.compute.internal"
Oct 30 21:28:07.655414 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.655384    5032 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:28:07.655593 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:28:07.655436    5032 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-129-195.us-east-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-195.us-east-2.compute.internal"
Oct 30 21:28:07.658205 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.658187    5032 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:28:07.658311 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:28:07.658222    5032 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-129-195.us-east-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-195.us-east-2.compute.internal"
Oct 30 21:28:07.734385 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.734333    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7efe602e8ba1524abe391aabbb698457-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-195.us-east-2.compute.internal\" (UID: \"7efe602e8ba1524abe391aabbb698457\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-195.us-east-2.compute.internal"
Oct 30 21:28:07.734528 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.734393    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7efe602e8ba1524abe391aabbb698457-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-195.us-east-2.compute.internal\" (UID: \"7efe602e8ba1524abe391aabbb698457\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-195.us-east-2.compute.internal"
Oct 30 21:28:07.734528 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.734431    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7c1602df5a81ac24aa1efba1059dbbe8-config\") pod \"kube-apiserver-proxy-ip-10-0-129-195.us-east-2.compute.internal\" (UID: \"7c1602df5a81ac24aa1efba1059dbbe8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-195.us-east-2.compute.internal"
Oct 30 21:28:07.834917 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.834875    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7efe602e8ba1524abe391aabbb698457-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-195.us-east-2.compute.internal\" (UID: \"7efe602e8ba1524abe391aabbb698457\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-195.us-east-2.compute.internal"
Oct 30 21:28:07.834917 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.834914    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7c1602df5a81ac24aa1efba1059dbbe8-config\") pod \"kube-apiserver-proxy-ip-10-0-129-195.us-east-2.compute.internal\" (UID: \"7c1602df5a81ac24aa1efba1059dbbe8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-195.us-east-2.compute.internal"
Oct 30 21:28:07.835127 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.834934    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7efe602e8ba1524abe391aabbb698457-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-195.us-east-2.compute.internal\" (UID: \"7efe602e8ba1524abe391aabbb698457\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-195.us-east-2.compute.internal"
Oct 30 21:28:07.835127 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.834977    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7efe602e8ba1524abe391aabbb698457-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-195.us-east-2.compute.internal\" (UID: \"7efe602e8ba1524abe391aabbb698457\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-195.us-east-2.compute.internal"
Oct 30 21:28:07.835127 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.834984    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7efe602e8ba1524abe391aabbb698457-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-195.us-east-2.compute.internal\" (UID: \"7efe602e8ba1524abe391aabbb698457\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-195.us-east-2.compute.internal"
Oct 30 21:28:07.835127 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:07.834993    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7c1602df5a81ac24aa1efba1059dbbe8-config\") pod \"kube-apiserver-proxy-ip-10-0-129-195.us-east-2.compute.internal\" (UID: \"7c1602df5a81ac24aa1efba1059dbbe8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-195.us-east-2.compute.internal"
Oct 30 21:28:08.294965 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.294924    5032 apiserver.go:52] "Watching apiserver"
Oct 30 21:28:08.311420 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.311390    5032 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 21:28:08.312144 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.312114    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-vwxrd","openshift-service-ca-operator/service-ca-operator-544d749c8c-9r5sb","kube-system/global-pull-secret-syncer-csbxh","kube-system/kas-connection-checker-685895c7c9-l4dbk","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-195.us-east-2.compute.internal","openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-9qppl","openshift-cluster-node-tuning-operator/tuned-7ntsz","openshift-network-diagnostics/network-check-target-4cvlt","openshift-console-operator/console-operator-7cd56b8dbc-sklxd","openshift-monitoring/cluster-monitoring-operator-884576c57-kdrrd","openshift-service-ca/service-ca-5fffc97bdf-wpm2q","openshift-ingress/router-default-5bbfd9465-x28lh","openshift-dns/node-resolver-97tsr","openshift-insights/insights-runtime-extractor-z2dd4","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qcj76","openshift-image-registry/image-registry-86f87fb58b-5clbv","openshift-network-diagnostics/network-check-source-6f78d59f99-tdmc8","openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-9p8mb","kube-system/kas-connection-checker-685895c7c9-9mt44","openshift-kube-storage-version-migrator/migrator-66df5d496b-l7nbh","openshift-dns/dns-default-sbhch","kube-system/kas-connection-checker-685895c7c9-zvqvt","openshift-ovn-kubernetes/ovnkube-node-gl47s","openshift-multus/multus-d9b8n","kube-system/konnectivity-agent-l8g24","openshift-image-registry/node-ca-jgns8","openshift-insights/insights-operator-7c4986d449-7cgk9","kube-system/kube-apiserver-proxy-ip-10-0-129-195.us-east-2.compute.internal","openshift-network-console/networking-console-plugin-86c478bb88-lwm89","openshift-insights/periodic-gathering-542ht-hjp6z","openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-lbjhv","openshift-image-registry/image-registry-76679b6db7-ltmt6","openshift-kube-storage-version-migrator/migrator-66df5d496b-gqdlr","openshift-ingress-canary/ingress-canary-hzvrj","openshift-network-operator/iptables-alerter-bn9ng","openshift-multus/multus-additional-cni-plugins-l67jk"]
Oct 30 21:28:08.313227 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.313207    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-kdrrd"
Oct 30 21:28:08.313476 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.313454    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-86f87fb58b-5clbv"
Oct 30 21:28:08.313780 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.313753    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-9qppl"
Oct 30 21:28:08.315921 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.315904    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5bbfd9465-x28lh"
Oct 30 21:28:08.317129 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.317102    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\""
Oct 30 21:28:08.317129 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.317117    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 21:28:08.317129 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.317119    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\""
Oct 30 21:28:08.317321 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.317234    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 21:28:08.317321 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.317286    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-lwm89"
Oct 30 21:28:08.321315 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.321294    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Oct 30 21:28:08.323891 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.323872    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vwxrd"
Oct 30 21:28:08.325826 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.325782    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Oct 30 21:28:08.325826 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.325786    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 21:28:08.326021 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.326000    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 21:28:08.326104 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.326081    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Oct 30 21:28:08.326511 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.326491    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76679b6db7-ltmt6"
Oct 30 21:28:08.327032 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.327012    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-sbhch"
Oct 30 21:28:08.327127 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.327068    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Oct 30 21:28:08.327127 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.327078    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Oct 30 21:28:08.327218 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.327145    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Oct 30 21:28:08.327318 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.327297    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Oct 30 21:28:08.327392 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.327342    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Oct 30 21:28:08.327392 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.327376    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-wst66\""
Oct 30 21:28:08.327484 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.327449    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Oct 30 21:28:08.327484 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.327470    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Oct 30 21:28:08.327586 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.327481    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-dockercfg-7m5rm\""
Oct 30 21:28:08.327586 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.327501    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Oct 30 21:28:08.327586 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.327472    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Oct 30 21:28:08.327586 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.327526    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Oct 30 21:28:08.327586 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.327537    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-knzjw\""
Oct 30 21:28:08.327586 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.327544    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-bvjbv\""
Oct 30 21:28:08.327586 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.327553    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\""
Oct 30 21:28:08.327847 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.327603    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Oct 30 21:28:08.327847 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.327678    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Oct 30 21:28:08.327847 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.327731    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Oct 30 21:28:08.328554 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.328536    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Oct 30 21:28:08.328650 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.328609    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Oct 30 21:28:08.328650 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.328610    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Oct 30 21:28:08.328650 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.328646    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 21:28:08.328779 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.328647    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Oct 30 21:28:08.328779 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.328711    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Oct 30 21:28:08.328779 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.328713    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Oct 30 21:28:08.328975 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.328959    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Oct 30 21:28:08.329037 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.328982    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Oct 30 21:28:08.329037 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.328992    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:28:08.329037 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.329033    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Oct 30 21:28:08.329183 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.329060    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:28:08.329183 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.329078    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Oct 30 21:28:08.329183 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.329097    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 21:28:08.330469 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.330453    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-z2dd4"
Oct 30 21:28:08.330469 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.330467    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Oct 30 21:28:08.330609 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.330539    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:28:08.330609 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.330460    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Oct 30 21:28:08.332014 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.331998    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 21:28:08.332071 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.332014    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 21:28:08.332071 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.332041    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 21:28:08.332144 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.332077    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:28:08.332144 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.332101    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:28:08.333672 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.333654    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-2pcx6\""
Oct 30 21:28:08.333775 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.333758    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 21:28:08.333862 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.333837    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 21:28:08.333862 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.333846    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 21:28:08.333961 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.333875    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:28:08.333961 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.333884    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 21:28:08.333961 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.333895    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 21:28:08.335650 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.335629    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 21:28:08.335763 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.335704    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-flwfj\""
Oct 30 21:28:08.336123 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.336100    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Oct 30 21:28:08.338444 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.338422    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 21:28:08.338546 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.338536    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 21:28:08.338639 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.338618    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 21:28:08.338720 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.338704    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-f4rkh\""
Oct 30 21:28:08.338769 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.338715    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 21:28:08.338769 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.338760    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Oct 30 21:28:08.338861 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.338761    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 21:28:08.338861 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.338853    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 21:28:08.338943 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.338889    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Oct 30 21:28:08.338943 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.338918    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 21:28:08.338943 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.338939    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 21:28:08.339066 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.338988    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 21:28:08.339066 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.339014    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 21:28:08.339066 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.339023    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 21:28:08.339066 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.339051    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 21:28:08.339314 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.339063    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 21:28:08.339314 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.339087    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 21:28:08.339314 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.339111    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 21:28:08.339314 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.339153    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 21:28:08.339314 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.338941    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 21:28:08.339314 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.339155    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 21:28:08.339314 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.339222    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Oct 30 21:28:08.339314 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.339277    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 21:28:08.339314 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.339300    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Oct 30 21:28:08.339656 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.339389    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 21:28:08.339656 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.339417    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 21:28:08.339767 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.339749    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:28:08.340167 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.340149    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 21:28:08.340268 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.340250    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-rpdb2\""
Oct 30 21:28:08.340325 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.340277    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-r96r5\""
Oct 30 21:28:08.340325 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.340277    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-cjjj4\""
Oct 30 21:28:08.340325 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.340278    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-99k6q\""
Oct 30 21:28:08.340325 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.340311    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-qcj4q\""
Oct 30 21:28:08.340325 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.340318    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-lz8sp\""
Oct 30 21:28:08.340571 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.340334    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-wxqgz\""
Oct 30 21:28:08.340571 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.340387    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-2wf4q\""
Oct 30 21:28:08.340571 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.340502    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-pqqf8\""
Oct 30 21:28:08.340571 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.340513    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 21:28:08.340571 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.340525    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-z8x7v\""
Oct 30 21:28:08.340571 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.340544    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-ct8zc\""
Oct 30 21:28:08.340571 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.340549    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-d996q\""
Oct 30 21:28:08.340571 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.340564    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-vnct9\""
Oct 30 21:28:08.340571 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.340568    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 21:28:08.340871 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.340580    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-g2c5m\""
Oct 30 21:28:08.340871 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.340646    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Oct 30 21:28:08.340871 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.340670    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-56zgn\""
Oct 30 21:28:08.340871 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.340675    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Oct 30 21:28:08.341036 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.341019    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Oct 30 21:28:08.341036 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.341025    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 21:28:08.403866 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.403837    5032 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-195.us-east-2.compute.internal"
Oct 30 21:28:08.404020 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.403956    5032 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-195.us-east-2.compute.internal"
Oct 30 21:28:08.410912 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.410895    5032 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 21:28:08.415915 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.415897    5032 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:28:08.415966 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:28:08.415938    5032 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-129-195.us-east-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-195.us-east-2.compute.internal"
Oct 30 21:28:08.416430 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.416419    5032 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:28:08.416479 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:28:08.416452    5032 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-129-195.us-east-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-195.us-east-2.compute.internal"
Oct 30 21:28:08.437304 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.437260    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/94ee3757-5440-42e1-8617-2a68744ef3f4-ca-trust-extracted\") pod \"image-registry-86f87fb58b-5clbv\" (UID: \"94ee3757-5440-42e1-8617-2a68744ef3f4\") " pod="openshift-image-registry/image-registry-86f87fb58b-5clbv"
Oct 30 21:28:08.437482 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.437347    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/94ee3757-5440-42e1-8617-2a68744ef3f4-trusted-ca\") pod \"image-registry-86f87fb58b-5clbv\" (UID: \"94ee3757-5440-42e1-8617-2a68744ef3f4\") " pod="openshift-image-registry/image-registry-86f87fb58b-5clbv"
Oct 30 21:28:08.437482 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.437463    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/94ee3757-5440-42e1-8617-2a68744ef3f4-ca-trust-extracted\") pod \"image-registry-86f87fb58b-5clbv\" (UID: \"94ee3757-5440-42e1-8617-2a68744ef3f4\") " pod="openshift-image-registry/image-registry-86f87fb58b-5clbv"
Oct 30 21:28:08.437623 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.437479    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6lpv\" (UniqueName: \"kubernetes.io/projected/94ee3757-5440-42e1-8617-2a68744ef3f4-kube-api-access-t6lpv\") pod \"image-registry-86f87fb58b-5clbv\" (UID: \"94ee3757-5440-42e1-8617-2a68744ef3f4\") " pod="openshift-image-registry/image-registry-86f87fb58b-5clbv"
Oct 30 21:28:08.437738 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.437662    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/67292bdc-7fe1-449d-af75-2baa90c16ea9-trusted-ca\") pod \"console-operator-7cd56b8dbc-sklxd\" (UID: \"67292bdc-7fe1-449d-af75-2baa90c16ea9\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-sklxd"
Oct 30 21:28:08.437738 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.437704    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcgbg\" (UniqueName: \"kubernetes.io/projected/bba1a087-6e04-4db8-ba82-7260a88e2843-kube-api-access-qcgbg\") pod \"network-check-source-6f78d59f99-tdmc8\" (UID: \"bba1a087-6e04-4db8-ba82-7260a88e2843\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-tdmc8"
Oct 30 21:28:08.437852 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.437753    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/84a732a8-045d-4a8b-bb54-859c8d46f08d-iptables-alerter-script\") pod \"iptables-alerter-bn9ng\" (UID: \"84a732a8-045d-4a8b-bb54-859c8d46f08d\") " pod="openshift-network-operator/iptables-alerter-bn9ng"
Oct 30 21:28:08.437852 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.437781    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f6750ff3-d2db-42b7-9c85-7896c9ba67c9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-l67jk\" (UID: \"f6750ff3-d2db-42b7-9c85-7896c9ba67c9\") " pod="openshift-multus/multus-additional-cni-plugins-l67jk"
Oct 30 21:28:08.437852 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.437807    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6ca21b9-20a4-4887-9d08-19d9c491a09e-service-ca-bundle\") pod \"insights-operator-7c4986d449-7cgk9\" (UID: \"d6ca21b9-20a4-4887-9d08-19d9c491a09e\") " pod="openshift-insights/insights-operator-7c4986d449-7cgk9"
Oct 30 21:28:08.437852 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.437830    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cf90e1fc-bac9-4c33-8dc4-dbca18974399-sys-fs\") pod \"aws-ebs-csi-driver-node-qcj76\" (UID: \"cf90e1fc-bac9-4c33-8dc4-dbca18974399\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qcj76"
Oct 30 21:28:08.438085 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.437855    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qf6w\" (UniqueName: \"kubernetes.io/projected/0bbc8570-4947-455a-b7f8-8e9fa8e023b3-kube-api-access-9qf6w\") pod \"insights-runtime-extractor-z2dd4\" (UID: \"0bbc8570-4947-455a-b7f8-8e9fa8e023b3\") " pod="openshift-insights/insights-runtime-extractor-z2dd4"
Oct 30 21:28:08.438085 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.437879    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-host-kubelet\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:28:08.438085 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.437903    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-system-cni-dir\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:28:08.438085 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.437926    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5e93253e-a238-4426-a584-b9de8e931cf7-cni-binary-copy\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:28:08.438085 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.437948    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f171661-33a2-4631-81c7-bc10a5db791f-original-pull-secret\") pod \"global-pull-secret-syncer-csbxh\" (UID: \"5f171661-33a2-4631-81c7-bc10a5db791f\") " pod="kube-system/global-pull-secret-syncer-csbxh"
Oct 30 21:28:08.438085 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.437970    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdf4s\" (UniqueName: \"kubernetes.io/projected/08498890-fb63-4aea-9cf7-a2ade5397df2-kube-api-access-hdf4s\") pod \"dns-default-sbhch\" (UID: \"08498890-fb63-4aea-9cf7-a2ade5397df2\") " pod="openshift-dns/dns-default-sbhch"
Oct 30 21:28:08.438085 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.437996    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5qjc\" (UniqueName: \"kubernetes.io/projected/06a0877a-2fce-405f-b97c-091d88e19543-kube-api-access-l5qjc\") pod \"migrator-66df5d496b-gqdlr\" (UID: \"06a0877a-2fce-405f-b97c-091d88e19543\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-gqdlr"
Oct 30 21:28:08.438085 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.438021    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/94ee3757-5440-42e1-8617-2a68744ef3f4-trusted-ca\") pod \"image-registry-86f87fb58b-5clbv\" (UID: \"94ee3757-5440-42e1-8617-2a68744ef3f4\") " pod="openshift-image-registry/image-registry-86f87fb58b-5clbv"
Oct 30 21:28:08.438085 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.438021    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/84a732a8-045d-4a8b-bb54-859c8d46f08d-host-slash\") pod \"iptables-alerter-bn9ng\" (UID: \"84a732a8-045d-4a8b-bb54-859c8d46f08d\") " pod="openshift-network-operator/iptables-alerter-bn9ng"
Oct 30 21:28:08.438085 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.438072    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-run-openvswitch\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:28:08.438608 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.438104    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-host-run-netns\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:28:08.438608 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.438132    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/90937bc9-0051-407d-9c82-014f0a9c65d4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-kdrrd\" (UID: \"90937bc9-0051-407d-9c82-014f0a9c65d4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-kdrrd"
Oct 30 21:28:08.438608 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.438162    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cf90e1fc-bac9-4c33-8dc4-dbca18974399-etc-selinux\") pod \"aws-ebs-csi-driver-node-qcj76\" (UID: \"cf90e1fc-bac9-4c33-8dc4-dbca18974399\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qcj76"
Oct 30 21:28:08.438608 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.438205    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/48efa9dc-3c20-4f81-86f3-c87ad697772c-host\") pod \"node-ca-jgns8\" (UID: \"48efa9dc-3c20-4f81-86f3-c87ad697772c\") " pod="openshift-image-registry/node-ca-jgns8"
Oct 30 21:28:08.438608 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.438220    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/67292bdc-7fe1-449d-af75-2baa90c16ea9-trusted-ca\") pod \"console-operator-7cd56b8dbc-sklxd\" (UID: \"67292bdc-7fe1-449d-af75-2baa90c16ea9\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-sklxd"
Oct 30 21:28:08.438608 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.438231    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/baa8d0c0-d477-4f30-a0f6-375fc84cc448-tmp-dir\") pod \"node-resolver-97tsr\" (UID: \"baa8d0c0-d477-4f30-a0f6-375fc84cc448\") " pod="openshift-dns/node-resolver-97tsr"
Oct 30 21:28:08.438608 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.438286    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-sys\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:28:08.438608 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.438406    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-var-lib-kubelet\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:28:08.438608 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.438414    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f171661-33a2-4631-81c7-bc10a5db791f-original-pull-secret\") pod \"global-pull-secret-syncer-csbxh\" (UID: \"5f171661-33a2-4631-81c7-bc10a5db791f\") " pod="kube-system/global-pull-secret-syncer-csbxh"
Oct 30 21:28:08.438608 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.438424    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6ca21b9-20a4-4887-9d08-19d9c491a09e-service-ca-bundle\") pod \"insights-operator-7c4986d449-7cgk9\" (UID: \"d6ca21b9-20a4-4887-9d08-19d9c491a09e\") " pod="openshift-insights/insights-operator-7c4986d449-7cgk9"
Oct 30 21:28:08.438608 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.438433    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-tmp\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:28:08.438608 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.438464    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f6750ff3-d2db-42b7-9c85-7896c9ba67c9-cnibin\") pod \"multus-additional-cni-plugins-l67jk\" (UID: \"f6750ff3-d2db-42b7-9c85-7896c9ba67c9\") " pod="openshift-multus/multus-additional-cni-plugins-l67jk"
Oct 30 21:28:08.438608 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.438498    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/11b44bf1-b3b1-4214-b7b4-4e508c618cd1-signing-key\") pod \"service-ca-5fffc97bdf-wpm2q\" (UID: \"11b44bf1-b3b1-4214-b7b4-4e508c618cd1\") " pod="openshift-service-ca/service-ca-5fffc97bdf-wpm2q"
Oct 30 21:28:08.438608 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.438517    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/baa8d0c0-d477-4f30-a0f6-375fc84cc448-tmp-dir\") pod \"node-resolver-97tsr\" (UID: \"baa8d0c0-d477-4f30-a0f6-375fc84cc448\") " pod="openshift-dns/node-resolver-97tsr"
Oct 30 21:28:08.438608 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.438533    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-tmp\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:28:08.438608 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.438587    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-etc-tuned\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:28:08.439340 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.438659    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5tf5\" (UniqueName: \"kubernetes.io/projected/84a732a8-045d-4a8b-bb54-859c8d46f08d-kube-api-access-w5tf5\") pod \"iptables-alerter-bn9ng\" (UID: \"84a732a8-045d-4a8b-bb54-859c8d46f08d\") " pod="openshift-network-operator/iptables-alerter-bn9ng"
Oct 30 21:28:08.439340 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.438700    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-etc-tuned\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:28:08.439340 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.438716    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-var-lib-openvswitch\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:28:08.439340 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.438791    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/11b44bf1-b3b1-4214-b7b4-4e508c618cd1-signing-key\") pod \"service-ca-5fffc97bdf-wpm2q\" (UID: \"11b44bf1-b3b1-4214-b7b4-4e508c618cd1\") " pod="openshift-service-ca/service-ca-5fffc97bdf-wpm2q"
Oct 30 21:28:08.439340 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.438792    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-multus-socket-dir-parent\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:28:08.439340 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.438777    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5e93253e-a238-4426-a584-b9de8e931cf7-cni-binary-copy\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:28:08.439340 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.438824    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/84a732a8-045d-4a8b-bb54-859c8d46f08d-iptables-alerter-script\") pod \"iptables-alerter-bn9ng\" (UID: \"84a732a8-045d-4a8b-bb54-859c8d46f08d\") " pod="openshift-network-operator/iptables-alerter-bn9ng"
Oct 30 21:28:08.439340 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.438833    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pklf4\" (UniqueName: \"kubernetes.io/projected/d6ca21b9-20a4-4887-9d08-19d9c491a09e-kube-api-access-pklf4\") pod \"insights-operator-7c4986d449-7cgk9\" (UID: \"d6ca21b9-20a4-4887-9d08-19d9c491a09e\") " pod="openshift-insights/insights-operator-7c4986d449-7cgk9"
Oct 30 21:28:08.439340 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.438908    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/90937bc9-0051-407d-9c82-014f0a9c65d4-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-kdrrd\" (UID: \"90937bc9-0051-407d-9c82-014f0a9c65d4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-kdrrd"
Oct 30 21:28:08.439340 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.439000    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/94ee3757-5440-42e1-8617-2a68744ef3f4-registry-certificates\") pod \"image-registry-86f87fb58b-5clbv\" (UID: \"94ee3757-5440-42e1-8617-2a68744ef3f4\") " pod="openshift-image-registry/image-registry-86f87fb58b-5clbv"
Oct 30 21:28:08.439340 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.439075    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/039cf439-d12d-450a-80c1-eb3efeb50e8a-registry-tls\") pod \"image-registry-76679b6db7-ltmt6\" (UID: \"039cf439-d12d-450a-80c1-eb3efeb50e8a\") " pod="openshift-image-registry/image-registry-76679b6db7-ltmt6"
Oct 30 21:28:08.439340 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.439139    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/11b44bf1-b3b1-4214-b7b4-4e508c618cd1-signing-cabundle\") pod \"service-ca-5fffc97bdf-wpm2q\" (UID: \"11b44bf1-b3b1-4214-b7b4-4e508c618cd1\") " pod="openshift-service-ca/service-ca-5fffc97bdf-wpm2q"
Oct 30 21:28:08.439340 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.439175    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3175ff47-1038-4139-9c0a-87062bb38dc2-cert\") pod \"ingress-canary-hzvrj\" (UID: \"3175ff47-1038-4139-9c0a-87062bb38dc2\") " pod="openshift-ingress-canary/ingress-canary-hzvrj"
Oct 30 21:28:08.439340 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.439242    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jsdkt\" (UniqueName: \"kubernetes.io/projected/8873f9b9-2ac5-409c-9299-fa024e00189c-kube-api-access-jsdkt\") pod \"service-ca-operator-544d749c8c-9r5sb\" (UID: \"8873f9b9-2ac5-409c-9299-fa024e00189c\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-9r5sb"
Oct 30 21:28:08.439340 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.439301    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-host-run-multus-certs\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:28:08.440174 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.439335    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/90937bc9-0051-407d-9c82-014f0a9c65d4-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-kdrrd\" (UID: \"90937bc9-0051-407d-9c82-014f0a9c65d4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-kdrrd"
Oct 30 21:28:08.440174 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.439381    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08498890-fb63-4aea-9cf7-a2ade5397df2-tmp-dir\") pod \"dns-default-sbhch\" (UID: \"08498890-fb63-4aea-9cf7-a2ade5397df2\") " pod="openshift-dns/dns-default-sbhch"
Oct 30 21:28:08.440174 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.439414    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/11b44bf1-b3b1-4214-b7b4-4e508c618cd1-signing-cabundle\") pod \"service-ca-5fffc97bdf-wpm2q\" (UID: \"11b44bf1-b3b1-4214-b7b4-4e508c618cd1\") " pod="openshift-service-ca/service-ca-5fffc97bdf-wpm2q"
Oct 30 21:28:08.440174 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.439523    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzmzn\" (UniqueName: \"kubernetes.io/projected/e9d2f04b-bb7d-422f-b497-5fa61580f95a-kube-api-access-bzmzn\") pod \"volume-data-source-validator-58c9f5f46d-lbjhv\" (UID: \"e9d2f04b-bb7d-422f-b497-5fa61580f95a\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-lbjhv"
Oct 30 21:28:08.440174 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.439576    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08498890-fb63-4aea-9cf7-a2ade5397df2-tmp-dir\") pod \"dns-default-sbhch\" (UID: \"08498890-fb63-4aea-9cf7-a2ade5397df2\") " pod="openshift-dns/dns-default-sbhch"
Oct 30 21:28:08.440174 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.439625    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0bbc8570-4947-455a-b7f8-8e9fa8e023b3-data-volume\") pod \"insights-runtime-extractor-z2dd4\" (UID: \"0bbc8570-4947-455a-b7f8-8e9fa8e023b3\") " pod="openshift-insights/insights-runtime-extractor-z2dd4"
Oct 30 21:28:08.440174 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.439634    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3175ff47-1038-4139-9c0a-87062bb38dc2-cert\") pod \"ingress-canary-hzvrj\" (UID: \"3175ff47-1038-4139-9c0a-87062bb38dc2\") " pod="openshift-ingress-canary/ingress-canary-hzvrj"
Oct 30 21:28:08.440174 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.439693    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rf7bl\" (UniqueName: \"kubernetes.io/projected/5d90d533-ebca-46db-ab59-733837db23e6-kube-api-access-rf7bl\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:28:08.440174 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.439764    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/039cf439-d12d-450a-80c1-eb3efeb50e8a-installation-pull-secrets\") pod \"image-registry-76679b6db7-ltmt6\" (UID: \"039cf439-d12d-450a-80c1-eb3efeb50e8a\") " pod="openshift-image-registry/image-registry-76679b6db7-ltmt6"
Oct 30 21:28:08.440174 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.439802    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pls28\" (UniqueName: \"kubernetes.io/projected/3c835b41-25cb-4f0c-8e7c-4ac9e2fee318-kube-api-access-pls28\") pod \"kas-connection-checker-685895c7c9-9mt44\" (UID: \"3c835b41-25cb-4f0c-8e7c-4ac9e2fee318\") " pod="kube-system/kas-connection-checker-685895c7c9-9mt44"
Oct 30 21:28:08.440174 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.439865    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/94ee3757-5440-42e1-8617-2a68744ef3f4-registry-certificates\") pod \"image-registry-86f87fb58b-5clbv\" (UID: \"94ee3757-5440-42e1-8617-2a68744ef3f4\") " pod="openshift-image-registry/image-registry-86f87fb58b-5clbv"
Oct 30 21:28:08.440174 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.439981    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0bbc8570-4947-455a-b7f8-8e9fa8e023b3-data-volume\") pod \"insights-runtime-extractor-z2dd4\" (UID: \"0bbc8570-4947-455a-b7f8-8e9fa8e023b3\") " pod="openshift-insights/insights-runtime-extractor-z2dd4"
Oct 30 21:28:08.440174 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.440019    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/039cf439-d12d-450a-80c1-eb3efeb50e8a-installation-pull-secrets\") pod \"image-registry-76679b6db7-ltmt6\" (UID: \"039cf439-d12d-450a-80c1-eb3efeb50e8a\") " pod="openshift-image-registry/image-registry-76679b6db7-ltmt6"
Oct 30 21:28:08.440174 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.440029    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-etc-sysconfig\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:28:08.440174 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.440102    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-host-slash\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:28:08.440648 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.440208    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/039cf439-d12d-450a-80c1-eb3efeb50e8a-trusted-ca\") pod \"image-registry-76679b6db7-ltmt6\" (UID: \"039cf439-d12d-450a-80c1-eb3efeb50e8a\") " pod="openshift-image-registry/image-registry-76679b6db7-ltmt6"
Oct 30 21:28:08.440648 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.440231    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ee6de179-7e35-46a5-86e1-c7c31428c149-konnectivity-ca\") pod \"konnectivity-agent-l8g24\" (UID: \"ee6de179-7e35-46a5-86e1-c7c31428c149\") " pod="kube-system/konnectivity-agent-l8g24"
Oct 30 21:28:08.440648 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.440248    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-host-cni-bin\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:28:08.440648 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.440284    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-host-var-lib-kubelet\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:28:08.440648 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.440445    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/039cf439-d12d-450a-80c1-eb3efeb50e8a-bound-sa-token\") pod \"image-registry-76679b6db7-ltmt6\" (UID: \"039cf439-d12d-450a-80c1-eb3efeb50e8a\") " pod="openshift-image-registry/image-registry-76679b6db7-ltmt6"
Oct 30 21:28:08.440648 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.440502    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd34a723-6c9e-48f1-ba7d-924d542b2bc7-service-ca-bundle\") pod \"periodic-gathering-542ht-hjp6z\" (UID: \"fd34a723-6c9e-48f1-ba7d-924d542b2bc7\") " pod="openshift-insights/periodic-gathering-542ht-hjp6z"
Oct 30 21:28:08.440648 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.440572    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ee6de179-7e35-46a5-86e1-c7c31428c149-konnectivity-ca\") pod \"konnectivity-agent-l8g24\" (UID: \"ee6de179-7e35-46a5-86e1-c7c31428c149\") " pod="kube-system/konnectivity-agent-l8g24"
Oct 30 21:28:08.440648 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.440604    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/039cf439-d12d-450a-80c1-eb3efeb50e8a-trusted-ca\") pod \"image-registry-76679b6db7-ltmt6\" (UID: \"039cf439-d12d-450a-80c1-eb3efeb50e8a\") " pod="openshift-image-registry/image-registry-76679b6db7-ltmt6"
Oct 30 21:28:08.440648 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.440613    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-host-run-k8s-cni-cncf-io\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:28:08.440983 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.440656    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/9b4f9524-db75-4206-aa76-04400d95706f-default-certificate\") pod \"router-default-5bbfd9465-x28lh\" (UID: \"9b4f9524-db75-4206-aa76-04400d95706f\") " pod="openshift-ingress/router-default-5bbfd9465-x28lh"
Oct 30 21:28:08.440983 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.440702    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cf90e1fc-bac9-4c33-8dc4-dbca18974399-socket-dir\") pod \"aws-ebs-csi-driver-node-qcj76\" (UID: \"cf90e1fc-bac9-4c33-8dc4-dbca18974399\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qcj76"
Oct 30 21:28:08.440983 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.440725    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8km8\" (UniqueName: \"kubernetes.io/projected/67292bdc-7fe1-449d-af75-2baa90c16ea9-kube-api-access-m8km8\") pod \"console-operator-7cd56b8dbc-sklxd\" (UID: \"67292bdc-7fe1-449d-af75-2baa90c16ea9\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-sklxd"
Oct 30 21:28:08.440983 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.440703    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd34a723-6c9e-48f1-ba7d-924d542b2bc7-service-ca-bundle\") pod \"periodic-gathering-542ht-hjp6z\" (UID: \"fd34a723-6c9e-48f1-ba7d-924d542b2bc7\") " pod="openshift-insights/periodic-gathering-542ht-hjp6z"
Oct 30 21:28:08.440983 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.440744    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffq8c\" (UniqueName: \"kubernetes.io/projected/fb314739-c678-4da8-9d54-147dc0612ee2-kube-api-access-ffq8c\") pod \"kas-connection-checker-685895c7c9-l4dbk\" (UID: \"fb314739-c678-4da8-9d54-147dc0612ee2\") " pod="kube-system/kas-connection-checker-685895c7c9-l4dbk"
Oct 30 21:28:08.440983 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.440780    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhvst\" (UniqueName: \"kubernetes.io/projected/f6750ff3-d2db-42b7-9c85-7896c9ba67c9-kube-api-access-qhvst\") pod \"multus-additional-cni-plugins-l67jk\" (UID: \"f6750ff3-d2db-42b7-9c85-7896c9ba67c9\") " pod="openshift-multus/multus-additional-cni-plugins-l67jk"
Oct 30 21:28:08.440983 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.440805    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-log-socket\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:28:08.440983 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.440827    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-host-run-ovn-kubernetes\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:28:08.440983 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.440839    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/9b4f9524-db75-4206-aa76-04400d95706f-default-certificate\") pod \"router-default-5bbfd9465-x28lh\" (UID: \"9b4f9524-db75-4206-aa76-04400d95706f\") " pod="openshift-ingress/router-default-5bbfd9465-x28lh"
Oct 30 21:28:08.440983 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.440854    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/d6ca21b9-20a4-4887-9d08-19d9c491a09e-snapshots\") pod \"insights-operator-7c4986d449-7cgk9\" (UID: \"d6ca21b9-20a4-4887-9d08-19d9c491a09e\") " pod="openshift-insights/insights-operator-7c4986d449-7cgk9"
Oct 30 21:28:08.440983 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.440882    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/753d4639-09e6-4850-9f4d-ed39b0406439-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-lwm89\" (UID: \"753d4639-09e6-4850-9f4d-ed39b0406439\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-lwm89"
Oct 30 21:28:08.440983 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.440963    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/d6ca21b9-20a4-4887-9d08-19d9c491a09e-snapshots\") pod \"insights-operator-7c4986d449-7cgk9\" (UID: \"d6ca21b9-20a4-4887-9d08-19d9c491a09e\") " pod="openshift-insights/insights-operator-7c4986d449-7cgk9"
Oct 30 21:28:08.440983 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.440975    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/039cf439-d12d-450a-80c1-eb3efeb50e8a-ca-trust-extracted\") pod \"image-registry-76679b6db7-ltmt6\" (UID: \"039cf439-d12d-450a-80c1-eb3efeb50e8a\") " pod="openshift-image-registry/image-registry-76679b6db7-ltmt6"
Oct 30 21:28:08.441636 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.441005    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8873f9b9-2ac5-409c-9299-fa024e00189c-config\") pod \"service-ca-operator-544d749c8c-9r5sb\" (UID: \"8873f9b9-2ac5-409c-9299-fa024e00189c\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-9r5sb"
Oct 30 21:28:08.441636 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.441021    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5d90d533-ebca-46db-ab59-733837db23e6-ovn-node-metrics-cert\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:28:08.441636 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.441054    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/67292bdc-7fe1-449d-af75-2baa90c16ea9-serving-cert\") pod \"console-operator-7cd56b8dbc-sklxd\" (UID: \"67292bdc-7fe1-449d-af75-2baa90c16ea9\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-sklxd"
Oct 30 21:28:08.441636 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.441073    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrrdm\" (UniqueName: \"kubernetes.io/projected/039cf439-d12d-450a-80c1-eb3efeb50e8a-kube-api-access-zrrdm\") pod \"image-registry-76679b6db7-ltmt6\" (UID: \"039cf439-d12d-450a-80c1-eb3efeb50e8a\") " pod="openshift-image-registry/image-registry-76679b6db7-ltmt6"
Oct 30 21:28:08.441636 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.441083    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/039cf439-d12d-450a-80c1-eb3efeb50e8a-ca-trust-extracted\") pod \"image-registry-76679b6db7-ltmt6\" (UID: \"039cf439-d12d-450a-80c1-eb3efeb50e8a\") " pod="openshift-image-registry/image-registry-76679b6db7-ltmt6"
Oct 30 21:28:08.441636 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.441095    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-hostroot\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:28:08.441636 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.441131    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/baa8d0c0-d477-4f30-a0f6-375fc84cc448-hosts-file\") pod \"node-resolver-97tsr\" (UID: \"baa8d0c0-d477-4f30-a0f6-375fc84cc448\") " pod="openshift-dns/node-resolver-97tsr"
Oct 30 21:28:08.441636 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.441167    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ee6de179-7e35-46a5-86e1-c7c31428c149-agent-certs\") pod \"konnectivity-agent-l8g24\" (UID: \"ee6de179-7e35-46a5-86e1-c7c31428c149\") " pod="kube-system/konnectivity-agent-l8g24"
Oct 30 21:28:08.441636 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.441195    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spppx\" (UniqueName: \"kubernetes.io/projected/8bb24a0d-3af0-4a05-bba4-4f9630b04bf3-kube-api-access-spppx\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-9p8mb\" (UID: \"8bb24a0d-3af0-4a05-bba4-4f9630b04bf3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-9p8mb"
Oct 30 21:28:08.441636 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.441207    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8873f9b9-2ac5-409c-9299-fa024e00189c-config\") pod \"service-ca-operator-544d749c8c-9r5sb\" (UID: \"8873f9b9-2ac5-409c-9299-fa024e00189c\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-9r5sb"
Oct 30 21:28:08.441636 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.441225    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0bbc8570-4947-455a-b7f8-8e9fa8e023b3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z2dd4\" (UID: \"0bbc8570-4947-455a-b7f8-8e9fa8e023b3\") " pod="openshift-insights/insights-runtime-extractor-z2dd4"
Oct 30 21:28:08.441636 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.441252    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-run-ovn\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:28:08.441636 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.441276    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-host-var-lib-cni-bin\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:28:08.441636 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.441308    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cf90e1fc-bac9-4c33-8dc4-dbca18974399-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qcj76\" (UID: \"cf90e1fc-bac9-4c33-8dc4-dbca18974399\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qcj76"
Oct 30 21:28:08.441636 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.441428    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/67292bdc-7fe1-449d-af75-2baa90c16ea9-serving-cert\") pod \"console-operator-7cd56b8dbc-sklxd\" (UID: \"67292bdc-7fe1-449d-af75-2baa90c16ea9\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-sklxd"
Oct 30 21:28:08.441636 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.441475    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ee6de179-7e35-46a5-86e1-c7c31428c149-agent-certs\") pod \"konnectivity-agent-l8g24\" (UID: \"ee6de179-7e35-46a5-86e1-c7c31428c149\") " pod="kube-system/konnectivity-agent-l8g24"
Oct 30 21:28:08.441636 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.441479    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-cnibin\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:28:08.441636 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.441536    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6ca21b9-20a4-4887-9d08-19d9c491a09e-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-7cgk9\" (UID: \"d6ca21b9-20a4-4887-9d08-19d9c491a09e\") " pod="openshift-insights/insights-operator-7c4986d449-7cgk9"
Oct 30 21:28:08.441636 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.441570    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/94ee3757-5440-42e1-8617-2a68744ef3f4-bound-sa-token\") pod \"image-registry-86f87fb58b-5clbv\" (UID: \"94ee3757-5440-42e1-8617-2a68744ef3f4\") " pod="openshift-image-registry/image-registry-86f87fb58b-5clbv"
Oct 30 21:28:08.441636 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.441603    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5d90d533-ebca-46db-ab59-733837db23e6-ovn-node-metrics-cert\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:28:08.441636 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.441610    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8f74s\" (UniqueName: \"kubernetes.io/projected/11b44bf1-b3b1-4214-b7b4-4e508c618cd1-kube-api-access-8f74s\") pod \"service-ca-5fffc97bdf-wpm2q\" (UID: \"11b44bf1-b3b1-4214-b7b4-4e508c618cd1\") " pod="openshift-service-ca/service-ca-5fffc97bdf-wpm2q"
Oct 30 21:28:08.441636 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.441650    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2trdz\" (UniqueName: \"kubernetes.io/projected/5e93253e-a238-4426-a584-b9de8e931cf7-kube-api-access-2trdz\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:28:08.442758 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.441678    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd34a723-6c9e-48f1-ba7d-924d542b2bc7-trusted-ca-bundle\") pod \"periodic-gathering-542ht-hjp6z\" (UID: \"fd34a723-6c9e-48f1-ba7d-924d542b2bc7\") " pod="openshift-insights/periodic-gathering-542ht-hjp6z"
Oct 30 21:28:08.442758 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.441838    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfphc\" (UniqueName: \"kubernetes.io/projected/3175ff47-1038-4139-9c0a-87062bb38dc2-kube-api-access-zfphc\") pod \"ingress-canary-hzvrj\" (UID: \"3175ff47-1038-4139-9c0a-87062bb38dc2\") " pod="openshift-ingress-canary/ingress-canary-hzvrj"
Oct 30 21:28:08.442758 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.441877    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-etc-sysctl-d\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:28:08.442758 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.441900    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-etc-sysctl-conf\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:28:08.442758 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.441948    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6ca21b9-20a4-4887-9d08-19d9c491a09e-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-7cgk9\" (UID: \"d6ca21b9-20a4-4887-9d08-19d9c491a09e\") " pod="openshift-insights/insights-operator-7c4986d449-7cgk9"
Oct 30 21:28:08.442758 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.441982    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd34a723-6c9e-48f1-ba7d-924d542b2bc7-trusted-ca-bundle\") pod \"periodic-gathering-542ht-hjp6z\" (UID: \"fd34a723-6c9e-48f1-ba7d-924d542b2bc7\") " pod="openshift-insights/periodic-gathering-542ht-hjp6z"
Oct 30 21:28:08.442758 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.441991    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-etc-systemd\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:28:08.442758 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.442039    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f6750ff3-d2db-42b7-9c85-7896c9ba67c9-cni-binary-copy\") pod \"multus-additional-cni-plugins-l67jk\" (UID: \"f6750ff3-d2db-42b7-9c85-7896c9ba67c9\") " pod="openshift-multus/multus-additional-cni-plugins-l67jk"
Oct 30 21:28:08.442758 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.442068    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-systemd-units\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:28:08.442758 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.442092    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-etc-openvswitch\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:28:08.442758 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.442126    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/94ee3757-5440-42e1-8617-2a68744ef3f4-installation-pull-secrets\") pod \"image-registry-86f87fb58b-5clbv\" (UID: \"94ee3757-5440-42e1-8617-2a68744ef3f4\") " pod="openshift-image-registry/image-registry-86f87fb58b-5clbv"
Oct 30 21:28:08.442758 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.442151    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5e93253e-a238-4426-a584-b9de8e931cf7-multus-daemon-config\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:28:08.442758 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.442176    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cf90e1fc-bac9-4c33-8dc4-dbca18974399-device-dir\") pod \"aws-ebs-csi-driver-node-qcj76\" (UID: \"cf90e1fc-bac9-4c33-8dc4-dbca18974399\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qcj76"
Oct 30 21:28:08.442758 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.442203    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8bb24a0d-3af0-4a05-bba4-4f9630b04bf3-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-9p8mb\" (UID: \"8bb24a0d-3af0-4a05-bba4-4f9630b04bf3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-9p8mb"
Oct 30 21:28:08.442758 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.442227    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-run-systemd\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:28:08.442758 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.442236    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f6750ff3-d2db-42b7-9c85-7896c9ba67c9-cni-binary-copy\") pod \"multus-additional-cni-plugins-l67jk\" (UID: \"f6750ff3-d2db-42b7-9c85-7896c9ba67c9\") " pod="openshift-multus/multus-additional-cni-plugins-l67jk"
Oct 30 21:28:08.442758 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.442253    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-multus-conf-dir\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:28:08.442758 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.442280    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/94ee3757-5440-42e1-8617-2a68744ef3f4-image-registry-private-configuration\") pod \"image-registry-86f87fb58b-5clbv\" (UID: \"94ee3757-5440-42e1-8617-2a68744ef3f4\") " pod="openshift-image-registry/image-registry-86f87fb58b-5clbv"
Oct 30 21:28:08.442758 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.442309    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/039cf439-d12d-450a-80c1-eb3efeb50e8a-registry-certificates\") pod \"image-registry-76679b6db7-ltmt6\" (UID: \"039cf439-d12d-450a-80c1-eb3efeb50e8a\") " pod="openshift-image-registry/image-registry-76679b6db7-ltmt6"
Oct 30 21:28:08.442758 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.442355    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5e93253e-a238-4426-a584-b9de8e931cf7-multus-daemon-config\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:28:08.442758 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.442378    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/08498890-fb63-4aea-9cf7-a2ade5397df2-config-volume\") pod \"dns-default-sbhch\" (UID: \"08498890-fb63-4aea-9cf7-a2ade5397df2\") " pod="openshift-dns/dns-default-sbhch"
Oct 30 21:28:08.442758 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.442452    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-run\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:28:08.442758 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.442537    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ms8rv\" (UniqueName: \"kubernetes.io/projected/3913f7d4-86cb-4e86-af5a-debd8b859252-kube-api-access-ms8rv\") pod \"network-check-target-4cvlt\" (UID: \"3913f7d4-86cb-4e86-af5a-debd8b859252\") " pod="openshift-network-diagnostics/network-check-target-4cvlt"
Oct 30 21:28:08.442758 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.442597    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/94ee3757-5440-42e1-8617-2a68744ef3f4-image-registry-private-configuration\") pod \"image-registry-86f87fb58b-5clbv\" (UID: \"94ee3757-5440-42e1-8617-2a68744ef3f4\") " pod="openshift-image-registry/image-registry-86f87fb58b-5clbv"
Oct 30 21:28:08.442758 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.442612    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/08498890-fb63-4aea-9cf7-a2ade5397df2-config-volume\") pod \"dns-default-sbhch\" (UID: \"08498890-fb63-4aea-9cf7-a2ade5397df2\") " pod="openshift-dns/dns-default-sbhch"
Oct 30 21:28:08.442758 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.442619    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8bb24a0d-3af0-4a05-bba4-4f9630b04bf3-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-9p8mb\" (UID: \"8bb24a0d-3af0-4a05-bba4-4f9630b04bf3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-9p8mb"
Oct 30 21:28:08.442758 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.442582    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfrc4\" (UniqueName: \"kubernetes.io/projected/9b4f9524-db75-4206-aa76-04400d95706f-kube-api-access-wfrc4\") pod \"router-default-5bbfd9465-x28lh\" (UID: \"9b4f9524-db75-4206-aa76-04400d95706f\") " pod="openshift-ingress/router-default-5bbfd9465-x28lh"
Oct 30 21:28:08.442758 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.442665    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/67292bdc-7fe1-449d-af75-2baa90c16ea9-config\") pod \"console-operator-7cd56b8dbc-sklxd\" (UID: \"67292bdc-7fe1-449d-af75-2baa90c16ea9\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-sklxd"
Oct 30 21:28:08.442758 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.442713    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/94ee3757-5440-42e1-8617-2a68744ef3f4-installation-pull-secrets\") pod \"image-registry-86f87fb58b-5clbv\" (UID: \"94ee3757-5440-42e1-8617-2a68744ef3f4\") " pod="openshift-image-registry/image-registry-86f87fb58b-5clbv"
Oct 30 21:28:08.442758 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.442699    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/039cf439-d12d-450a-80c1-eb3efeb50e8a-image-registry-private-configuration\") pod \"image-registry-76679b6db7-ltmt6\" (UID: \"039cf439-d12d-450a-80c1-eb3efeb50e8a\") " pod="openshift-image-registry/image-registry-76679b6db7-ltmt6"
Oct 30 21:28:08.444184 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.442838    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-etc-modprobe-d\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:28:08.444184 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.442880    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f6750ff3-d2db-42b7-9c85-7896c9ba67c9-system-cni-dir\") pod \"multus-additional-cni-plugins-l67jk\" (UID: \"f6750ff3-d2db-42b7-9c85-7896c9ba67c9\") " pod="openshift-multus/multus-additional-cni-plugins-l67jk"
Oct 30 21:28:08.444184 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.442892    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/039cf439-d12d-450a-80c1-eb3efeb50e8a-image-registry-private-configuration\") pod \"image-registry-76679b6db7-ltmt6\" (UID: \"039cf439-d12d-450a-80c1-eb3efeb50e8a\") " pod="openshift-image-registry/image-registry-76679b6db7-ltmt6"
Oct 30 21:28:08.444184 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.442892    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/67292bdc-7fe1-449d-af75-2baa90c16ea9-config\") pod \"console-operator-7cd56b8dbc-sklxd\" (UID: \"67292bdc-7fe1-449d-af75-2baa90c16ea9\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-sklxd"
Oct 30 21:28:08.444184 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.442923    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f6750ff3-d2db-42b7-9c85-7896c9ba67c9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-l67jk\" (UID: \"f6750ff3-d2db-42b7-9c85-7896c9ba67c9\") " pod="openshift-multus/multus-additional-cni-plugins-l67jk"
Oct 30 21:28:08.444184 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.442950    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-multus-cni-dir\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:28:08.444184 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.442970    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d6ca21b9-20a4-4887-9d08-19d9c491a09e-tmp\") pod \"insights-operator-7c4986d449-7cgk9\" (UID: \"d6ca21b9-20a4-4887-9d08-19d9c491a09e\") " pod="openshift-insights/insights-operator-7c4986d449-7cgk9"
Oct 30 21:28:08.444184 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.442987    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8bb24a0d-3af0-4a05-bba4-4f9630b04bf3-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-9p8mb\" (UID: \"8bb24a0d-3af0-4a05-bba4-4f9630b04bf3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-9p8mb"
Oct 30 21:28:08.444184 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.443006    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqwd9\" (UniqueName: \"kubernetes.io/projected/9f54b2c6-b1f9-42da-abdc-275fc8827b71-kube-api-access-jqwd9\") pod \"cluster-samples-operator-6ddb6c8b94-9qppl\" (UID: \"9f54b2c6-b1f9-42da-abdc-275fc8827b71\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-9qppl"
Oct 30 21:28:08.444184 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.443024    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhrs4\" (UniqueName: \"kubernetes.io/projected/faca011a-4ff1-4de0-9af1-ed9b0f253aeb-kube-api-access-dhrs4\") pod \"migrator-66df5d496b-l7nbh\" (UID: \"faca011a-4ff1-4de0-9af1-ed9b0f253aeb\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-l7nbh"
Oct 30 21:28:08.444184 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.443039    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5d90d533-ebca-46db-ab59-733837db23e6-env-overrides\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:28:08.444184 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.443056    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-host-var-lib-cni-multus\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:28:08.444184 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.443063    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d6ca21b9-20a4-4887-9d08-19d9c491a09e-tmp\") pod \"insights-operator-7c4986d449-7cgk9\" (UID: \"d6ca21b9-20a4-4887-9d08-19d9c491a09e\") " pod="openshift-insights/insights-operator-7c4986d449-7cgk9"
Oct 30 21:28:08.444184 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.443072    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5f171661-33a2-4631-81c7-bc10a5db791f-dbus\") pod \"global-pull-secret-syncer-csbxh\" (UID: \"5f171661-33a2-4631-81c7-bc10a5db791f\") " pod="kube-system/global-pull-secret-syncer-csbxh"
Oct 30 21:28:08.444184 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.443089    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/94ee3757-5440-42e1-8617-2a68744ef3f4-registry-tls\") pod \"image-registry-86f87fb58b-5clbv\" (UID: \"94ee3757-5440-42e1-8617-2a68744ef3f4\") " pod="openshift-image-registry/image-registry-86f87fb58b-5clbv"
Oct 30 21:28:08.444184 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.443106    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f6750ff3-d2db-42b7-9c85-7896c9ba67c9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-l67jk\" (UID: \"f6750ff3-d2db-42b7-9c85-7896c9ba67c9\") " pod="openshift-multus/multus-additional-cni-plugins-l67jk"
Oct 30 21:28:08.444184 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.443152    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96bzs\" (UniqueName: \"kubernetes.io/projected/352b5b4c-c640-4300-abf3-aa70b08cbb4b-kube-api-access-96bzs\") pod \"network-metrics-daemon-vwxrd\" (UID: \"352b5b4c-c640-4300-abf3-aa70b08cbb4b\") " pod="openshift-multus/network-metrics-daemon-vwxrd"
Oct 30 21:28:08.444184 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.443174    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08498890-fb63-4aea-9cf7-a2ade5397df2-metrics-tls\") pod \"dns-default-sbhch\" (UID: \"08498890-fb63-4aea-9cf7-a2ade5397df2\") " pod="openshift-dns/dns-default-sbhch"
Oct 30 21:28:08.444184 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.443181    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8bb24a0d-3af0-4a05-bba4-4f9630b04bf3-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-9p8mb\" (UID: \"8bb24a0d-3af0-4a05-bba4-4f9630b04bf3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-9p8mb"
Oct 30 21:28:08.444184 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.443181    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/039cf439-d12d-450a-80c1-eb3efeb50e8a-registry-certificates\") pod \"image-registry-76679b6db7-ltmt6\" (UID: \"039cf439-d12d-450a-80c1-eb3efeb50e8a\") " pod="openshift-image-registry/image-registry-76679b6db7-ltmt6"
Oct 30 21:28:08.444184 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.443197    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-host-cni-netd\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:28:08.444184 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.443346    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5d90d533-ebca-46db-ab59-733837db23e6-env-overrides\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:28:08.444184 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.443436    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-host-run-netns\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:28:08.444184 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.443458    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/753d4639-09e6-4850-9f4d-ed39b0406439-nginx-conf\") pod \"networking-console-plugin-86c478bb88-lwm89\" (UID: \"753d4639-09e6-4850-9f4d-ed39b0406439\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-lwm89"
Oct 30 21:28:08.444184 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.443481    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5f171661-33a2-4631-81c7-bc10a5db791f-kubelet-config\") pod \"global-pull-secret-syncer-csbxh\" (UID: \"5f171661-33a2-4631-81c7-bc10a5db791f\") " pod="kube-system/global-pull-secret-syncer-csbxh"
Oct 30 21:28:08.444184 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.443529    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtmzr\" (UniqueName: \"kubernetes.io/projected/48efa9dc-3c20-4f81-86f3-c87ad697772c-kube-api-access-vtmzr\") pod \"node-ca-jgns8\" (UID: \"48efa9dc-3c20-4f81-86f3-c87ad697772c\") " pod="openshift-image-registry/node-ca-jgns8"
Oct 30 21:28:08.444184 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.443562    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/352b5b4c-c640-4300-abf3-aa70b08cbb4b-metrics-certs\") pod \"network-metrics-daemon-vwxrd\" (UID: \"352b5b4c-c640-4300-abf3-aa70b08cbb4b\") " pod="openshift-multus/network-metrics-daemon-vwxrd"
Oct 30 21:28:08.444184 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.443618    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/753d4639-09e6-4850-9f4d-ed39b0406439-nginx-conf\") pod \"networking-console-plugin-86c478bb88-lwm89\" (UID: \"753d4639-09e6-4850-9f4d-ed39b0406439\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-lwm89"
Oct 30 21:28:08.444184 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.443795    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b4f9524-db75-4206-aa76-04400d95706f-service-ca-bundle\") pod \"router-default-5bbfd9465-x28lh\" (UID: \"9b4f9524-db75-4206-aa76-04400d95706f\") " pod="openshift-ingress/router-default-5bbfd9465-x28lh"
Oct 30 21:28:08.444184 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.443832    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/fd34a723-6c9e-48f1-ba7d-924d542b2bc7-archives-path\") pod \"periodic-gathering-542ht-hjp6z\" (UID: \"fd34a723-6c9e-48f1-ba7d-924d542b2bc7\") " pod="openshift-insights/periodic-gathering-542ht-hjp6z"
Oct 30 21:28:08.444184 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.443876    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klwsc\" (UniqueName: \"kubernetes.io/projected/baa8d0c0-d477-4f30-a0f6-375fc84cc448-kube-api-access-klwsc\") pod \"node-resolver-97tsr\" (UID: \"baa8d0c0-d477-4f30-a0f6-375fc84cc448\") " pod="openshift-dns/node-resolver-97tsr"
Oct 30 21:28:08.444184 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.443898    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-etc-kubernetes\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:28:08.444184 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.443916    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-host\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:28:08.444184 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.443931    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0bbc8570-4947-455a-b7f8-8e9fa8e023b3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-z2dd4\" (UID: \"0bbc8570-4947-455a-b7f8-8e9fa8e023b3\") " pod="openshift-insights/insights-runtime-extractor-z2dd4"
Oct 30 21:28:08.444184 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.443985    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b4f9524-db75-4206-aa76-04400d95706f-metrics-certs\") pod \"router-default-5bbfd9465-x28lh\" (UID: \"9b4f9524-db75-4206-aa76-04400d95706f\") " pod="openshift-ingress/router-default-5bbfd9465-x28lh"
Oct 30 21:28:08.444184 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.444000    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/fd34a723-6c9e-48f1-ba7d-924d542b2bc7-archives-path\") pod \"periodic-gathering-542ht-hjp6z\" (UID: \"fd34a723-6c9e-48f1-ba7d-924d542b2bc7\") " pod="openshift-insights/periodic-gathering-542ht-hjp6z"
Oct 30 21:28:08.444184 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.444014    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/48efa9dc-3c20-4f81-86f3-c87ad697772c-serviceca\") pod \"node-ca-jgns8\" (UID: \"48efa9dc-3c20-4f81-86f3-c87ad697772c\") " pod="openshift-image-registry/node-ca-jgns8"
Oct 30 21:28:08.444184 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.444032    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnx5t\" (UniqueName: \"kubernetes.io/projected/fd34a723-6c9e-48f1-ba7d-924d542b2bc7-kube-api-access-fnx5t\") pod \"periodic-gathering-542ht-hjp6z\" (UID: \"fd34a723-6c9e-48f1-ba7d-924d542b2bc7\") " pod="openshift-insights/periodic-gathering-542ht-hjp6z"
Oct 30 21:28:08.444184 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.444064    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/9f54b2c6-b1f9-42da-abdc-275fc8827b71-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-9qppl\" (UID: \"9f54b2c6-b1f9-42da-abdc-275fc8827b71\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-9qppl"
Oct 30 21:28:08.444184 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.444113    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0bbc8570-4947-455a-b7f8-8e9fa8e023b3-crio-socket\") pod \"insights-runtime-extractor-z2dd4\" (UID: \"0bbc8570-4947-455a-b7f8-8e9fa8e023b3\") " pod="openshift-insights/insights-runtime-extractor-z2dd4"
Oct 30 21:28:08.444184 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.444151    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-node-log\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:28:08.445614 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.444216    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5d90d533-ebca-46db-ab59-733837db23e6-ovnkube-script-lib\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:28:08.445614 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.444244    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/9f54b2c6-b1f9-42da-abdc-275fc8827b71-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-9qppl\" (UID: \"9f54b2c6-b1f9-42da-abdc-275fc8827b71\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-9qppl"
Oct 30 21:28:08.445614 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.444257    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/9b4f9524-db75-4206-aa76-04400d95706f-stats-auth\") pod \"router-default-5bbfd9465-x28lh\" (UID: \"9b4f9524-db75-4206-aa76-04400d95706f\") " pod="openshift-ingress/router-default-5bbfd9465-x28lh"
Oct 30 21:28:08.445614 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.444293    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wp72\" (UniqueName: \"kubernetes.io/projected/0d83c70b-fab8-4872-969f-11f74c0ddea0-kube-api-access-9wp72\") pod \"kas-connection-checker-685895c7c9-zvqvt\" (UID: \"0d83c70b-fab8-4872-969f-11f74c0ddea0\") " pod="kube-system/kas-connection-checker-685895c7c9-zvqvt"
Oct 30 21:28:08.445614 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.444320    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5d90d533-ebca-46db-ab59-733837db23e6-ovnkube-config\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:28:08.445614 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.444355    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-etc-kubernetes\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:28:08.445614 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.444404    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56qdr\" (UniqueName: \"kubernetes.io/projected/90937bc9-0051-407d-9c82-014f0a9c65d4-kube-api-access-56qdr\") pod \"cluster-monitoring-operator-884576c57-kdrrd\" (UID: \"90937bc9-0051-407d-9c82-014f0a9c65d4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-kdrrd"
Oct 30 21:28:08.445614 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.444429    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cf90e1fc-bac9-4c33-8dc4-dbca18974399-registration-dir\") pod \"aws-ebs-csi-driver-node-qcj76\" (UID: \"cf90e1fc-bac9-4c33-8dc4-dbca18974399\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qcj76"
Oct 30 21:28:08.445614 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.444429    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5d90d533-ebca-46db-ab59-733837db23e6-ovnkube-script-lib\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:28:08.445614 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.444472    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8873f9b9-2ac5-409c-9299-fa024e00189c-serving-cert\") pod \"service-ca-operator-544d749c8c-9r5sb\" (UID: \"8873f9b9-2ac5-409c-9299-fa024e00189c\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-9r5sb"
Oct 30 21:28:08.445614 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.444505    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d6ca21b9-20a4-4887-9d08-19d9c491a09e-serving-cert\") pod \"insights-operator-7c4986d449-7cgk9\" (UID: \"d6ca21b9-20a4-4887-9d08-19d9c491a09e\") " pod="openshift-insights/insights-operator-7c4986d449-7cgk9"
Oct 30 21:28:08.445614 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.444525    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5d90d533-ebca-46db-ab59-733837db23e6-ovnkube-config\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:28:08.445614 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.444533    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-lib-modules\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:28:08.445614 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.444541    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/9b4f9524-db75-4206-aa76-04400d95706f-stats-auth\") pod \"router-default-5bbfd9465-x28lh\" (UID: \"9b4f9524-db75-4206-aa76-04400d95706f\") " pod="openshift-ingress/router-default-5bbfd9465-x28lh"
Oct 30 21:28:08.445614 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.444569    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9f54b2c6-b1f9-42da-abdc-275fc8827b71-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-9qppl\" (UID: \"9f54b2c6-b1f9-42da-abdc-275fc8827b71\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-9qppl"
Oct 30 21:28:08.445614 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.444603    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f6750ff3-d2db-42b7-9c85-7896c9ba67c9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-l67jk\" (UID: \"f6750ff3-d2db-42b7-9c85-7896c9ba67c9\") " pod="openshift-multus/multus-additional-cni-plugins-l67jk"
Oct 30 21:28:08.445614 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.444629    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/48efa9dc-3c20-4f81-86f3-c87ad697772c-serviceca\") pod \"node-ca-jgns8\" (UID: \"48efa9dc-3c20-4f81-86f3-c87ad697772c\") " pod="openshift-image-registry/node-ca-jgns8"
Oct 30 21:28:08.445614 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.444645    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:28:08.445614 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.444651    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8873f9b9-2ac5-409c-9299-fa024e00189c-serving-cert\") pod \"service-ca-operator-544d749c8c-9r5sb\" (UID: \"8873f9b9-2ac5-409c-9299-fa024e00189c\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-9r5sb"
Oct 30 21:28:08.445614 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.444731    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ct4zm\" (UniqueName: \"kubernetes.io/projected/cf90e1fc-bac9-4c33-8dc4-dbca18974399-kube-api-access-ct4zm\") pod \"aws-ebs-csi-driver-node-qcj76\" (UID: \"cf90e1fc-bac9-4c33-8dc4-dbca18974399\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qcj76"
Oct 30 21:28:08.445614 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.444771    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dv6wd\" (UniqueName: \"kubernetes.io/projected/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-kube-api-access-dv6wd\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:28:08.445614 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.444777    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f6750ff3-d2db-42b7-9c85-7896c9ba67c9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-l67jk\" (UID: \"f6750ff3-d2db-42b7-9c85-7896c9ba67c9\") " pod="openshift-multus/multus-additional-cni-plugins-l67jk"
Oct 30 21:28:08.445614 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.444985    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d6ca21b9-20a4-4887-9d08-19d9c491a09e-serving-cert\") pod \"insights-operator-7c4986d449-7cgk9\" (UID: \"d6ca21b9-20a4-4887-9d08-19d9c491a09e\") " pod="openshift-insights/insights-operator-7c4986d449-7cgk9"
Oct 30 21:28:08.448705 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.448682    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcgbg\" (UniqueName: \"kubernetes.io/projected/bba1a087-6e04-4db8-ba82-7260a88e2843-kube-api-access-qcgbg\") pod \"network-check-source-6f78d59f99-tdmc8\" (UID: \"bba1a087-6e04-4db8-ba82-7260a88e2843\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-tdmc8"
Oct 30 21:28:08.449003 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.448984    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6lpv\" (UniqueName: \"kubernetes.io/projected/94ee3757-5440-42e1-8617-2a68744ef3f4-kube-api-access-t6lpv\") pod \"image-registry-86f87fb58b-5clbv\" (UID: \"94ee3757-5440-42e1-8617-2a68744ef3f4\") " pod="openshift-image-registry/image-registry-86f87fb58b-5clbv"
Oct 30 21:28:08.449415 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.449400    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qf6w\" (UniqueName: \"kubernetes.io/projected/0bbc8570-4947-455a-b7f8-8e9fa8e023b3-kube-api-access-9qf6w\") pod \"insights-runtime-extractor-z2dd4\" (UID: \"0bbc8570-4947-455a-b7f8-8e9fa8e023b3\") " pod="openshift-insights/insights-runtime-extractor-z2dd4"
Oct 30 21:28:08.457339 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.457314    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5qjc\" (UniqueName: \"kubernetes.io/projected/06a0877a-2fce-405f-b97c-091d88e19543-kube-api-access-l5qjc\") pod \"migrator-66df5d496b-gqdlr\" (UID: \"06a0877a-2fce-405f-b97c-091d88e19543\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-gqdlr"
Oct 30 21:28:08.458395 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.458374    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5tf5\" (UniqueName: \"kubernetes.io/projected/84a732a8-045d-4a8b-bb54-859c8d46f08d-kube-api-access-w5tf5\") pod \"iptables-alerter-bn9ng\" (UID: \"84a732a8-045d-4a8b-bb54-859c8d46f08d\") " pod="openshift-network-operator/iptables-alerter-bn9ng"
Oct 30 21:28:08.463137 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.463114    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8f74s\" (UniqueName: \"kubernetes.io/projected/11b44bf1-b3b1-4214-b7b4-4e508c618cd1-kube-api-access-8f74s\") pod \"service-ca-5fffc97bdf-wpm2q\" (UID: \"11b44bf1-b3b1-4214-b7b4-4e508c618cd1\") " pod="openshift-service-ca/service-ca-5fffc97bdf-wpm2q"
Oct 30 21:28:08.463596 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.463578    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfrc4\" (UniqueName: \"kubernetes.io/projected/9b4f9524-db75-4206-aa76-04400d95706f-kube-api-access-wfrc4\") pod \"router-default-5bbfd9465-x28lh\" (UID: \"9b4f9524-db75-4206-aa76-04400d95706f\") " pod="openshift-ingress/router-default-5bbfd9465-x28lh"
Oct 30 21:28:08.463766 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.463745    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtmzr\" (UniqueName: \"kubernetes.io/projected/48efa9dc-3c20-4f81-86f3-c87ad697772c-kube-api-access-vtmzr\") pod \"node-ca-jgns8\" (UID: \"48efa9dc-3c20-4f81-86f3-c87ad697772c\") " pod="openshift-image-registry/node-ca-jgns8"
Oct 30 21:28:08.463854 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.463801    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pls28\" (UniqueName: \"kubernetes.io/projected/3c835b41-25cb-4f0c-8e7c-4ac9e2fee318-kube-api-access-pls28\") pod \"kas-connection-checker-685895c7c9-9mt44\" (UID: \"3c835b41-25cb-4f0c-8e7c-4ac9e2fee318\") " pod="kube-system/kas-connection-checker-685895c7c9-9mt44"
Oct 30 21:28:08.463854 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.463816    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-96bzs\" (UniqueName: \"kubernetes.io/projected/352b5b4c-c640-4300-abf3-aa70b08cbb4b-kube-api-access-96bzs\") pod \"network-metrics-daemon-vwxrd\" (UID: \"352b5b4c-c640-4300-abf3-aa70b08cbb4b\") " pod="openshift-multus/network-metrics-daemon-vwxrd"
Oct 30 21:28:08.464454 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.464436    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzmzn\" (UniqueName: \"kubernetes.io/projected/e9d2f04b-bb7d-422f-b497-5fa61580f95a-kube-api-access-bzmzn\") pod \"volume-data-source-validator-58c9f5f46d-lbjhv\" (UID: \"e9d2f04b-bb7d-422f-b497-5fa61580f95a\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-lbjhv"
Oct 30 21:28:08.465704 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.465683    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jsdkt\" (UniqueName: \"kubernetes.io/projected/8873f9b9-2ac5-409c-9299-fa024e00189c-kube-api-access-jsdkt\") pod \"service-ca-operator-544d749c8c-9r5sb\" (UID: \"8873f9b9-2ac5-409c-9299-fa024e00189c\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-9r5sb"
Oct 30 21:28:08.465766 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.465700    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-spppx\" (UniqueName: \"kubernetes.io/projected/8bb24a0d-3af0-4a05-bba4-4f9630b04bf3-kube-api-access-spppx\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-9p8mb\" (UID: \"8bb24a0d-3af0-4a05-bba4-4f9630b04bf3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-9p8mb"
Oct 30 21:28:08.468420 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.468401    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-klwsc\" (UniqueName: \"kubernetes.io/projected/baa8d0c0-d477-4f30-a0f6-375fc84cc448-kube-api-access-klwsc\") pod \"node-resolver-97tsr\" (UID: \"baa8d0c0-d477-4f30-a0f6-375fc84cc448\") " pod="openshift-dns/node-resolver-97tsr"
Oct 30 21:28:08.468529 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.468512    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wp72\" (UniqueName: \"kubernetes.io/projected/0d83c70b-fab8-4872-969f-11f74c0ddea0-kube-api-access-9wp72\") pod \"kas-connection-checker-685895c7c9-zvqvt\" (UID: \"0d83c70b-fab8-4872-969f-11f74c0ddea0\") " pod="kube-system/kas-connection-checker-685895c7c9-zvqvt"
Oct 30 21:28:08.468782 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.468523    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pklf4\" (UniqueName: \"kubernetes.io/projected/d6ca21b9-20a4-4887-9d08-19d9c491a09e-kube-api-access-pklf4\") pod \"insights-operator-7c4986d449-7cgk9\" (UID: \"d6ca21b9-20a4-4887-9d08-19d9c491a09e\") " pod="openshift-insights/insights-operator-7c4986d449-7cgk9"
Oct 30 21:28:08.469082 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.469058    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhrs4\" (UniqueName: \"kubernetes.io/projected/faca011a-4ff1-4de0-9af1-ed9b0f253aeb-kube-api-access-dhrs4\") pod \"migrator-66df5d496b-l7nbh\" (UID: \"faca011a-4ff1-4de0-9af1-ed9b0f253aeb\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-l7nbh"
Oct 30 21:28:08.469172 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.469091    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdf4s\" (UniqueName: \"kubernetes.io/projected/08498890-fb63-4aea-9cf7-a2ade5397df2-kube-api-access-hdf4s\") pod \"dns-default-sbhch\" (UID: \"08498890-fb63-4aea-9cf7-a2ade5397df2\") " pod="openshift-dns/dns-default-sbhch"
Oct 30 21:28:08.469172 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.469151    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/039cf439-d12d-450a-80c1-eb3efeb50e8a-bound-sa-token\") pod \"image-registry-76679b6db7-ltmt6\" (UID: \"039cf439-d12d-450a-80c1-eb3efeb50e8a\") " pod="openshift-image-registry/image-registry-76679b6db7-ltmt6"
Oct 30 21:28:08.469476 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.469455    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ms8rv\" (UniqueName: \"kubernetes.io/projected/3913f7d4-86cb-4e86-af5a-debd8b859252-kube-api-access-ms8rv\") pod \"network-check-target-4cvlt\" (UID: \"3913f7d4-86cb-4e86-af5a-debd8b859252\") " pod="openshift-network-diagnostics/network-check-target-4cvlt"
Oct 30 21:28:08.469622 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.469601    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffq8c\" (UniqueName: \"kubernetes.io/projected/fb314739-c678-4da8-9d54-147dc0612ee2-kube-api-access-ffq8c\") pod \"kas-connection-checker-685895c7c9-l4dbk\" (UID: \"fb314739-c678-4da8-9d54-147dc0612ee2\") " pod="kube-system/kas-connection-checker-685895c7c9-l4dbk"
Oct 30 21:28:08.473909 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.470634    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ct4zm\" (UniqueName: \"kubernetes.io/projected/cf90e1fc-bac9-4c33-8dc4-dbca18974399-kube-api-access-ct4zm\") pod \"aws-ebs-csi-driver-node-qcj76\" (UID: \"cf90e1fc-bac9-4c33-8dc4-dbca18974399\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qcj76"
Oct 30 21:28:08.475292 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.475050    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dv6wd\" (UniqueName: \"kubernetes.io/projected/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-kube-api-access-dv6wd\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:28:08.475292 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.475212    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8km8\" (UniqueName: \"kubernetes.io/projected/67292bdc-7fe1-449d-af75-2baa90c16ea9-kube-api-access-m8km8\") pod \"console-operator-7cd56b8dbc-sklxd\" (UID: \"67292bdc-7fe1-449d-af75-2baa90c16ea9\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-sklxd"
Oct 30 21:28:08.475292 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.475242    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrrdm\" (UniqueName: \"kubernetes.io/projected/039cf439-d12d-450a-80c1-eb3efeb50e8a-kube-api-access-zrrdm\") pod \"image-registry-76679b6db7-ltmt6\" (UID: \"039cf439-d12d-450a-80c1-eb3efeb50e8a\") " pod="openshift-image-registry/image-registry-76679b6db7-ltmt6"
Oct 30 21:28:08.475292 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.475247    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqwd9\" (UniqueName: \"kubernetes.io/projected/9f54b2c6-b1f9-42da-abdc-275fc8827b71-kube-api-access-jqwd9\") pod \"cluster-samples-operator-6ddb6c8b94-9qppl\" (UID: \"9f54b2c6-b1f9-42da-abdc-275fc8827b71\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-9qppl"
Oct 30 21:28:08.475292 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.475256    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfphc\" (UniqueName: \"kubernetes.io/projected/3175ff47-1038-4139-9c0a-87062bb38dc2-kube-api-access-zfphc\") pod \"ingress-canary-hzvrj\" (UID: \"3175ff47-1038-4139-9c0a-87062bb38dc2\") " pod="openshift-ingress-canary/ingress-canary-hzvrj"
Oct 30 21:28:08.475574 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.475494    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/94ee3757-5440-42e1-8617-2a68744ef3f4-bound-sa-token\") pod \"image-registry-86f87fb58b-5clbv\" (UID: \"94ee3757-5440-42e1-8617-2a68744ef3f4\") " pod="openshift-image-registry/image-registry-86f87fb58b-5clbv"
Oct 30 21:28:08.475753 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.475732    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnx5t\" (UniqueName: \"kubernetes.io/projected/fd34a723-6c9e-48f1-ba7d-924d542b2bc7-kube-api-access-fnx5t\") pod \"periodic-gathering-542ht-hjp6z\" (UID: \"fd34a723-6c9e-48f1-ba7d-924d542b2bc7\") " pod="openshift-insights/periodic-gathering-542ht-hjp6z"
Oct 30 21:28:08.475885 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.475866    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhvst\" (UniqueName: \"kubernetes.io/projected/f6750ff3-d2db-42b7-9c85-7896c9ba67c9-kube-api-access-qhvst\") pod \"multus-additional-cni-plugins-l67jk\" (UID: \"f6750ff3-d2db-42b7-9c85-7896c9ba67c9\") " pod="openshift-multus/multus-additional-cni-plugins-l67jk"
Oct 30 21:28:08.476112 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.476095    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-56qdr\" (UniqueName: \"kubernetes.io/projected/90937bc9-0051-407d-9c82-014f0a9c65d4-kube-api-access-56qdr\") pod \"cluster-monitoring-operator-884576c57-kdrrd\" (UID: \"90937bc9-0051-407d-9c82-014f0a9c65d4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-kdrrd"
Oct 30 21:28:08.476598 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.476578    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2trdz\" (UniqueName: \"kubernetes.io/projected/5e93253e-a238-4426-a584-b9de8e931cf7-kube-api-access-2trdz\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:28:08.476669 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.476627    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rf7bl\" (UniqueName: \"kubernetes.io/projected/5d90d533-ebca-46db-ab59-733837db23e6-kube-api-access-rf7bl\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:28:08.545638 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.545561    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-log-socket\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:28:08.545638 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.545595    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-host-run-ovn-kubernetes\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:28:08.545638 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.545614    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/753d4639-09e6-4850-9f4d-ed39b0406439-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-lwm89\" (UID: \"753d4639-09e6-4850-9f4d-ed39b0406439\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-lwm89"
Oct 30 21:28:08.545854 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.545641    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-hostroot\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:28:08.545854 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.545664    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/baa8d0c0-d477-4f30-a0f6-375fc84cc448-hosts-file\") pod \"node-resolver-97tsr\" (UID: \"baa8d0c0-d477-4f30-a0f6-375fc84cc448\") " pod="openshift-dns/node-resolver-97tsr"
Oct 30 21:28:08.545854 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.545669    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-log-socket\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:28:08.545854 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.545688    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-host-run-ovn-kubernetes\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:28:08.545854 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.545726    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/baa8d0c0-d477-4f30-a0f6-375fc84cc448-hosts-file\") pod \"node-resolver-97tsr\" (UID: \"baa8d0c0-d477-4f30-a0f6-375fc84cc448\") " pod="openshift-dns/node-resolver-97tsr"
Oct 30 21:28:08.545854 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.545756    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-hostroot\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:28:08.545854 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:28:08.545768    5032 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 21:28:08.545854 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.545777    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0bbc8570-4947-455a-b7f8-8e9fa8e023b3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z2dd4\" (UID: \"0bbc8570-4947-455a-b7f8-8e9fa8e023b3\") " pod="openshift-insights/insights-runtime-extractor-z2dd4"
Oct 30 21:28:08.545854 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.545800    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-run-ovn\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:28:08.545854 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.545825    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-host-var-lib-cni-bin\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:28:08.545854 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:28:08.545844    5032 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/753d4639-09e6-4850-9f4d-ed39b0406439-networking-console-plugin-cert podName:753d4639-09e6-4850-9f4d-ed39b0406439 nodeName:}" failed. No retries permitted until 2026-10-30 21:28:09.045828017 +0000 UTC m=+1.847640474 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/753d4639-09e6-4850-9f4d-ed39b0406439-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-lwm89" (UID: "753d4639-09e6-4850-9f4d-ed39b0406439") : secret "networking-console-plugin-cert" not found
Oct 30 21:28:08.546214 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:28:08.545856    5032 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 21:28:08.546214 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.545875    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cf90e1fc-bac9-4c33-8dc4-dbca18974399-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qcj76\" (UID: \"cf90e1fc-bac9-4c33-8dc4-dbca18974399\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qcj76"
Oct 30 21:28:08.546214 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.545903    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-cnibin\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:28:08.546214 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:28:08.545916    5032 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0bbc8570-4947-455a-b7f8-8e9fa8e023b3-insights-runtime-extractor-tls podName:0bbc8570-4947-455a-b7f8-8e9fa8e023b3 nodeName:}" failed. No retries permitted until 2026-10-30 21:28:09.045904809 +0000 UTC m=+1.847717275 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0bbc8570-4947-455a-b7f8-8e9fa8e023b3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-z2dd4" (UID: "0bbc8570-4947-455a-b7f8-8e9fa8e023b3") : secret "insights-runtime-extractor-tls" not found
Oct 30 21:28:08.546214 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.545862    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-host-var-lib-cni-bin\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:28:08.546214 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.545861    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-run-ovn\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:28:08.546214 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.545951    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-etc-sysctl-d\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:28:08.546214 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.545973    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cf90e1fc-bac9-4c33-8dc4-dbca18974399-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qcj76\" (UID: \"cf90e1fc-bac9-4c33-8dc4-dbca18974399\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qcj76"
Oct 30 21:28:08.546214 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.545980    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-etc-sysctl-conf\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:28:08.546214 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.546002    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-etc-systemd\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:28:08.546214 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.545956    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-cnibin\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:28:08.546214 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.546024    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-systemd-units\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:28:08.546214 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.546051    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-etc-openvswitch\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:28:08.546214 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.546056    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-systemd-units\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:28:08.546214 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.546061    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-etc-systemd\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:28:08.546214 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.546077    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cf90e1fc-bac9-4c33-8dc4-dbca18974399-device-dir\") pod \"aws-ebs-csi-driver-node-qcj76\" (UID: \"cf90e1fc-bac9-4c33-8dc4-dbca18974399\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qcj76"
Oct 30 21:28:08.546214 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.546089    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-etc-sysctl-conf\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:28:08.546214 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.546096    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-etc-openvswitch\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:28:08.546214 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.546095    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-etc-sysctl-d\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:28:08.546214 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.546102    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-run-systemd\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:28:08.546214 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.546131    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-multus-conf-dir\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:28:08.546214 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.546132    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-run-systemd\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:28:08.546214 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.546157    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cf90e1fc-bac9-4c33-8dc4-dbca18974399-device-dir\") pod \"aws-ebs-csi-driver-node-qcj76\" (UID: \"cf90e1fc-bac9-4c33-8dc4-dbca18974399\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qcj76"
Oct 30 21:28:08.546214 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.546160    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-multus-conf-dir\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:28:08.546214 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.546180    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-run\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:28:08.546214 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.546205    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-etc-modprobe-d\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:28:08.546214 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.546221    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f6750ff3-d2db-42b7-9c85-7896c9ba67c9-system-cni-dir\") pod \"multus-additional-cni-plugins-l67jk\" (UID: \"f6750ff3-d2db-42b7-9c85-7896c9ba67c9\") " pod="openshift-multus/multus-additional-cni-plugins-l67jk"
Oct 30 21:28:08.546214 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.546235    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-multus-cni-dir\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.546248    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-run\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.546253    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-host-var-lib-cni-multus\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.546270    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f6750ff3-d2db-42b7-9c85-7896c9ba67c9-system-cni-dir\") pod \"multus-additional-cni-plugins-l67jk\" (UID: \"f6750ff3-d2db-42b7-9c85-7896c9ba67c9\") " pod="openshift-multus/multus-additional-cni-plugins-l67jk"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.546280    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-host-var-lib-cni-multus\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.546284    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5f171661-33a2-4631-81c7-bc10a5db791f-dbus\") pod \"global-pull-secret-syncer-csbxh\" (UID: \"5f171661-33a2-4631-81c7-bc10a5db791f\") " pod="kube-system/global-pull-secret-syncer-csbxh"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.546305    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/94ee3757-5440-42e1-8617-2a68744ef3f4-registry-tls\") pod \"image-registry-86f87fb58b-5clbv\" (UID: \"94ee3757-5440-42e1-8617-2a68744ef3f4\") " pod="openshift-image-registry/image-registry-86f87fb58b-5clbv"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.546319    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-multus-cni-dir\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.546322    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08498890-fb63-4aea-9cf7-a2ade5397df2-metrics-tls\") pod \"dns-default-sbhch\" (UID: \"08498890-fb63-4aea-9cf7-a2ade5397df2\") " pod="openshift-dns/dns-default-sbhch"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.546321    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-etc-modprobe-d\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.546345    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-host-cni-netd\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.546385    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-host-run-netns\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:28:08.546398    5032 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.546408    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5f171661-33a2-4631-81c7-bc10a5db791f-kubelet-config\") pod \"global-pull-secret-syncer-csbxh\" (UID: \"5f171661-33a2-4631-81c7-bc10a5db791f\") " pod="kube-system/global-pull-secret-syncer-csbxh"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.546415    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5f171661-33a2-4631-81c7-bc10a5db791f-dbus\") pod \"global-pull-secret-syncer-csbxh\" (UID: \"5f171661-33a2-4631-81c7-bc10a5db791f\") " pod="kube-system/global-pull-secret-syncer-csbxh"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.546432    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/352b5b4c-c640-4300-abf3-aa70b08cbb4b-metrics-certs\") pod \"network-metrics-daemon-vwxrd\" (UID: \"352b5b4c-c640-4300-abf3-aa70b08cbb4b\") " pod="openshift-multus/network-metrics-daemon-vwxrd"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.546454    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b4f9524-db75-4206-aa76-04400d95706f-service-ca-bundle\") pod \"router-default-5bbfd9465-x28lh\" (UID: \"9b4f9524-db75-4206-aa76-04400d95706f\") " pod="openshift-ingress/router-default-5bbfd9465-x28lh"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.546459    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-host-cni-netd\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.546467    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-host-run-netns\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.546482    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-etc-kubernetes\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:28:08.546532    5032 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08498890-fb63-4aea-9cf7-a2ade5397df2-metrics-tls podName:08498890-fb63-4aea-9cf7-a2ade5397df2 nodeName:}" failed. No retries permitted until 2026-10-30 21:28:09.046493422 +0000 UTC m=+1.848305882 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/08498890-fb63-4aea-9cf7-a2ade5397df2-metrics-tls") pod "dns-default-sbhch" (UID: "08498890-fb63-4aea-9cf7-a2ade5397df2") : secret "dns-default-metrics-tls" not found
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.546538    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-etc-kubernetes\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:28:08.546551    5032 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:28:08.546561    5032 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9b4f9524-db75-4206-aa76-04400d95706f-service-ca-bundle podName:9b4f9524-db75-4206-aa76-04400d95706f nodeName:}" failed. No retries permitted until 2026-10-30 21:28:09.046550485 +0000 UTC m=+1.848362951 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9b4f9524-db75-4206-aa76-04400d95706f-service-ca-bundle") pod "router-default-5bbfd9465-x28lh" (UID: "9b4f9524-db75-4206-aa76-04400d95706f") : configmap references non-existent config key: service-ca.crt
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.546552    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-host\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.546594    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0bbc8570-4947-455a-b7f8-8e9fa8e023b3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-z2dd4\" (UID: \"0bbc8570-4947-455a-b7f8-8e9fa8e023b3\") " pod="openshift-insights/insights-runtime-extractor-z2dd4"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.546617    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-host\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.546620    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b4f9524-db75-4206-aa76-04400d95706f-metrics-certs\") pod \"router-default-5bbfd9465-x28lh\" (UID: \"9b4f9524-db75-4206-aa76-04400d95706f\") " pod="openshift-ingress/router-default-5bbfd9465-x28lh"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.546537    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5f171661-33a2-4631-81c7-bc10a5db791f-kubelet-config\") pod \"global-pull-secret-syncer-csbxh\" (UID: \"5f171661-33a2-4631-81c7-bc10a5db791f\") " pod="kube-system/global-pull-secret-syncer-csbxh"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.546681    5032 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:28:08.546767    5032 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/352b5b4c-c640-4300-abf3-aa70b08cbb4b-metrics-certs podName:352b5b4c-c640-4300-abf3-aa70b08cbb4b nodeName:}" failed. No retries permitted until 2026-10-30 21:28:09.046749431 +0000 UTC m=+1.848561904 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/352b5b4c-c640-4300-abf3-aa70b08cbb4b-metrics-certs") pod "network-metrics-daemon-vwxrd" (UID: "352b5b4c-c640-4300-abf3-aa70b08cbb4b") : secret "metrics-daemon-secret" not found
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.546792    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0bbc8570-4947-455a-b7f8-8e9fa8e023b3-crio-socket\") pod \"insights-runtime-extractor-z2dd4\" (UID: \"0bbc8570-4947-455a-b7f8-8e9fa8e023b3\") " pod="openshift-insights/insights-runtime-extractor-z2dd4"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.546824    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-node-log\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.546853    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-etc-kubernetes\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.546878    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cf90e1fc-bac9-4c33-8dc4-dbca18974399-registration-dir\") pod \"aws-ebs-csi-driver-node-qcj76\" (UID: \"cf90e1fc-bac9-4c33-8dc4-dbca18974399\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qcj76"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.546902    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-lib-modules\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.546929    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9f54b2c6-b1f9-42da-abdc-275fc8827b71-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-9qppl\" (UID: \"9f54b2c6-b1f9-42da-abdc-275fc8827b71\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-9qppl"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.546959    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.546990    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-node-log\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.546994    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f6750ff3-d2db-42b7-9c85-7896c9ba67c9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-l67jk\" (UID: \"f6750ff3-d2db-42b7-9c85-7896c9ba67c9\") " pod="openshift-multus/multus-additional-cni-plugins-l67jk"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.547039    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cf90e1fc-bac9-4c33-8dc4-dbca18974399-sys-fs\") pod \"aws-ebs-csi-driver-node-qcj76\" (UID: \"cf90e1fc-bac9-4c33-8dc4-dbca18974399\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qcj76"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.547042    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cf90e1fc-bac9-4c33-8dc4-dbca18974399-registration-dir\") pod \"aws-ebs-csi-driver-node-qcj76\" (UID: \"cf90e1fc-bac9-4c33-8dc4-dbca18974399\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qcj76"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.547065    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-host-kubelet\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.547091    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-system-cni-dir\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.547144    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/84a732a8-045d-4a8b-bb54-859c8d46f08d-host-slash\") pod \"iptables-alerter-bn9ng\" (UID: \"84a732a8-045d-4a8b-bb54-859c8d46f08d\") " pod="openshift-network-operator/iptables-alerter-bn9ng"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.547159    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-lib-modules\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.547168    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-run-openvswitch\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.547192    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-host-run-netns\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.547213    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-etc-kubernetes\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.547219    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/90937bc9-0051-407d-9c82-014f0a9c65d4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-kdrrd\" (UID: \"90937bc9-0051-407d-9c82-014f0a9c65d4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-kdrrd"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.547247    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cf90e1fc-bac9-4c33-8dc4-dbca18974399-etc-selinux\") pod \"aws-ebs-csi-driver-node-qcj76\" (UID: \"cf90e1fc-bac9-4c33-8dc4-dbca18974399\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qcj76"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.547251    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0bbc8570-4947-455a-b7f8-8e9fa8e023b3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-z2dd4\" (UID: \"0bbc8570-4947-455a-b7f8-8e9fa8e023b3\") " pod="openshift-insights/insights-runtime-extractor-z2dd4"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.547261    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-system-cni-dir\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.547270    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/48efa9dc-3c20-4f81-86f3-c87ad697772c-host\") pod \"node-ca-jgns8\" (UID: \"48efa9dc-3c20-4f81-86f3-c87ad697772c\") " pod="openshift-image-registry/node-ca-jgns8"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.547051    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f6750ff3-d2db-42b7-9c85-7896c9ba67c9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-l67jk\" (UID: \"f6750ff3-d2db-42b7-9c85-7896c9ba67c9\") " pod="openshift-multus/multus-additional-cni-plugins-l67jk"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.547301    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-sys\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.547312    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0bbc8570-4947-455a-b7f8-8e9fa8e023b3-crio-socket\") pod \"insights-runtime-extractor-z2dd4\" (UID: \"0bbc8570-4947-455a-b7f8-8e9fa8e023b3\") " pod="openshift-insights/insights-runtime-extractor-z2dd4"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.547326    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-var-lib-kubelet\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.547329    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cf90e1fc-bac9-4c33-8dc4-dbca18974399-etc-selinux\") pod \"aws-ebs-csi-driver-node-qcj76\" (UID: \"cf90e1fc-bac9-4c33-8dc4-dbca18974399\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qcj76"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.547346    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/48efa9dc-3c20-4f81-86f3-c87ad697772c-host\") pod \"node-ca-jgns8\" (UID: \"48efa9dc-3c20-4f81-86f3-c87ad697772c\") " pod="openshift-image-registry/node-ca-jgns8"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.547351    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f6750ff3-d2db-42b7-9c85-7896c9ba67c9-cnibin\") pod \"multus-additional-cni-plugins-l67jk\" (UID: \"f6750ff3-d2db-42b7-9c85-7896c9ba67c9\") " pod="openshift-multus/multus-additional-cni-plugins-l67jk"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.547397    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-sys\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.547401    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f6750ff3-d2db-42b7-9c85-7896c9ba67c9-cnibin\") pod \"multus-additional-cni-plugins-l67jk\" (UID: \"f6750ff3-d2db-42b7-9c85-7896c9ba67c9\") " pod="openshift-multus/multus-additional-cni-plugins-l67jk"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.547404    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-var-lib-openvswitch\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.547447    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-host-run-netns\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.547445    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-run-openvswitch\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.547469    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.547493    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-multus-socket-dir-parent\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.547504    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-var-lib-openvswitch\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.547523    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-host-kubelet\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.547495    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cf90e1fc-bac9-4c33-8dc4-dbca18974399-sys-fs\") pod \"aws-ebs-csi-driver-node-qcj76\" (UID: \"cf90e1fc-bac9-4c33-8dc4-dbca18974399\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qcj76"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.547571    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-var-lib-kubelet\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.547573    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/84a732a8-045d-4a8b-bb54-859c8d46f08d-host-slash\") pod \"iptables-alerter-bn9ng\" (UID: \"84a732a8-045d-4a8b-bb54-859c8d46f08d\") " pod="openshift-network-operator/iptables-alerter-bn9ng"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.547523    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/039cf439-d12d-450a-80c1-eb3efeb50e8a-registry-tls\") pod \"image-registry-76679b6db7-ltmt6\" (UID: \"039cf439-d12d-450a-80c1-eb3efeb50e8a\") " pod="openshift-image-registry/image-registry-76679b6db7-ltmt6"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.547614    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-host-run-multus-certs\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:28:08.547626    5032 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.547644    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-etc-sysconfig\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:28:08.547682    5032 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90937bc9-0051-407d-9c82-014f0a9c65d4-cluster-monitoring-operator-tls podName:90937bc9-0051-407d-9c82-014f0a9c65d4 nodeName:}" failed. No retries permitted until 2026-10-30 21:28:09.047668277 +0000 UTC m=+1.849480739 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/90937bc9-0051-407d-9c82-014f0a9c65d4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-kdrrd" (UID: "90937bc9-0051-407d-9c82-014f0a9c65d4") : secret "cluster-monitoring-operator-tls" not found
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.547573    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-multus-socket-dir-parent\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.547699    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-host-slash\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.547703    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7deac9ed-2d3a-48c3-bcf8-566ffdef0869-etc-sysconfig\") pod \"tuned-7ntsz\" (UID: \"7deac9ed-2d3a-48c3-bcf8-566ffdef0869\") " pod="openshift-cluster-node-tuning-operator/tuned-7ntsz"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.547726    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-host-cni-bin\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.547677    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-host-run-multus-certs\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.547750    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-host-var-lib-kubelet\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.547783    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-host-slash\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.547785    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-host-run-k8s-cni-cncf-io\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.547815    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cf90e1fc-bac9-4c33-8dc4-dbca18974399-socket-dir\") pod \"aws-ebs-csi-driver-node-qcj76\" (UID: \"cf90e1fc-bac9-4c33-8dc4-dbca18974399\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qcj76"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.547817    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5d90d533-ebca-46db-ab59-733837db23e6-host-cni-bin\") pod \"ovnkube-node-gl47s\" (UID: \"5d90d533-ebca-46db-ab59-733837db23e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.547816    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-host-run-k8s-cni-cncf-io\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.547861    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cf90e1fc-bac9-4c33-8dc4-dbca18974399-socket-dir\") pod \"aws-ebs-csi-driver-node-qcj76\" (UID: \"cf90e1fc-bac9-4c33-8dc4-dbca18974399\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qcj76"
Oct 30 21:28:08.547904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.547893    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5e93253e-a238-4426-a584-b9de8e931cf7-host-var-lib-kubelet\") pod \"multus-d9b8n\" (UID: \"5e93253e-a238-4426-a584-b9de8e931cf7\") " pod="openshift-multus/multus-d9b8n"
Oct 30 21:28:08.551236 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.549397    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/94ee3757-5440-42e1-8617-2a68744ef3f4-registry-tls\") pod \"image-registry-86f87fb58b-5clbv\" (UID: \"94ee3757-5440-42e1-8617-2a68744ef3f4\") " pod="openshift-image-registry/image-registry-86f87fb58b-5clbv"
Oct 30 21:28:08.551236 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.549830    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9f54b2c6-b1f9-42da-abdc-275fc8827b71-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-9qppl\" (UID: \"9f54b2c6-b1f9-42da-abdc-275fc8827b71\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-9qppl"
Oct 30 21:28:08.551236 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.550002    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/039cf439-d12d-450a-80c1-eb3efeb50e8a-registry-tls\") pod \"image-registry-76679b6db7-ltmt6\" (UID: \"039cf439-d12d-450a-80c1-eb3efeb50e8a\") " pod="openshift-image-registry/image-registry-76679b6db7-ltmt6"
Oct 30 21:28:08.551236 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.550103    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b4f9524-db75-4206-aa76-04400d95706f-metrics-certs\") pod \"router-default-5bbfd9465-x28lh\" (UID: \"9b4f9524-db75-4206-aa76-04400d95706f\") " pod="openshift-ingress/router-default-5bbfd9465-x28lh"
Oct 30 21:28:08.614420 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.614383    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-9qppl"
Oct 30 21:28:08.614564 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.614431    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-86f87fb58b-5clbv"
Oct 30 21:28:08.615453 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.615434    5032 scope.go:122] "RemoveContainer" containerID="f667ffa124949edf8581923668ce1139c1ad553b96aeaa80c8bc06474a49998a"
Oct 30 21:28:08.626936 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.626909    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76679b6db7-ltmt6"
Oct 30 21:28:08.927785 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.927689    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-9qppl"]
Oct 30 21:28:08.968992 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.968966    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-86f87fb58b-5clbv"]
Oct 30 21:28:08.972889 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.972096    5032 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 21:28:08.972889 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:08.972323    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94ee3757_5440_42e1_8617_2a68744ef3f4.slice/crio-8a95cd95aced123726cf205b6e1b2f9a9430212f7cd83936c4e10ed0b98fc054 WatchSource:0}: Error finding container 8a95cd95aced123726cf205b6e1b2f9a9430212f7cd83936c4e10ed0b98fc054: Status 404 returned error can't find the container with id 8a95cd95aced123726cf205b6e1b2f9a9430212f7cd83936c4e10ed0b98fc054
Oct 30 21:28:08.998465 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:08.998438    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-76679b6db7-ltmt6"]
Oct 30 21:28:09.002652 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:09.002621    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod039cf439_d12d_450a_80c1_eb3efeb50e8a.slice/crio-7ce3e4dac809a9eee2dd50a2b0facda4fa1b1b7342abc9a3eed42cbf9920fb9d WatchSource:0}: Error finding container 7ce3e4dac809a9eee2dd50a2b0facda4fa1b1b7342abc9a3eed42cbf9920fb9d: Status 404 returned error can't find the container with id 7ce3e4dac809a9eee2dd50a2b0facda4fa1b1b7342abc9a3eed42cbf9920fb9d
Oct 30 21:28:09.052609 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:09.052583    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/90937bc9-0051-407d-9c82-014f0a9c65d4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-kdrrd\" (UID: \"90937bc9-0051-407d-9c82-014f0a9c65d4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-kdrrd"
Oct 30 21:28:09.052723 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:09.052639    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/753d4639-09e6-4850-9f4d-ed39b0406439-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-lwm89\" (UID: \"753d4639-09e6-4850-9f4d-ed39b0406439\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-lwm89"
Oct 30 21:28:09.052723 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:09.052712    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0bbc8570-4947-455a-b7f8-8e9fa8e023b3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z2dd4\" (UID: \"0bbc8570-4947-455a-b7f8-8e9fa8e023b3\") " pod="openshift-insights/insights-runtime-extractor-z2dd4"
Oct 30 21:28:09.052840 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:28:09.052732    5032 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 21:28:09.052840 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:09.052767    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08498890-fb63-4aea-9cf7-a2ade5397df2-metrics-tls\") pod \"dns-default-sbhch\" (UID: \"08498890-fb63-4aea-9cf7-a2ade5397df2\") " pod="openshift-dns/dns-default-sbhch"
Oct 30 21:28:09.052840 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:28:09.052778    5032 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/753d4639-09e6-4850-9f4d-ed39b0406439-networking-console-plugin-cert podName:753d4639-09e6-4850-9f4d-ed39b0406439 nodeName:}" failed. No retries permitted until 2026-10-30 21:28:10.05276538 +0000 UTC m=+2.854577841 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/753d4639-09e6-4850-9f4d-ed39b0406439-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-lwm89" (UID: "753d4639-09e6-4850-9f4d-ed39b0406439") : secret "networking-console-plugin-cert" not found
Oct 30 21:28:09.052840 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:09.052801    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/352b5b4c-c640-4300-abf3-aa70b08cbb4b-metrics-certs\") pod \"network-metrics-daemon-vwxrd\" (UID: \"352b5b4c-c640-4300-abf3-aa70b08cbb4b\") " pod="openshift-multus/network-metrics-daemon-vwxrd"
Oct 30 21:28:09.052840 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:09.052830    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b4f9524-db75-4206-aa76-04400d95706f-service-ca-bundle\") pod \"router-default-5bbfd9465-x28lh\" (UID: \"9b4f9524-db75-4206-aa76-04400d95706f\") " pod="openshift-ingress/router-default-5bbfd9465-x28lh"
Oct 30 21:28:09.053101 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:28:09.052833    5032 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 30 21:28:09.053101 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:28:09.052885    5032 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 30 21:28:09.053101 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:28:09.052897    5032 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90937bc9-0051-407d-9c82-014f0a9c65d4-cluster-monitoring-operator-tls podName:90937bc9-0051-407d-9c82-014f0a9c65d4 nodeName:}" failed. No retries permitted until 2026-10-30 21:28:10.052889472 +0000 UTC m=+2.854701937 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/90937bc9-0051-407d-9c82-014f0a9c65d4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-kdrrd" (UID: "90937bc9-0051-407d-9c82-014f0a9c65d4") : secret "cluster-monitoring-operator-tls" not found
Oct 30 21:28:09.053101 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:28:09.052944    5032 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 21:28:09.053101 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:28:09.052949    5032 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9b4f9524-db75-4206-aa76-04400d95706f-service-ca-bundle podName:9b4f9524-db75-4206-aa76-04400d95706f nodeName:}" failed. No retries permitted until 2026-10-30 21:28:10.05293686 +0000 UTC m=+2.854749302 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9b4f9524-db75-4206-aa76-04400d95706f-service-ca-bundle") pod "router-default-5bbfd9465-x28lh" (UID: "9b4f9524-db75-4206-aa76-04400d95706f") : configmap references non-existent config key: service-ca.crt
Oct 30 21:28:09.053101 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:28:09.052961    5032 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/352b5b4c-c640-4300-abf3-aa70b08cbb4b-metrics-certs podName:352b5b4c-c640-4300-abf3-aa70b08cbb4b nodeName:}" failed. No retries permitted until 2026-10-30 21:28:10.05295511 +0000 UTC m=+2.854767552 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/352b5b4c-c640-4300-abf3-aa70b08cbb4b-metrics-certs") pod "network-metrics-daemon-vwxrd" (UID: "352b5b4c-c640-4300-abf3-aa70b08cbb4b") : secret "metrics-daemon-secret" not found
Oct 30 21:28:09.053101 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:28:09.052990    5032 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08498890-fb63-4aea-9cf7-a2ade5397df2-metrics-tls podName:08498890-fb63-4aea-9cf7-a2ade5397df2 nodeName:}" failed. No retries permitted until 2026-10-30 21:28:10.052974063 +0000 UTC m=+2.854786515 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/08498890-fb63-4aea-9cf7-a2ade5397df2-metrics-tls") pod "dns-default-sbhch" (UID: "08498890-fb63-4aea-9cf7-a2ade5397df2") : secret "dns-default-metrics-tls" not found
Oct 30 21:28:09.053101 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:28:09.053013    5032 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 21:28:09.053101 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:28:09.053039    5032 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0bbc8570-4947-455a-b7f8-8e9fa8e023b3-insights-runtime-extractor-tls podName:0bbc8570-4947-455a-b7f8-8e9fa8e023b3 nodeName:}" failed. No retries permitted until 2026-10-30 21:28:10.053031834 +0000 UTC m=+2.854844316 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0bbc8570-4947-455a-b7f8-8e9fa8e023b3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-z2dd4" (UID: "0bbc8570-4947-455a-b7f8-8e9fa8e023b3") : secret "insights-runtime-extractor-tls" not found
Oct 30 21:28:09.268773 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:09.268673    5032 kubelet.go:2727] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7cd56b8dbc-sklxd"
Oct 30 21:28:09.355171 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:09.355144    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:28:09.411111 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:09.411084    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-sklxd_67292bdc-7fe1-449d-af75-2baa90c16ea9/console-operator/1.log"
Oct 30 21:28:09.411290 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:09.411183    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-sklxd" event={"ID":"67292bdc-7fe1-449d-af75-2baa90c16ea9","Type":"ContainerStarted","Data":"127679567c56d759f2f610848a5d8bac27ba1af590f62155888532a91d612182"}
Oct 30 21:28:09.412935 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:09.412910    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76679b6db7-ltmt6" event={"ID":"039cf439-d12d-450a-80c1-eb3efeb50e8a","Type":"ContainerStarted","Data":"b9d509b1b6463a1b3dd6efc1021cb82cdd6139c5d78f759ed3a0d3ced055ebe5"}
Oct 30 21:28:09.413080 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:09.412949    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76679b6db7-ltmt6" event={"ID":"039cf439-d12d-450a-80c1-eb3efeb50e8a","Type":"ContainerStarted","Data":"7ce3e4dac809a9eee2dd50a2b0facda4fa1b1b7342abc9a3eed42cbf9920fb9d"}
Oct 30 21:28:09.414129 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:09.414106    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-9qppl" event={"ID":"9f54b2c6-b1f9-42da-abdc-275fc8827b71","Type":"ContainerStarted","Data":"de4b6336f5b3dec28c260a680c073e5630b6c8fc70058318cdb8392aaf30568f"}
Oct 30 21:28:09.416263 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:09.415739    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-86f87fb58b-5clbv" event={"ID":"94ee3757-5440-42e1-8617-2a68744ef3f4","Type":"ContainerStarted","Data":"dc727b9cb12e0b753212acb1bd0d7a41416b5367c51e9bb65267bc006b18a1f6"}
Oct 30 21:28:09.416263 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:09.415797    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-86f87fb58b-5clbv" event={"ID":"94ee3757-5440-42e1-8617-2a68744ef3f4","Type":"ContainerStarted","Data":"8a95cd95aced123726cf205b6e1b2f9a9430212f7cd83936c4e10ed0b98fc054"}
Oct 30 21:28:10.064303 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:10.064251    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/90937bc9-0051-407d-9c82-014f0a9c65d4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-kdrrd\" (UID: \"90937bc9-0051-407d-9c82-014f0a9c65d4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-kdrrd"
Oct 30 21:28:10.064566 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:10.064343    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/753d4639-09e6-4850-9f4d-ed39b0406439-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-lwm89\" (UID: \"753d4639-09e6-4850-9f4d-ed39b0406439\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-lwm89"
Oct 30 21:28:10.064566 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:10.064400    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0bbc8570-4947-455a-b7f8-8e9fa8e023b3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z2dd4\" (UID: \"0bbc8570-4947-455a-b7f8-8e9fa8e023b3\") " pod="openshift-insights/insights-runtime-extractor-z2dd4"
Oct 30 21:28:10.064566 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:10.064459    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08498890-fb63-4aea-9cf7-a2ade5397df2-metrics-tls\") pod \"dns-default-sbhch\" (UID: \"08498890-fb63-4aea-9cf7-a2ade5397df2\") " pod="openshift-dns/dns-default-sbhch"
Oct 30 21:28:10.064566 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:10.064491    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/352b5b4c-c640-4300-abf3-aa70b08cbb4b-metrics-certs\") pod \"network-metrics-daemon-vwxrd\" (UID: \"352b5b4c-c640-4300-abf3-aa70b08cbb4b\") " pod="openshift-multus/network-metrics-daemon-vwxrd"
Oct 30 21:28:10.064566 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:10.064511    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b4f9524-db75-4206-aa76-04400d95706f-service-ca-bundle\") pod \"router-default-5bbfd9465-x28lh\" (UID: \"9b4f9524-db75-4206-aa76-04400d95706f\") " pod="openshift-ingress/router-default-5bbfd9465-x28lh"
Oct 30 21:28:10.064764 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:28:10.064641    5032 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9b4f9524-db75-4206-aa76-04400d95706f-service-ca-bundle podName:9b4f9524-db75-4206-aa76-04400d95706f nodeName:}" failed. No retries permitted until 2026-10-30 21:28:12.064626645 +0000 UTC m=+4.866439101 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9b4f9524-db75-4206-aa76-04400d95706f-service-ca-bundle") pod "router-default-5bbfd9465-x28lh" (UID: "9b4f9524-db75-4206-aa76-04400d95706f") : configmap references non-existent config key: service-ca.crt
Oct 30 21:28:10.065056 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:28:10.065037    5032 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 30 21:28:10.065142 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:28:10.065095    5032 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/352b5b4c-c640-4300-abf3-aa70b08cbb4b-metrics-certs podName:352b5b4c-c640-4300-abf3-aa70b08cbb4b nodeName:}" failed. No retries permitted until 2026-10-30 21:28:12.065078798 +0000 UTC m=+4.866891259 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/352b5b4c-c640-4300-abf3-aa70b08cbb4b-metrics-certs") pod "network-metrics-daemon-vwxrd" (UID: "352b5b4c-c640-4300-abf3-aa70b08cbb4b") : secret "metrics-daemon-secret" not found
Oct 30 21:28:10.065199 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:28:10.065165    5032 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 21:28:10.065250 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:28:10.065204    5032 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/753d4639-09e6-4850-9f4d-ed39b0406439-networking-console-plugin-cert podName:753d4639-09e6-4850-9f4d-ed39b0406439 nodeName:}" failed. No retries permitted until 2026-10-30 21:28:12.065192865 +0000 UTC m=+4.867005321 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/753d4639-09e6-4850-9f4d-ed39b0406439-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-lwm89" (UID: "753d4639-09e6-4850-9f4d-ed39b0406439") : secret "networking-console-plugin-cert" not found
Oct 30 21:28:10.065309 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:28:10.065264    5032 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 21:28:10.065309 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:28:10.065298    5032 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0bbc8570-4947-455a-b7f8-8e9fa8e023b3-insights-runtime-extractor-tls podName:0bbc8570-4947-455a-b7f8-8e9fa8e023b3 nodeName:}" failed. No retries permitted until 2026-10-30 21:28:12.065287531 +0000 UTC m=+4.867099988 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0bbc8570-4947-455a-b7f8-8e9fa8e023b3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-z2dd4" (UID: "0bbc8570-4947-455a-b7f8-8e9fa8e023b3") : secret "insights-runtime-extractor-tls" not found
Oct 30 21:28:10.067326 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:10.067303    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08498890-fb63-4aea-9cf7-a2ade5397df2-metrics-tls\") pod \"dns-default-sbhch\" (UID: \"08498890-fb63-4aea-9cf7-a2ade5397df2\") " pod="openshift-dns/dns-default-sbhch"
Oct 30 21:28:10.067485 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:10.067462    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/90937bc9-0051-407d-9c82-014f0a9c65d4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-kdrrd\" (UID: \"90937bc9-0051-407d-9c82-014f0a9c65d4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-kdrrd"
Oct 30 21:28:10.113794 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:10.113757    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-kdrrd"
Oct 30 21:28:10.128170 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:10.128143    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-sbhch"
Oct 30 21:28:10.403147 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:10.403067    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-kdrrd"]
Oct 30 21:28:10.406603 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:10.406564    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90937bc9_0051_407d_9c82_014f0a9c65d4.slice/crio-013f0a6c6aedc2af1437a06c1de6f17d0d6f59bf46938f36cadaa157bfd00731 WatchSource:0}: Error finding container 013f0a6c6aedc2af1437a06c1de6f17d0d6f59bf46938f36cadaa157bfd00731: Status 404 returned error can't find the container with id 013f0a6c6aedc2af1437a06c1de6f17d0d6f59bf46938f36cadaa157bfd00731
Oct 30 21:28:10.408147 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:10.408120    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-sbhch"]
Oct 30 21:28:10.411424 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:10.411401    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08498890_fb63_4aea_9cf7_a2ade5397df2.slice/crio-ec11d6d98f5bb44d2eedf2b21e54213a41f9535c04a6d3a6f77ab131147547d5 WatchSource:0}: Error finding container ec11d6d98f5bb44d2eedf2b21e54213a41f9535c04a6d3a6f77ab131147547d5: Status 404 returned error can't find the container with id ec11d6d98f5bb44d2eedf2b21e54213a41f9535c04a6d3a6f77ab131147547d5
Oct 30 21:28:10.420349 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:10.420318    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-sbhch" event={"ID":"08498890-fb63-4aea-9cf7-a2ade5397df2","Type":"ContainerStarted","Data":"ec11d6d98f5bb44d2eedf2b21e54213a41f9535c04a6d3a6f77ab131147547d5"}
Oct 30 21:28:10.421666 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:10.421646    5032 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 21:28:10.421759 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:10.421646    5032 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 21:28:10.421759 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:10.421632    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-kdrrd" event={"ID":"90937bc9-0051-407d-9c82-014f0a9c65d4","Type":"ContainerStarted","Data":"013f0a6c6aedc2af1437a06c1de6f17d0d6f59bf46938f36cadaa157bfd00731"}
Oct 30 21:28:10.421844 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:10.421650    5032 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 21:28:10.909750 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:10.909585    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-sklxd"
Oct 30 21:28:10.919571 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:10.919538    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-sklxd"
Oct 30 21:28:11.012575 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:11.012533    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-76679b6db7-ltmt6"
Oct 30 21:28:11.262078 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:11.262046    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:28:11.262275 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:11.262260    5032 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 21:28:11.428253 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:11.428210    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-9qppl" event={"ID":"9f54b2c6-b1f9-42da-abdc-275fc8827b71","Type":"ContainerStarted","Data":"56e8f3f720a20b09b23c6e36adef7bada74c673c76451153ad33660817cbc54d"}
Oct 30 21:28:11.428765 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:11.428282    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-9qppl" event={"ID":"9f54b2c6-b1f9-42da-abdc-275fc8827b71","Type":"ContainerStarted","Data":"9290474ca3977024a23ae846728deeb3f67c7d207d4dea3003d71e7e73f3a696"}
Oct 30 21:28:12.085066 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:12.084982    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/352b5b4c-c640-4300-abf3-aa70b08cbb4b-metrics-certs\") pod \"network-metrics-daemon-vwxrd\" (UID: \"352b5b4c-c640-4300-abf3-aa70b08cbb4b\") " pod="openshift-multus/network-metrics-daemon-vwxrd"
Oct 30 21:28:12.085066 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:12.085033    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b4f9524-db75-4206-aa76-04400d95706f-service-ca-bundle\") pod \"router-default-5bbfd9465-x28lh\" (UID: \"9b4f9524-db75-4206-aa76-04400d95706f\") " pod="openshift-ingress/router-default-5bbfd9465-x28lh"
Oct 30 21:28:12.085268 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:12.085113    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/753d4639-09e6-4850-9f4d-ed39b0406439-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-lwm89\" (UID: \"753d4639-09e6-4850-9f4d-ed39b0406439\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-lwm89"
Oct 30 21:28:12.085268 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:28:12.085150    5032 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 30 21:28:12.085268 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:12.085154    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0bbc8570-4947-455a-b7f8-8e9fa8e023b3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z2dd4\" (UID: \"0bbc8570-4947-455a-b7f8-8e9fa8e023b3\") " pod="openshift-insights/insights-runtime-extractor-z2dd4"
Oct 30 21:28:12.085268 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:28:12.085223    5032 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9b4f9524-db75-4206-aa76-04400d95706f-service-ca-bundle podName:9b4f9524-db75-4206-aa76-04400d95706f nodeName:}" failed. No retries permitted until 2026-10-30 21:28:16.085203364 +0000 UTC m=+8.887015819 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9b4f9524-db75-4206-aa76-04400d95706f-service-ca-bundle") pod "router-default-5bbfd9465-x28lh" (UID: "9b4f9524-db75-4206-aa76-04400d95706f") : configmap references non-existent config key: service-ca.crt
Oct 30 21:28:12.085268 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:28:12.085230    5032 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 21:28:12.085457 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:28:12.085292    5032 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/352b5b4c-c640-4300-abf3-aa70b08cbb4b-metrics-certs podName:352b5b4c-c640-4300-abf3-aa70b08cbb4b nodeName:}" failed. No retries permitted until 2026-10-30 21:28:16.085276169 +0000 UTC m=+8.887088613 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/352b5b4c-c640-4300-abf3-aa70b08cbb4b-metrics-certs") pod "network-metrics-daemon-vwxrd" (UID: "352b5b4c-c640-4300-abf3-aa70b08cbb4b") : secret "metrics-daemon-secret" not found
Oct 30 21:28:12.085457 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:28:12.085311    5032 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0bbc8570-4947-455a-b7f8-8e9fa8e023b3-insights-runtime-extractor-tls podName:0bbc8570-4947-455a-b7f8-8e9fa8e023b3 nodeName:}" failed. No retries permitted until 2026-10-30 21:28:16.085301602 +0000 UTC m=+8.887114045 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0bbc8570-4947-455a-b7f8-8e9fa8e023b3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-z2dd4" (UID: "0bbc8570-4947-455a-b7f8-8e9fa8e023b3") : secret "insights-runtime-extractor-tls" not found
Oct 30 21:28:12.087741 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:12.087714    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/753d4639-09e6-4850-9f4d-ed39b0406439-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-lwm89\" (UID: \"753d4639-09e6-4850-9f4d-ed39b0406439\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-lwm89"
Oct 30 21:28:12.217645 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:12.217618    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-lwm89"
Oct 30 21:28:12.433862 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:12.433824    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-kdrrd" event={"ID":"90937bc9-0051-407d-9c82-014f0a9c65d4","Type":"ContainerStarted","Data":"9946fae410b0ef20a1e045755549d02d0cd4128acc855add2fd28648dcdd02d3"}
Oct 30 21:28:12.435543 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:12.435494    5032 generic.go:356] "Generic (PLEG): container finished" podID="d6ca21b9-20a4-4887-9d08-19d9c491a09e" containerID="6c092a58968d74d97f4617cfb9ba42b31f18f329717a98a66041a041254a53f1" exitCode=0
Oct 30 21:28:12.435660 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:12.435563    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-7cgk9" event={"ID":"d6ca21b9-20a4-4887-9d08-19d9c491a09e","Type":"ContainerDied","Data":"6c092a58968d74d97f4617cfb9ba42b31f18f329717a98a66041a041254a53f1"}
Oct 30 21:28:12.435932 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:12.435915    5032 scope.go:122] "RemoveContainer" containerID="6c092a58968d74d97f4617cfb9ba42b31f18f329717a98a66041a041254a53f1"
Oct 30 21:28:12.437767 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:12.437749    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-sbhch" event={"ID":"08498890-fb63-4aea-9cf7-a2ade5397df2","Type":"ContainerStarted","Data":"9b0772b6aede160622eed5801ebd43902eceb5857c43502fbb981cc209355384"}
Oct 30 21:28:12.437853 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:12.437794    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-sbhch" event={"ID":"08498890-fb63-4aea-9cf7-a2ade5397df2","Type":"ContainerStarted","Data":"08e9bb0f0b2afaa3a500a318ba4375b6a0b5e3682b50d10fe91b06e1c3861906"}
Oct 30 21:28:12.518121 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:12.518086    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-lwm89"]
Oct 30 21:28:12.521194 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:12.521166    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod753d4639_09e6_4850_9f4d_ed39b0406439.slice/crio-ca0ce95ad9154abe44a0fd7a4ffa3f0d4ab8302d234fb766ec7241bdeeb66b8d WatchSource:0}: Error finding container ca0ce95ad9154abe44a0fd7a4ffa3f0d4ab8302d234fb766ec7241bdeeb66b8d: Status 404 returned error can't find the container with id ca0ce95ad9154abe44a0fd7a4ffa3f0d4ab8302d234fb766ec7241bdeeb66b8d
Oct 30 21:28:13.446572 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:13.446525    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-lwm89" event={"ID":"753d4639-09e6-4850-9f4d-ed39b0406439","Type":"ContainerStarted","Data":"ca0ce95ad9154abe44a0fd7a4ffa3f0d4ab8302d234fb766ec7241bdeeb66b8d"}
Oct 30 21:28:13.448768 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:13.448737    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-7cgk9" event={"ID":"d6ca21b9-20a4-4887-9d08-19d9c491a09e","Type":"ContainerStarted","Data":"fbda5bef9663b6b4a18ab025f4264eaa7d3655322fefbee3a31a3fbe2302469e"}
Oct 30 21:28:13.449465 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:13.449085    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-sbhch"
Oct 30 21:28:14.454425 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:14.454385    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-lwm89" event={"ID":"753d4639-09e6-4850-9f4d-ed39b0406439","Type":"ContainerStarted","Data":"5e81cbd0a3050c5a05a0c2d90fae184ba9be5ff1eb208d7cf704925b5a2c8ea7"}
Oct 30 21:28:15.753670 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:15.753631    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gl47s"
Oct 30 21:28:16.125444 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:16.125347    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/352b5b4c-c640-4300-abf3-aa70b08cbb4b-metrics-certs\") pod \"network-metrics-daemon-vwxrd\" (UID: \"352b5b4c-c640-4300-abf3-aa70b08cbb4b\") " pod="openshift-multus/network-metrics-daemon-vwxrd"
Oct 30 21:28:16.125444 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:16.125410    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b4f9524-db75-4206-aa76-04400d95706f-service-ca-bundle\") pod \"router-default-5bbfd9465-x28lh\" (UID: \"9b4f9524-db75-4206-aa76-04400d95706f\") " pod="openshift-ingress/router-default-5bbfd9465-x28lh"
Oct 30 21:28:16.125624 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:16.125479    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0bbc8570-4947-455a-b7f8-8e9fa8e023b3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z2dd4\" (UID: \"0bbc8570-4947-455a-b7f8-8e9fa8e023b3\") " pod="openshift-insights/insights-runtime-extractor-z2dd4"
Oct 30 21:28:16.126054 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:16.126025    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b4f9524-db75-4206-aa76-04400d95706f-service-ca-bundle\") pod \"router-default-5bbfd9465-x28lh\" (UID: \"9b4f9524-db75-4206-aa76-04400d95706f\") " pod="openshift-ingress/router-default-5bbfd9465-x28lh"
Oct 30 21:28:16.127828 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:16.127805    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/352b5b4c-c640-4300-abf3-aa70b08cbb4b-metrics-certs\") pod \"network-metrics-daemon-vwxrd\" (UID: \"352b5b4c-c640-4300-abf3-aa70b08cbb4b\") " pod="openshift-multus/network-metrics-daemon-vwxrd"
Oct 30 21:28:16.128020 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:16.128003    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0bbc8570-4947-455a-b7f8-8e9fa8e023b3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z2dd4\" (UID: \"0bbc8570-4947-455a-b7f8-8e9fa8e023b3\") " pod="openshift-insights/insights-runtime-extractor-z2dd4"
Oct 30 21:28:16.130867 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:16.130850    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-z2dd4"
Oct 30 21:28:16.417003 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:16.416913    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5bbfd9465-x28lh"
Oct 30 21:28:16.418494 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:16.418462    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-z2dd4"]
Oct 30 21:28:16.422735 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:16.422703    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bbc8570_4947_455a_b7f8_8e9fa8e023b3.slice/crio-be2c0ff9c83a8ae31b4b80ebc0d41a314c128ad3e3d3382027bcba4e91a1ad16 WatchSource:0}: Error finding container be2c0ff9c83a8ae31b4b80ebc0d41a314c128ad3e3d3382027bcba4e91a1ad16: Status 404 returned error can't find the container with id be2c0ff9c83a8ae31b4b80ebc0d41a314c128ad3e3d3382027bcba4e91a1ad16
Oct 30 21:28:16.424182 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:16.424160    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vwxrd"
Oct 30 21:28:16.468883 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:16.468841    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-z2dd4" event={"ID":"0bbc8570-4947-455a-b7f8-8e9fa8e023b3","Type":"ContainerStarted","Data":"be2c0ff9c83a8ae31b4b80ebc0d41a314c128ad3e3d3382027bcba4e91a1ad16"}
Oct 30 21:28:16.706916 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:16.706888    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vwxrd"]
Oct 30 21:28:16.710309 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:16.710282    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5bbfd9465-x28lh"]
Oct 30 21:28:16.710437 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:16.710325    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod352b5b4c_c640_4300_abf3_aa70b08cbb4b.slice/crio-a016d34f775ec355c33b45329586687c6b1cad6f28548cff40fddb59f4b70f78 WatchSource:0}: Error finding container a016d34f775ec355c33b45329586687c6b1cad6f28548cff40fddb59f4b70f78: Status 404 returned error can't find the container with id a016d34f775ec355c33b45329586687c6b1cad6f28548cff40fddb59f4b70f78
Oct 30 21:28:16.713564 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:16.713545    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b4f9524_db75_4206_aa76_04400d95706f.slice/crio-88e4a4e9d92d5ef565d6904e73015126ef8e07328d1e2fc0a4a59320bc3c18ba WatchSource:0}: Error finding container 88e4a4e9d92d5ef565d6904e73015126ef8e07328d1e2fc0a4a59320bc3c18ba: Status 404 returned error can't find the container with id 88e4a4e9d92d5ef565d6904e73015126ef8e07328d1e2fc0a4a59320bc3c18ba
Oct 30 21:28:16.784722 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:16.784673    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-86f87fb58b-5clbv"
Oct 30 21:28:17.474075 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:17.473979    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vwxrd" event={"ID":"352b5b4c-c640-4300-abf3-aa70b08cbb4b","Type":"ContainerStarted","Data":"a016d34f775ec355c33b45329586687c6b1cad6f28548cff40fddb59f4b70f78"}
Oct 30 21:28:17.476176 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:17.476143    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-z2dd4" event={"ID":"0bbc8570-4947-455a-b7f8-8e9fa8e023b3","Type":"ContainerStarted","Data":"25f8dd70590518efa1f4417c2aeb09c1ae126dc49a2288f35e305328024e655a"}
Oct 30 21:28:17.476313 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:17.476193    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-z2dd4" event={"ID":"0bbc8570-4947-455a-b7f8-8e9fa8e023b3","Type":"ContainerStarted","Data":"b84a1dd9e02abb52cfafd15afb24646bd0a39f1ff0884dd150e3770fb8e0771c"}
Oct 30 21:28:17.478423 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:17.477925    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5bbfd9465-x28lh" event={"ID":"9b4f9524-db75-4206-aa76-04400d95706f","Type":"ContainerStarted","Data":"9c63863e9ba6487076f40e3d3222ddb28a0ff28f2cc6b42fc343cbdf90ebd4fb"}
Oct 30 21:28:17.478423 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:17.477966    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5bbfd9465-x28lh" event={"ID":"9b4f9524-db75-4206-aa76-04400d95706f","Type":"ContainerStarted","Data":"88e4a4e9d92d5ef565d6904e73015126ef8e07328d1e2fc0a4a59320bc3c18ba"}
Oct 30 21:28:18.219226 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:18.219189    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-4cvlt"
Oct 30 21:28:18.418054 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:18.418017    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-5bbfd9465-x28lh"
Oct 30 21:28:18.421065 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:18.421041    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-5bbfd9465-x28lh"
Oct 30 21:28:18.485484 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:18.485391    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vwxrd" event={"ID":"352b5b4c-c640-4300-abf3-aa70b08cbb4b","Type":"ContainerStarted","Data":"bd7a09adb7cbf8d04d7ac46b7a491e3c1ef3e737033071b30c4f2f7cd7154ea1"}
Oct 30 21:28:18.485484 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:18.485460    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vwxrd" event={"ID":"352b5b4c-c640-4300-abf3-aa70b08cbb4b","Type":"ContainerStarted","Data":"8a595f2a8dd2dfcdaab7dc5b47cb4b4417c90d180e2fdc26d903dd99d476a2b4"}
Oct 30 21:28:19.492618 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:19.492580    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-z2dd4" event={"ID":"0bbc8570-4947-455a-b7f8-8e9fa8e023b3","Type":"ContainerStarted","Data":"ed900e06a259beaa4a8c931e25ad4b70fc90fd19761adada010af8adc5368193"}
Oct 30 21:28:19.493158 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:19.492670    5032 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 21:28:22.371851 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:22.371811    5032 patch_prober.go:29] interesting pod/image-registry-86f87fb58b-5clbv container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Oct 30 21:28:22.372224 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:22.371870    5032 prober.go:122] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-86f87fb58b-5clbv" podUID="94ee3757-5440-42e1-8617-2a68744ef3f4" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:28:23.184595 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:23.184559    5032 patch_prober.go:29] interesting pod/image-registry-76679b6db7-ltmt6 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Oct 30 21:28:23.184759 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:23.184616    5032 prober.go:122] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-76679b6db7-ltmt6" podUID="039cf439-d12d-450a-80c1-eb3efeb50e8a" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:28:23.457199 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:23.457168    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-sbhch"
Oct 30 21:28:23.861322 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:23.861239    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-5bbfd9465-x28lh"
Oct 30 21:28:23.861488 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:23.861426    5032 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 21:28:23.862725 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:23.862701    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-5bbfd9465-x28lh"
Oct 30 21:28:25.300623 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.300592    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-nssh6-9f5sq"]
Oct 30 21:28:25.304102 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.304079    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-nssh6-9f5sq"
Oct 30 21:28:25.329784 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.329749    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-kgc78"]
Oct 30 21:28:25.333096 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.333070    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-nssh6-9f5sq"]
Oct 30 21:28:25.333265 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.333183    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-kgc78"
Oct 30 21:28:25.335423 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.335396    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-86f87fb58b-5clbv"]
Oct 30 21:28:25.343860 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.343829    5032 patch_prober.go:29] interesting pod/image-registry-86f87fb58b-5clbv container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Oct 30 21:28:25.344028 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.343906    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-86f87fb58b-5clbv" podUID="94ee3757-5440-42e1-8617-2a68744ef3f4" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:28:25.352718 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.352692    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-lfrs4\""
Oct 30 21:28:25.352815 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.352740    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Oct 30 21:28:25.393224 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.393196    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-kgc78"]
Oct 30 21:28:25.396845 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.396807    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-6dd789c8b7-qk429"]
Oct 30 21:28:25.400409 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.400387    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6dd789c8b7-qk429"
Oct 30 21:28:25.400594 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.400573    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/2495cf48-596f-4ff0-afa7-4e5a9521d6ab-archives-path\") pod \"periodic-gathering-nssh6-9f5sq\" (UID: \"2495cf48-596f-4ff0-afa7-4e5a9521d6ab\") " pod="openshift-insights/periodic-gathering-nssh6-9f5sq"
Oct 30 21:28:25.400659 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.400605    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2495cf48-596f-4ff0-afa7-4e5a9521d6ab-trusted-ca-bundle\") pod \"periodic-gathering-nssh6-9f5sq\" (UID: \"2495cf48-596f-4ff0-afa7-4e5a9521d6ab\") " pod="openshift-insights/periodic-gathering-nssh6-9f5sq"
Oct 30 21:28:25.400714 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.400696    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkfsh\" (UniqueName: \"kubernetes.io/projected/2495cf48-596f-4ff0-afa7-4e5a9521d6ab-kube-api-access-hkfsh\") pod \"periodic-gathering-nssh6-9f5sq\" (UID: \"2495cf48-596f-4ff0-afa7-4e5a9521d6ab\") " pod="openshift-insights/periodic-gathering-nssh6-9f5sq"
Oct 30 21:28:25.400799 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.400784    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2495cf48-596f-4ff0-afa7-4e5a9521d6ab-service-ca-bundle\") pod \"periodic-gathering-nssh6-9f5sq\" (UID: \"2495cf48-596f-4ff0-afa7-4e5a9521d6ab\") " pod="openshift-insights/periodic-gathering-nssh6-9f5sq"
Oct 30 21:28:25.416101 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.416072    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Oct 30 21:28:25.416284 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.416248    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Oct 30 21:28:25.416284 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.416157    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6dd789c8b7-qk429"]
Oct 30 21:28:25.444961 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.444933    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"downloads-dockercfg-ctm79\""
Oct 30 21:28:25.452851 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.452823    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-848d9d9bc8-mjlrc"]
Oct 30 21:28:25.456039 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.456023    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-848d9d9bc8-mjlrc"
Oct 30 21:28:25.480730 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.480699    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-848d9d9bc8-mjlrc"]
Oct 30 21:28:25.501618 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.501583    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2495cf48-596f-4ff0-afa7-4e5a9521d6ab-service-ca-bundle\") pod \"periodic-gathering-nssh6-9f5sq\" (UID: \"2495cf48-596f-4ff0-afa7-4e5a9521d6ab\") " pod="openshift-insights/periodic-gathering-nssh6-9f5sq"
Oct 30 21:28:25.501754 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.501658    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/2495cf48-596f-4ff0-afa7-4e5a9521d6ab-archives-path\") pod \"periodic-gathering-nssh6-9f5sq\" (UID: \"2495cf48-596f-4ff0-afa7-4e5a9521d6ab\") " pod="openshift-insights/periodic-gathering-nssh6-9f5sq"
Oct 30 21:28:25.501754 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.501689    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2495cf48-596f-4ff0-afa7-4e5a9521d6ab-trusted-ca-bundle\") pod \"periodic-gathering-nssh6-9f5sq\" (UID: \"2495cf48-596f-4ff0-afa7-4e5a9521d6ab\") " pod="openshift-insights/periodic-gathering-nssh6-9f5sq"
Oct 30 21:28:25.501974 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.501862    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkfsh\" (UniqueName: \"kubernetes.io/projected/2495cf48-596f-4ff0-afa7-4e5a9521d6ab-kube-api-access-hkfsh\") pod \"periodic-gathering-nssh6-9f5sq\" (UID: \"2495cf48-596f-4ff0-afa7-4e5a9521d6ab\") " pod="openshift-insights/periodic-gathering-nssh6-9f5sq"
Oct 30 21:28:25.501974 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.501900    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/40aace49-bddf-4a7f-aaaa-92fec5204b1c-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-kgc78\" (UID: \"40aace49-bddf-4a7f-aaaa-92fec5204b1c\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-kgc78"
Oct 30 21:28:25.501974 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.501937    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqwvr\" (UniqueName: \"kubernetes.io/projected/ea11393a-a054-4a6d-9194-156a7adc6c7e-kube-api-access-rqwvr\") pod \"downloads-6dd789c8b7-qk429\" (UID: \"ea11393a-a054-4a6d-9194-156a7adc6c7e\") " pod="openshift-console/downloads-6dd789c8b7-qk429"
Oct 30 21:28:25.502233 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.502003    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ea11393a-a054-4a6d-9194-156a7adc6c7e-tmp\") pod \"downloads-6dd789c8b7-qk429\" (UID: \"ea11393a-a054-4a6d-9194-156a7adc6c7e\") " pod="openshift-console/downloads-6dd789c8b7-qk429"
Oct 30 21:28:25.502436 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.502412    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2495cf48-596f-4ff0-afa7-4e5a9521d6ab-service-ca-bundle\") pod \"periodic-gathering-nssh6-9f5sq\" (UID: \"2495cf48-596f-4ff0-afa7-4e5a9521d6ab\") " pod="openshift-insights/periodic-gathering-nssh6-9f5sq"
Oct 30 21:28:25.502577 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.502555    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/2495cf48-596f-4ff0-afa7-4e5a9521d6ab-archives-path\") pod \"periodic-gathering-nssh6-9f5sq\" (UID: \"2495cf48-596f-4ff0-afa7-4e5a9521d6ab\") " pod="openshift-insights/periodic-gathering-nssh6-9f5sq"
Oct 30 21:28:25.503388 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.503350    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2495cf48-596f-4ff0-afa7-4e5a9521d6ab-trusted-ca-bundle\") pod \"periodic-gathering-nssh6-9f5sq\" (UID: \"2495cf48-596f-4ff0-afa7-4e5a9521d6ab\") " pod="openshift-insights/periodic-gathering-nssh6-9f5sq"
Oct 30 21:28:25.531023 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.530984    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkfsh\" (UniqueName: \"kubernetes.io/projected/2495cf48-596f-4ff0-afa7-4e5a9521d6ab-kube-api-access-hkfsh\") pod \"periodic-gathering-nssh6-9f5sq\" (UID: \"2495cf48-596f-4ff0-afa7-4e5a9521d6ab\") " pod="openshift-insights/periodic-gathering-nssh6-9f5sq"
Oct 30 21:28:25.603447 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.603345    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cf2eedc0-044e-46d4-ab33-71c5b6d211b8-bound-sa-token\") pod \"image-registry-848d9d9bc8-mjlrc\" (UID: \"cf2eedc0-044e-46d4-ab33-71c5b6d211b8\") " pod="openshift-image-registry/image-registry-848d9d9bc8-mjlrc"
Oct 30 21:28:25.603447 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.603411    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cf2eedc0-044e-46d4-ab33-71c5b6d211b8-registry-tls\") pod \"image-registry-848d9d9bc8-mjlrc\" (UID: \"cf2eedc0-044e-46d4-ab33-71c5b6d211b8\") " pod="openshift-image-registry/image-registry-848d9d9bc8-mjlrc"
Oct 30 21:28:25.603629 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.603459    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cf2eedc0-044e-46d4-ab33-71c5b6d211b8-trusted-ca\") pod \"image-registry-848d9d9bc8-mjlrc\" (UID: \"cf2eedc0-044e-46d4-ab33-71c5b6d211b8\") " pod="openshift-image-registry/image-registry-848d9d9bc8-mjlrc"
Oct 30 21:28:25.603629 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.603544    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cf2eedc0-044e-46d4-ab33-71c5b6d211b8-installation-pull-secrets\") pod \"image-registry-848d9d9bc8-mjlrc\" (UID: \"cf2eedc0-044e-46d4-ab33-71c5b6d211b8\") " pod="openshift-image-registry/image-registry-848d9d9bc8-mjlrc"
Oct 30 21:28:25.603629 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.603587    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cf2eedc0-044e-46d4-ab33-71c5b6d211b8-registry-certificates\") pod \"image-registry-848d9d9bc8-mjlrc\" (UID: \"cf2eedc0-044e-46d4-ab33-71c5b6d211b8\") " pod="openshift-image-registry/image-registry-848d9d9bc8-mjlrc"
Oct 30 21:28:25.603629 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.603606    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cf2eedc0-044e-46d4-ab33-71c5b6d211b8-ca-trust-extracted\") pod \"image-registry-848d9d9bc8-mjlrc\" (UID: \"cf2eedc0-044e-46d4-ab33-71c5b6d211b8\") " pod="openshift-image-registry/image-registry-848d9d9bc8-mjlrc"
Oct 30 21:28:25.603750 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.603682    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/40aace49-bddf-4a7f-aaaa-92fec5204b1c-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-kgc78\" (UID: \"40aace49-bddf-4a7f-aaaa-92fec5204b1c\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-kgc78"
Oct 30 21:28:25.603750 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.603713    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqwvr\" (UniqueName: \"kubernetes.io/projected/ea11393a-a054-4a6d-9194-156a7adc6c7e-kube-api-access-rqwvr\") pod \"downloads-6dd789c8b7-qk429\" (UID: \"ea11393a-a054-4a6d-9194-156a7adc6c7e\") " pod="openshift-console/downloads-6dd789c8b7-qk429"
Oct 30 21:28:25.603750 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.603737    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ea11393a-a054-4a6d-9194-156a7adc6c7e-tmp\") pod \"downloads-6dd789c8b7-qk429\" (UID: \"ea11393a-a054-4a6d-9194-156a7adc6c7e\") " pod="openshift-console/downloads-6dd789c8b7-qk429"
Oct 30 21:28:25.603874 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.603762    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cf2eedc0-044e-46d4-ab33-71c5b6d211b8-image-registry-private-configuration\") pod \"image-registry-848d9d9bc8-mjlrc\" (UID: \"cf2eedc0-044e-46d4-ab33-71c5b6d211b8\") " pod="openshift-image-registry/image-registry-848d9d9bc8-mjlrc"
Oct 30 21:28:25.603874 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.603797    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q94b7\" (UniqueName: \"kubernetes.io/projected/cf2eedc0-044e-46d4-ab33-71c5b6d211b8-kube-api-access-q94b7\") pod \"image-registry-848d9d9bc8-mjlrc\" (UID: \"cf2eedc0-044e-46d4-ab33-71c5b6d211b8\") " pod="openshift-image-registry/image-registry-848d9d9bc8-mjlrc"
Oct 30 21:28:25.604115 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.604097    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ea11393a-a054-4a6d-9194-156a7adc6c7e-tmp\") pod \"downloads-6dd789c8b7-qk429\" (UID: \"ea11393a-a054-4a6d-9194-156a7adc6c7e\") " pod="openshift-console/downloads-6dd789c8b7-qk429"
Oct 30 21:28:25.606067 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.606050    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/40aace49-bddf-4a7f-aaaa-92fec5204b1c-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-kgc78\" (UID: \"40aace49-bddf-4a7f-aaaa-92fec5204b1c\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-kgc78"
Oct 30 21:28:25.617239 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.617215    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-nssh6-9f5sq"
Oct 30 21:28:25.620781 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.620757    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqwvr\" (UniqueName: \"kubernetes.io/projected/ea11393a-a054-4a6d-9194-156a7adc6c7e-kube-api-access-rqwvr\") pod \"downloads-6dd789c8b7-qk429\" (UID: \"ea11393a-a054-4a6d-9194-156a7adc6c7e\") " pod="openshift-console/downloads-6dd789c8b7-qk429"
Oct 30 21:28:25.656186 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.656153    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-kgc78"
Oct 30 21:28:25.704718 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.704684    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cf2eedc0-044e-46d4-ab33-71c5b6d211b8-image-registry-private-configuration\") pod \"image-registry-848d9d9bc8-mjlrc\" (UID: \"cf2eedc0-044e-46d4-ab33-71c5b6d211b8\") " pod="openshift-image-registry/image-registry-848d9d9bc8-mjlrc"
Oct 30 21:28:25.704880 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.704723    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q94b7\" (UniqueName: \"kubernetes.io/projected/cf2eedc0-044e-46d4-ab33-71c5b6d211b8-kube-api-access-q94b7\") pod \"image-registry-848d9d9bc8-mjlrc\" (UID: \"cf2eedc0-044e-46d4-ab33-71c5b6d211b8\") " pod="openshift-image-registry/image-registry-848d9d9bc8-mjlrc"
Oct 30 21:28:25.704880 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.704776    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cf2eedc0-044e-46d4-ab33-71c5b6d211b8-bound-sa-token\") pod \"image-registry-848d9d9bc8-mjlrc\" (UID: \"cf2eedc0-044e-46d4-ab33-71c5b6d211b8\") " pod="openshift-image-registry/image-registry-848d9d9bc8-mjlrc"
Oct 30 21:28:25.704880 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.704803    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cf2eedc0-044e-46d4-ab33-71c5b6d211b8-registry-tls\") pod \"image-registry-848d9d9bc8-mjlrc\" (UID: \"cf2eedc0-044e-46d4-ab33-71c5b6d211b8\") " pod="openshift-image-registry/image-registry-848d9d9bc8-mjlrc"
Oct 30 21:28:25.704880 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.704825    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cf2eedc0-044e-46d4-ab33-71c5b6d211b8-trusted-ca\") pod \"image-registry-848d9d9bc8-mjlrc\" (UID: \"cf2eedc0-044e-46d4-ab33-71c5b6d211b8\") " pod="openshift-image-registry/image-registry-848d9d9bc8-mjlrc"
Oct 30 21:28:25.704880 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.704856    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cf2eedc0-044e-46d4-ab33-71c5b6d211b8-installation-pull-secrets\") pod \"image-registry-848d9d9bc8-mjlrc\" (UID: \"cf2eedc0-044e-46d4-ab33-71c5b6d211b8\") " pod="openshift-image-registry/image-registry-848d9d9bc8-mjlrc"
Oct 30 21:28:25.705142 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.704897    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cf2eedc0-044e-46d4-ab33-71c5b6d211b8-registry-certificates\") pod \"image-registry-848d9d9bc8-mjlrc\" (UID: \"cf2eedc0-044e-46d4-ab33-71c5b6d211b8\") " pod="openshift-image-registry/image-registry-848d9d9bc8-mjlrc"
Oct 30 21:28:25.705205 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.705180    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cf2eedc0-044e-46d4-ab33-71c5b6d211b8-ca-trust-extracted\") pod \"image-registry-848d9d9bc8-mjlrc\" (UID: \"cf2eedc0-044e-46d4-ab33-71c5b6d211b8\") " pod="openshift-image-registry/image-registry-848d9d9bc8-mjlrc"
Oct 30 21:28:25.705831 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.705631    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cf2eedc0-044e-46d4-ab33-71c5b6d211b8-ca-trust-extracted\") pod \"image-registry-848d9d9bc8-mjlrc\" (UID: \"cf2eedc0-044e-46d4-ab33-71c5b6d211b8\") " pod="openshift-image-registry/image-registry-848d9d9bc8-mjlrc"
Oct 30 21:28:25.705965 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.705870    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cf2eedc0-044e-46d4-ab33-71c5b6d211b8-registry-certificates\") pod \"image-registry-848d9d9bc8-mjlrc\" (UID: \"cf2eedc0-044e-46d4-ab33-71c5b6d211b8\") " pod="openshift-image-registry/image-registry-848d9d9bc8-mjlrc"
Oct 30 21:28:25.707460 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.705990    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cf2eedc0-044e-46d4-ab33-71c5b6d211b8-trusted-ca\") pod \"image-registry-848d9d9bc8-mjlrc\" (UID: \"cf2eedc0-044e-46d4-ab33-71c5b6d211b8\") " pod="openshift-image-registry/image-registry-848d9d9bc8-mjlrc"
Oct 30 21:28:25.708016 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.707988    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cf2eedc0-044e-46d4-ab33-71c5b6d211b8-image-registry-private-configuration\") pod \"image-registry-848d9d9bc8-mjlrc\" (UID: \"cf2eedc0-044e-46d4-ab33-71c5b6d211b8\") " pod="openshift-image-registry/image-registry-848d9d9bc8-mjlrc"
Oct 30 21:28:25.709883 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.709859    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cf2eedc0-044e-46d4-ab33-71c5b6d211b8-registry-tls\") pod \"image-registry-848d9d9bc8-mjlrc\" (UID: \"cf2eedc0-044e-46d4-ab33-71c5b6d211b8\") " pod="openshift-image-registry/image-registry-848d9d9bc8-mjlrc"
Oct 30 21:28:25.710767 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.710746    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cf2eedc0-044e-46d4-ab33-71c5b6d211b8-installation-pull-secrets\") pod \"image-registry-848d9d9bc8-mjlrc\" (UID: \"cf2eedc0-044e-46d4-ab33-71c5b6d211b8\") " pod="openshift-image-registry/image-registry-848d9d9bc8-mjlrc"
Oct 30 21:28:25.713817 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.713649    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6dd789c8b7-qk429"
Oct 30 21:28:25.722449 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.722419    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cf2eedc0-044e-46d4-ab33-71c5b6d211b8-bound-sa-token\") pod \"image-registry-848d9d9bc8-mjlrc\" (UID: \"cf2eedc0-044e-46d4-ab33-71c5b6d211b8\") " pod="openshift-image-registry/image-registry-848d9d9bc8-mjlrc"
Oct 30 21:28:25.722734 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.722714    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q94b7\" (UniqueName: \"kubernetes.io/projected/cf2eedc0-044e-46d4-ab33-71c5b6d211b8-kube-api-access-q94b7\") pod \"image-registry-848d9d9bc8-mjlrc\" (UID: \"cf2eedc0-044e-46d4-ab33-71c5b6d211b8\") " pod="openshift-image-registry/image-registry-848d9d9bc8-mjlrc"
Oct 30 21:28:25.773076 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.772811    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-848d9d9bc8-mjlrc"
Oct 30 21:28:25.894033 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.893964    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-nssh6-9f5sq"]
Oct 30 21:28:25.896728 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:25.896700    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2495cf48_596f_4ff0_afa7_4e5a9521d6ab.slice/crio-167d6f9e13b104e2a5c5a4ca9b52839bdbd8f737aef9fad3ddba99ac7c189801 WatchSource:0}: Error finding container 167d6f9e13b104e2a5c5a4ca9b52839bdbd8f737aef9fad3ddba99ac7c189801: Status 404 returned error can't find the container with id 167d6f9e13b104e2a5c5a4ca9b52839bdbd8f737aef9fad3ddba99ac7c189801
Oct 30 21:28:25.947409 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.947353    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-kgc78"]
Oct 30 21:28:25.950376 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:25.950330    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40aace49_bddf_4a7f_aaaa_92fec5204b1c.slice/crio-285b00049c841ceb15a75a9c633009db7e237026ca8c1e5d36b01b09195e779f WatchSource:0}: Error finding container 285b00049c841ceb15a75a9c633009db7e237026ca8c1e5d36b01b09195e779f: Status 404 returned error can't find the container with id 285b00049c841ceb15a75a9c633009db7e237026ca8c1e5d36b01b09195e779f
Oct 30 21:28:25.998263 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:25.998229    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6dd789c8b7-qk429"]
Oct 30 21:28:26.002269 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:26.002241    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea11393a_a054_4a6d_9194_156a7adc6c7e.slice/crio-27e83ee5ddcb07ceb2146059b3c344272c76e961d3e00c104ac33dab63893672 WatchSource:0}: Error finding container 27e83ee5ddcb07ceb2146059b3c344272c76e961d3e00c104ac33dab63893672: Status 404 returned error can't find the container with id 27e83ee5ddcb07ceb2146059b3c344272c76e961d3e00c104ac33dab63893672
Oct 30 21:28:26.065508 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:26.065475    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-848d9d9bc8-mjlrc"]
Oct 30 21:28:26.068686 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:26.068659    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf2eedc0_044e_46d4_ab33_71c5b6d211b8.slice/crio-4b425c91ef5b3251e007945b0a7b43396fdc3637badccaaf5ff48559a685bd77 WatchSource:0}: Error finding container 4b425c91ef5b3251e007945b0a7b43396fdc3637badccaaf5ff48559a685bd77: Status 404 returned error can't find the container with id 4b425c91ef5b3251e007945b0a7b43396fdc3637badccaaf5ff48559a685bd77
Oct 30 21:28:26.524255 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:26.524212    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-nssh6-9f5sq" event={"ID":"2495cf48-596f-4ff0-afa7-4e5a9521d6ab","Type":"ContainerStarted","Data":"398bac5f4f91d3bd75182331b2a6af8040644db3b37d7453d641f3fd5e924ef5"}
Oct 30 21:28:26.524791 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:26.524284    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-nssh6-9f5sq" event={"ID":"2495cf48-596f-4ff0-afa7-4e5a9521d6ab","Type":"ContainerStarted","Data":"167d6f9e13b104e2a5c5a4ca9b52839bdbd8f737aef9fad3ddba99ac7c189801"}
Oct 30 21:28:26.525733 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:26.525695    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-kgc78" event={"ID":"40aace49-bddf-4a7f-aaaa-92fec5204b1c","Type":"ContainerStarted","Data":"285b00049c841ceb15a75a9c633009db7e237026ca8c1e5d36b01b09195e779f"}
Oct 30 21:28:26.527555 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:26.527531    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-848d9d9bc8-mjlrc" event={"ID":"cf2eedc0-044e-46d4-ab33-71c5b6d211b8","Type":"ContainerStarted","Data":"cb1e03e3a9ba34ad2d152e96a829d7b826e1b09194a6052d86392b07d74d31a4"}
Oct 30 21:28:26.527665 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:26.527575    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-848d9d9bc8-mjlrc" event={"ID":"cf2eedc0-044e-46d4-ab33-71c5b6d211b8","Type":"ContainerStarted","Data":"4b425c91ef5b3251e007945b0a7b43396fdc3637badccaaf5ff48559a685bd77"}
Oct 30 21:28:26.527665 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:26.527655    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-848d9d9bc8-mjlrc"
Oct 30 21:28:26.529059 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:26.529027    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6dd789c8b7-qk429" event={"ID":"ea11393a-a054-4a6d-9194-156a7adc6c7e","Type":"ContainerStarted","Data":"27e83ee5ddcb07ceb2146059b3c344272c76e961d3e00c104ac33dab63893672"}
Oct 30 21:28:26.545869 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:26.545813    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-nssh6-9f5sq" podStartSLOduration=1.5457929529999999 podStartE2EDuration="1.545792953s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:28:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:28:26.544671106 +0000 UTC m=+19.346483571" watchObservedRunningTime="2026-10-30 21:28:26.545792953 +0000 UTC m=+19.347605418"
Oct 30 21:28:27.536482 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:27.536437    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-kgc78" event={"ID":"40aace49-bddf-4a7f-aaaa-92fec5204b1c","Type":"ContainerStarted","Data":"9e38cf11f8f02e9f867b55556f3986c1831c8a38e9c604ef498a84f28a561348"}
Oct 30 21:28:27.536958 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:27.536869    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-kgc78"
Oct 30 21:28:27.542462 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:27.542432    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-kgc78"
Oct 30 21:28:27.558689 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:27.558614    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/image-registry-848d9d9bc8-mjlrc" podStartSLOduration=2.558592376 podStartE2EDuration="2.558592376s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:28:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:28:26.569868765 +0000 UTC m=+19.371681230" watchObservedRunningTime="2026-10-30 21:28:27.558592376 +0000 UTC m=+20.360404820"
Oct 30 21:28:27.584685 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:27.584621    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-kgc78" podStartSLOduration=1.5434538770000001 podStartE2EDuration="2.584603452s" totalImagesPullingTime="1.041149575s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:28:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:28:27.559017872 +0000 UTC m=+20.360830338" watchObservedRunningTime="2026-10-30 21:28:27.584603452 +0000 UTC m=+20.386415917"
Oct 30 21:28:30.273501 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:30.273462    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-sbhch_08498890-fb63-4aea-9cf7-a2ade5397df2/dns/0.log"
Oct 30 21:28:30.477373 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:30.477328    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-sbhch_08498890-fb63-4aea-9cf7-a2ade5397df2/kube-rbac-proxy/0.log"
Oct 30 21:28:30.674353 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:30.674280    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-97tsr_baa8d0c0-d477-4f30-a0f6-375fc84cc448/dns-node-resolver/0.log"
Oct 30 21:28:31.274029 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:31.273990    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-76679b6db7-ltmt6_039cf439-d12d-450a-80c1-eb3efeb50e8a/registry/0.log"
Oct 30 21:28:31.434163 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:31.434135    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-76679b6db7-ltmt6"
Oct 30 21:28:31.475480 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:31.475455    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-848d9d9bc8-mjlrc_cf2eedc0-044e-46d4-ab33-71c5b6d211b8/registry/0.log"
Oct 30 21:28:31.673387 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:31.673284    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-86f87fb58b-5clbv_94ee3757-5440-42e1-8617-2a68744ef3f4/registry/0.log"
Oct 30 21:28:32.256505 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:32.256467    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-ggbl6"]
Oct 30 21:28:32.284943 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:32.284916    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-jgns8_48efa9dc-3c20-4f81-86f3-c87ad697772c/node-ca/0.log"
Oct 30 21:28:32.293457 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:32.293432    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ggbl6"
Oct 30 21:28:32.328227 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:32.328201    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 30 21:28:32.328558 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:32.328538    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-9vdcb\""
Oct 30 21:28:32.328558 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:32.328538    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 30 21:28:32.333777 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:32.333749    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 30 21:28:32.333907 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:32.333790    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 30 21:28:32.469666 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:32.469626    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/88eb2d7e-b5dd-4769-b30c-756140a8cc1d-sys\") pod \"node-exporter-ggbl6\" (UID: \"88eb2d7e-b5dd-4769-b30c-756140a8cc1d\") " pod="openshift-monitoring/node-exporter-ggbl6"
Oct 30 21:28:32.469844 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:32.469732    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2wnj\" (UniqueName: \"kubernetes.io/projected/88eb2d7e-b5dd-4769-b30c-756140a8cc1d-kube-api-access-g2wnj\") pod \"node-exporter-ggbl6\" (UID: \"88eb2d7e-b5dd-4769-b30c-756140a8cc1d\") " pod="openshift-monitoring/node-exporter-ggbl6"
Oct 30 21:28:32.469844 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:32.469780    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/88eb2d7e-b5dd-4769-b30c-756140a8cc1d-node-exporter-accelerators-collector-config\") pod \"node-exporter-ggbl6\" (UID: \"88eb2d7e-b5dd-4769-b30c-756140a8cc1d\") " pod="openshift-monitoring/node-exporter-ggbl6"
Oct 30 21:28:32.469952 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:32.469862    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/88eb2d7e-b5dd-4769-b30c-756140a8cc1d-root\") pod \"node-exporter-ggbl6\" (UID: \"88eb2d7e-b5dd-4769-b30c-756140a8cc1d\") " pod="openshift-monitoring/node-exporter-ggbl6"
Oct 30 21:28:32.469952 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:32.469893    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/88eb2d7e-b5dd-4769-b30c-756140a8cc1d-node-exporter-wtmp\") pod \"node-exporter-ggbl6\" (UID: \"88eb2d7e-b5dd-4769-b30c-756140a8cc1d\") " pod="openshift-monitoring/node-exporter-ggbl6"
Oct 30 21:28:32.469952 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:32.469931    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/88eb2d7e-b5dd-4769-b30c-756140a8cc1d-node-exporter-tls\") pod \"node-exporter-ggbl6\" (UID: \"88eb2d7e-b5dd-4769-b30c-756140a8cc1d\") " pod="openshift-monitoring/node-exporter-ggbl6"
Oct 30 21:28:32.470210 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:32.469960    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/88eb2d7e-b5dd-4769-b30c-756140a8cc1d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ggbl6\" (UID: \"88eb2d7e-b5dd-4769-b30c-756140a8cc1d\") " pod="openshift-monitoring/node-exporter-ggbl6"
Oct 30 21:28:32.470289 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:32.470273    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/88eb2d7e-b5dd-4769-b30c-756140a8cc1d-metrics-client-ca\") pod \"node-exporter-ggbl6\" (UID: \"88eb2d7e-b5dd-4769-b30c-756140a8cc1d\") " pod="openshift-monitoring/node-exporter-ggbl6"
Oct 30 21:28:32.470349 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:32.470335    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/88eb2d7e-b5dd-4769-b30c-756140a8cc1d-node-exporter-textfile\") pod \"node-exporter-ggbl6\" (UID: \"88eb2d7e-b5dd-4769-b30c-756140a8cc1d\") " pod="openshift-monitoring/node-exporter-ggbl6"
Oct 30 21:28:32.475735 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:32.473189    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5bbfd9465-x28lh_9b4f9524-db75-4206-aa76-04400d95706f/router/0.log"
Oct 30 21:28:32.571547 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:32.571456    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/88eb2d7e-b5dd-4769-b30c-756140a8cc1d-node-exporter-textfile\") pod \"node-exporter-ggbl6\" (UID: \"88eb2d7e-b5dd-4769-b30c-756140a8cc1d\") " pod="openshift-monitoring/node-exporter-ggbl6"
Oct 30 21:28:32.571547 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:32.571532    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/88eb2d7e-b5dd-4769-b30c-756140a8cc1d-sys\") pod \"node-exporter-ggbl6\" (UID: \"88eb2d7e-b5dd-4769-b30c-756140a8cc1d\") " pod="openshift-monitoring/node-exporter-ggbl6"
Oct 30 21:28:32.571747 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:32.571559    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2wnj\" (UniqueName: \"kubernetes.io/projected/88eb2d7e-b5dd-4769-b30c-756140a8cc1d-kube-api-access-g2wnj\") pod \"node-exporter-ggbl6\" (UID: \"88eb2d7e-b5dd-4769-b30c-756140a8cc1d\") " pod="openshift-monitoring/node-exporter-ggbl6"
Oct 30 21:28:32.571747 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:32.571598    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/88eb2d7e-b5dd-4769-b30c-756140a8cc1d-node-exporter-accelerators-collector-config\") pod \"node-exporter-ggbl6\" (UID: \"88eb2d7e-b5dd-4769-b30c-756140a8cc1d\") " pod="openshift-monitoring/node-exporter-ggbl6"
Oct 30 21:28:32.571747 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:32.571673    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/88eb2d7e-b5dd-4769-b30c-756140a8cc1d-root\") pod \"node-exporter-ggbl6\" (UID: \"88eb2d7e-b5dd-4769-b30c-756140a8cc1d\") " pod="openshift-monitoring/node-exporter-ggbl6"
Oct 30 21:28:32.571747 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:32.571697    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/88eb2d7e-b5dd-4769-b30c-756140a8cc1d-node-exporter-wtmp\") pod \"node-exporter-ggbl6\" (UID: \"88eb2d7e-b5dd-4769-b30c-756140a8cc1d\") " pod="openshift-monitoring/node-exporter-ggbl6"
Oct 30 21:28:32.571747 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:32.571729    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/88eb2d7e-b5dd-4769-b30c-756140a8cc1d-node-exporter-tls\") pod \"node-exporter-ggbl6\" (UID: \"88eb2d7e-b5dd-4769-b30c-756140a8cc1d\") " pod="openshift-monitoring/node-exporter-ggbl6"
Oct 30 21:28:32.571985 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:32.571753    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/88eb2d7e-b5dd-4769-b30c-756140a8cc1d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ggbl6\" (UID: \"88eb2d7e-b5dd-4769-b30c-756140a8cc1d\") " pod="openshift-monitoring/node-exporter-ggbl6"
Oct 30 21:28:32.571985 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:32.571780    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/88eb2d7e-b5dd-4769-b30c-756140a8cc1d-metrics-client-ca\") pod \"node-exporter-ggbl6\" (UID: \"88eb2d7e-b5dd-4769-b30c-756140a8cc1d\") " pod="openshift-monitoring/node-exporter-ggbl6"
Oct 30 21:28:32.572554 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:32.572528    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/88eb2d7e-b5dd-4769-b30c-756140a8cc1d-metrics-client-ca\") pod \"node-exporter-ggbl6\" (UID: \"88eb2d7e-b5dd-4769-b30c-756140a8cc1d\") " pod="openshift-monitoring/node-exporter-ggbl6"
Oct 30 21:28:32.572819 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:32.572795    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/88eb2d7e-b5dd-4769-b30c-756140a8cc1d-node-exporter-textfile\") pod \"node-exporter-ggbl6\" (UID: \"88eb2d7e-b5dd-4769-b30c-756140a8cc1d\") " pod="openshift-monitoring/node-exporter-ggbl6"
Oct 30 21:28:32.572905 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:32.572867    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/88eb2d7e-b5dd-4769-b30c-756140a8cc1d-sys\") pod \"node-exporter-ggbl6\" (UID: \"88eb2d7e-b5dd-4769-b30c-756140a8cc1d\") " pod="openshift-monitoring/node-exporter-ggbl6"
Oct 30 21:28:32.573763 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:32.573730    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/88eb2d7e-b5dd-4769-b30c-756140a8cc1d-node-exporter-accelerators-collector-config\") pod \"node-exporter-ggbl6\" (UID: \"88eb2d7e-b5dd-4769-b30c-756140a8cc1d\") " pod="openshift-monitoring/node-exporter-ggbl6"
Oct 30 21:28:32.573850 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:32.573801    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/88eb2d7e-b5dd-4769-b30c-756140a8cc1d-root\") pod \"node-exporter-ggbl6\" (UID: \"88eb2d7e-b5dd-4769-b30c-756140a8cc1d\") " pod="openshift-monitoring/node-exporter-ggbl6"
Oct 30 21:28:32.573940 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:32.573920    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/88eb2d7e-b5dd-4769-b30c-756140a8cc1d-node-exporter-wtmp\") pod \"node-exporter-ggbl6\" (UID: \"88eb2d7e-b5dd-4769-b30c-756140a8cc1d\") " pod="openshift-monitoring/node-exporter-ggbl6"
Oct 30 21:28:32.577711 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:32.577513    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/88eb2d7e-b5dd-4769-b30c-756140a8cc1d-node-exporter-tls\") pod \"node-exporter-ggbl6\" (UID: \"88eb2d7e-b5dd-4769-b30c-756140a8cc1d\") " pod="openshift-monitoring/node-exporter-ggbl6"
Oct 30 21:28:32.577711 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:32.577556    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/88eb2d7e-b5dd-4769-b30c-756140a8cc1d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ggbl6\" (UID: \"88eb2d7e-b5dd-4769-b30c-756140a8cc1d\") " pod="openshift-monitoring/node-exporter-ggbl6"
Oct 30 21:28:32.588394 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:32.586585    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2wnj\" (UniqueName: \"kubernetes.io/projected/88eb2d7e-b5dd-4769-b30c-756140a8cc1d-kube-api-access-g2wnj\") pod \"node-exporter-ggbl6\" (UID: \"88eb2d7e-b5dd-4769-b30c-756140a8cc1d\") " pod="openshift-monitoring/node-exporter-ggbl6"
Oct 30 21:28:32.658048 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:32.658011    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ggbl6"
Oct 30 21:28:32.673475 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:32.673445    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hzvrj_3175ff47-1038-4139-9c0a-87062bb38dc2/serve-healthcheck-canary/0.log"
Oct 30 21:28:32.687914 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:28:32.687885    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88eb2d7e_b5dd_4769_b30c_756140a8cc1d.slice/crio-48bc354171ba048f53de6cab272d3087f212e11042b33e4745867981fdb99eda WatchSource:0}: Error finding container 48bc354171ba048f53de6cab272d3087f212e11042b33e4745867981fdb99eda: Status 404 returned error can't find the container with id 48bc354171ba048f53de6cab272d3087f212e11042b33e4745867981fdb99eda
Oct 30 21:28:33.570078 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:33.570043    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ggbl6" event={"ID":"88eb2d7e-b5dd-4769-b30c-756140a8cc1d","Type":"ContainerStarted","Data":"48bc354171ba048f53de6cab272d3087f212e11042b33e4745867981fdb99eda"}
Oct 30 21:28:34.577095 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:34.577008    5032 generic.go:356] "Generic (PLEG): container finished" podID="88eb2d7e-b5dd-4769-b30c-756140a8cc1d" containerID="38f1b1544a6a1ba8f7bef935018a4b2dbe649bca4bb8d7358535411149f10090" exitCode=0
Oct 30 21:28:34.577095 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:34.577073    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ggbl6" event={"ID":"88eb2d7e-b5dd-4769-b30c-756140a8cc1d","Type":"ContainerDied","Data":"38f1b1544a6a1ba8f7bef935018a4b2dbe649bca4bb8d7358535411149f10090"}
Oct 30 21:28:35.344567 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:35.344540    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-86f87fb58b-5clbv"
Oct 30 21:28:35.585004 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:35.584967    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ggbl6" event={"ID":"88eb2d7e-b5dd-4769-b30c-756140a8cc1d","Type":"ContainerStarted","Data":"56d1a9a91684a7f316251627853b6a658f01cc5aec2f95c409bede6ce26a91b0"}
Oct 30 21:28:35.585494 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:35.585032    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ggbl6" event={"ID":"88eb2d7e-b5dd-4769-b30c-756140a8cc1d","Type":"ContainerStarted","Data":"f1ff9b295456d2b5bce40ef00d636080afe7237e931ae4d05bf5734388ee42c4"}
Oct 30 21:28:35.612044 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:35.611913    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-ggbl6" podStartSLOduration=1.957797809 podStartE2EDuration="3.61189445s" totalImagesPullingTime="1.555617889s" totalInitContainerRuntime="98.478752ms" isStatefulPod=true podCreationTimestamp="2026-10-30 21:28:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:28:35.611196821 +0000 UTC m=+28.413009287" watchObservedRunningTime="2026-10-30 21:28:35.61189445 +0000 UTC m=+28.413706916"
Oct 30 21:28:38.223747 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:38.223710    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-4cvlt"
Oct 30 21:28:38.295968 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:38.295938    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-l8g24"
Oct 30 21:28:42.622674 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:42.622630    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6dd789c8b7-qk429" event={"ID":"ea11393a-a054-4a6d-9194-156a7adc6c7e","Type":"ContainerStarted","Data":"fc3bccde03e210cbd8e1f902e9c84de145f4d9c6d2152474d426054131c9e3a7"}
Oct 30 21:28:42.623180 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:42.622807    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-6dd789c8b7-qk429"
Oct 30 21:28:42.638305 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:42.638266    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-6dd789c8b7-qk429"
Oct 30 21:28:42.651703 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:42.651639    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/downloads-6dd789c8b7-qk429" podStartSLOduration=1.498300706 podStartE2EDuration="17.651625494s" totalImagesPullingTime="16.153324788s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:28:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:28:42.651352957 +0000 UTC m=+35.453165444" watchObservedRunningTime="2026-10-30 21:28:42.651625494 +0000 UTC m=+35.453437973"
Oct 30 21:28:47.543042 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:47.543014    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-848d9d9bc8-mjlrc"
Oct 30 21:28:47.608343 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:47.608309    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-76679b6db7-ltmt6"]
Oct 30 21:28:48.367082 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:28:48.367039    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 21:28:50.359966 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:50.359878    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-86f87fb58b-5clbv" podUID="94ee3757-5440-42e1-8617-2a68744ef3f4" containerName="registry" containerID="cri-o://dc727b9cb12e0b753212acb1bd0d7a41416b5367c51e9bb65267bc006b18a1f6" gracePeriod=30
Oct 30 21:28:50.652678 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:50.652646    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-86f87fb58b-5clbv"
Oct 30 21:28:50.692303 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:50.692266    5032 generic.go:356] "Generic (PLEG): container finished" podID="94ee3757-5440-42e1-8617-2a68744ef3f4" containerID="dc727b9cb12e0b753212acb1bd0d7a41416b5367c51e9bb65267bc006b18a1f6" exitCode=0
Oct 30 21:28:50.692467 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:50.692377    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-86f87fb58b-5clbv"
Oct 30 21:28:50.692467 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:50.692398    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-86f87fb58b-5clbv" event={"ID":"94ee3757-5440-42e1-8617-2a68744ef3f4","Type":"ContainerDied","Data":"dc727b9cb12e0b753212acb1bd0d7a41416b5367c51e9bb65267bc006b18a1f6"}
Oct 30 21:28:50.692544 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:50.692484    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-86f87fb58b-5clbv" event={"ID":"94ee3757-5440-42e1-8617-2a68744ef3f4","Type":"ContainerDied","Data":"8a95cd95aced123726cf205b6e1b2f9a9430212f7cd83936c4e10ed0b98fc054"}
Oct 30 21:28:50.692544 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:50.692510    5032 scope.go:122] "RemoveContainer" containerID="dc727b9cb12e0b753212acb1bd0d7a41416b5367c51e9bb65267bc006b18a1f6"
Oct 30 21:28:50.708758 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:50.708734    5032 scope.go:122] "RemoveContainer" containerID="dc727b9cb12e0b753212acb1bd0d7a41416b5367c51e9bb65267bc006b18a1f6"
Oct 30 21:28:50.709125 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:28:50.709098    5032 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc727b9cb12e0b753212acb1bd0d7a41416b5367c51e9bb65267bc006b18a1f6\": container with ID starting with dc727b9cb12e0b753212acb1bd0d7a41416b5367c51e9bb65267bc006b18a1f6 not found: ID does not exist" containerID="dc727b9cb12e0b753212acb1bd0d7a41416b5367c51e9bb65267bc006b18a1f6"
Oct 30 21:28:50.709227 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:50.709136    5032 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc727b9cb12e0b753212acb1bd0d7a41416b5367c51e9bb65267bc006b18a1f6"} err="failed to get container status \"dc727b9cb12e0b753212acb1bd0d7a41416b5367c51e9bb65267bc006b18a1f6\": rpc error: code = NotFound desc = could not find container \"dc727b9cb12e0b753212acb1bd0d7a41416b5367c51e9bb65267bc006b18a1f6\": container with ID starting with dc727b9cb12e0b753212acb1bd0d7a41416b5367c51e9bb65267bc006b18a1f6 not found: ID does not exist"
Oct 30 21:28:50.744621 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:50.744585    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/94ee3757-5440-42e1-8617-2a68744ef3f4-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/94ee3757-5440-42e1-8617-2a68744ef3f4-registry-certificates\") pod \"94ee3757-5440-42e1-8617-2a68744ef3f4\" (UID: \"94ee3757-5440-42e1-8617-2a68744ef3f4\") "
Oct 30 21:28:50.744807 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:50.744657    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/94ee3757-5440-42e1-8617-2a68744ef3f4-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/94ee3757-5440-42e1-8617-2a68744ef3f4-image-registry-private-configuration\") pod \"94ee3757-5440-42e1-8617-2a68744ef3f4\" (UID: \"94ee3757-5440-42e1-8617-2a68744ef3f4\") "
Oct 30 21:28:50.744807 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:50.744698    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/94ee3757-5440-42e1-8617-2a68744ef3f4-kube-api-access-t6lpv\" (UniqueName: \"kubernetes.io/projected/94ee3757-5440-42e1-8617-2a68744ef3f4-kube-api-access-t6lpv\") pod \"94ee3757-5440-42e1-8617-2a68744ef3f4\" (UID: \"94ee3757-5440-42e1-8617-2a68744ef3f4\") "
Oct 30 21:28:50.744807 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:50.744720    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/94ee3757-5440-42e1-8617-2a68744ef3f4-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/94ee3757-5440-42e1-8617-2a68744ef3f4-installation-pull-secrets\") pod \"94ee3757-5440-42e1-8617-2a68744ef3f4\" (UID: \"94ee3757-5440-42e1-8617-2a68744ef3f4\") "
Oct 30 21:28:50.744807 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:50.744747    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/94ee3757-5440-42e1-8617-2a68744ef3f4-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/94ee3757-5440-42e1-8617-2a68744ef3f4-bound-sa-token\") pod \"94ee3757-5440-42e1-8617-2a68744ef3f4\" (UID: \"94ee3757-5440-42e1-8617-2a68744ef3f4\") "
Oct 30 21:28:50.744807 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:50.744764    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/94ee3757-5440-42e1-8617-2a68744ef3f4-registry-tls\" (UniqueName: \"kubernetes.io/projected/94ee3757-5440-42e1-8617-2a68744ef3f4-registry-tls\") pod \"94ee3757-5440-42e1-8617-2a68744ef3f4\" (UID: \"94ee3757-5440-42e1-8617-2a68744ef3f4\") "
Oct 30 21:28:50.744807 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:50.744790    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/94ee3757-5440-42e1-8617-2a68744ef3f4-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/94ee3757-5440-42e1-8617-2a68744ef3f4-ca-trust-extracted\") pod \"94ee3757-5440-42e1-8617-2a68744ef3f4\" (UID: \"94ee3757-5440-42e1-8617-2a68744ef3f4\") "
Oct 30 21:28:50.745126 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:50.744853    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/94ee3757-5440-42e1-8617-2a68744ef3f4-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/94ee3757-5440-42e1-8617-2a68744ef3f4-trusted-ca\") pod \"94ee3757-5440-42e1-8617-2a68744ef3f4\" (UID: \"94ee3757-5440-42e1-8617-2a68744ef3f4\") "
Oct 30 21:28:50.745126 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:50.745071    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/94ee3757-5440-42e1-8617-2a68744ef3f4-registry-certificates" pod "94ee3757-5440-42e1-8617-2a68744ef3f4" (UID: "94ee3757-5440-42e1-8617-2a68744ef3f4"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:28:50.745535 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:50.745507    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/94ee3757-5440-42e1-8617-2a68744ef3f4-trusted-ca" pod "94ee3757-5440-42e1-8617-2a68744ef3f4" (UID: "94ee3757-5440-42e1-8617-2a68744ef3f4"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:28:50.747375 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:50.747333    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94ee3757-5440-42e1-8617-2a68744ef3f4-kube-api-access-t6lpv" pod "94ee3757-5440-42e1-8617-2a68744ef3f4" (UID: "94ee3757-5440-42e1-8617-2a68744ef3f4"). InnerVolumeSpecName "kube-api-access-t6lpv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:28:50.747495 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:50.747448    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/94ee3757-5440-42e1-8617-2a68744ef3f4-image-registry-private-configuration" pod "94ee3757-5440-42e1-8617-2a68744ef3f4" (UID: "94ee3757-5440-42e1-8617-2a68744ef3f4"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:28:50.747495 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:50.747469    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94ee3757-5440-42e1-8617-2a68744ef3f4-bound-sa-token" pod "94ee3757-5440-42e1-8617-2a68744ef3f4" (UID: "94ee3757-5440-42e1-8617-2a68744ef3f4"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:28:50.747599 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:50.747477    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94ee3757-5440-42e1-8617-2a68744ef3f4-registry-tls" pod "94ee3757-5440-42e1-8617-2a68744ef3f4" (UID: "94ee3757-5440-42e1-8617-2a68744ef3f4"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:28:50.747730 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:50.747707    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/94ee3757-5440-42e1-8617-2a68744ef3f4-installation-pull-secrets" pod "94ee3757-5440-42e1-8617-2a68744ef3f4" (UID: "94ee3757-5440-42e1-8617-2a68744ef3f4"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:28:50.752088 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:50.752062    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94ee3757-5440-42e1-8617-2a68744ef3f4-ca-trust-extracted" pod "94ee3757-5440-42e1-8617-2a68744ef3f4" (UID: "94ee3757-5440-42e1-8617-2a68744ef3f4"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:28:50.845453 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:50.845419    5032 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/94ee3757-5440-42e1-8617-2a68744ef3f4-registry-certificates\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:28:50.845453 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:50.845448    5032 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/94ee3757-5440-42e1-8617-2a68744ef3f4-image-registry-private-configuration\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:28:50.845453 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:50.845459    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t6lpv\" (UniqueName: \"kubernetes.io/projected/94ee3757-5440-42e1-8617-2a68744ef3f4-kube-api-access-t6lpv\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:28:50.845678 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:50.845468    5032 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/94ee3757-5440-42e1-8617-2a68744ef3f4-installation-pull-secrets\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:28:50.845678 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:50.845479    5032 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/94ee3757-5440-42e1-8617-2a68744ef3f4-bound-sa-token\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:28:50.845678 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:50.845488    5032 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/94ee3757-5440-42e1-8617-2a68744ef3f4-registry-tls\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:28:50.845678 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:50.845496    5032 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/94ee3757-5440-42e1-8617-2a68744ef3f4-ca-trust-extracted\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:28:50.845678 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:50.845505    5032 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/94ee3757-5440-42e1-8617-2a68744ef3f4-trusted-ca\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:28:51.022056 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:51.022023    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-86f87fb58b-5clbv"]
Oct 30 21:28:51.024546 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:51.024523    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-86f87fb58b-5clbv"]
Oct 30 21:28:51.343945 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:51.343863    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="94ee3757-5440-42e1-8617-2a68744ef3f4" path="/var/lib/kubelet/pods/94ee3757-5440-42e1-8617-2a68744ef3f4/volumes"
Oct 30 21:28:56.813856 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:56.813818    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-542ht-hjp6z"]
Oct 30 21:28:56.814349 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:56.814010    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-insights/periodic-gathering-542ht-hjp6z" podUID="fd34a723-6c9e-48f1-ba7d-924d542b2bc7" containerName="insights-gathering" containerID="cri-o://891019481c213734fb14c99190ebe423c215bdaca160040e5852097370c6f0c2" gracePeriod=30
Oct 30 21:28:57.112922 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:57.112887    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-542ht-hjp6z"
Oct 30 21:28:57.201099 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:57.201056    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fd34a723-6c9e-48f1-ba7d-924d542b2bc7-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd34a723-6c9e-48f1-ba7d-924d542b2bc7-service-ca-bundle\") pod \"fd34a723-6c9e-48f1-ba7d-924d542b2bc7\" (UID: \"fd34a723-6c9e-48f1-ba7d-924d542b2bc7\") "
Oct 30 21:28:57.201099 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:57.201101    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd34a723-6c9e-48f1-ba7d-924d542b2bc7-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/fd34a723-6c9e-48f1-ba7d-924d542b2bc7-archives-path\") pod \"fd34a723-6c9e-48f1-ba7d-924d542b2bc7\" (UID: \"fd34a723-6c9e-48f1-ba7d-924d542b2bc7\") "
Oct 30 21:28:57.201316 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:57.201168    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fd34a723-6c9e-48f1-ba7d-924d542b2bc7-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd34a723-6c9e-48f1-ba7d-924d542b2bc7-trusted-ca-bundle\") pod \"fd34a723-6c9e-48f1-ba7d-924d542b2bc7\" (UID: \"fd34a723-6c9e-48f1-ba7d-924d542b2bc7\") "
Oct 30 21:28:57.201316 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:57.201197    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fd34a723-6c9e-48f1-ba7d-924d542b2bc7-kube-api-access-fnx5t\" (UniqueName: \"kubernetes.io/projected/fd34a723-6c9e-48f1-ba7d-924d542b2bc7-kube-api-access-fnx5t\") pod \"fd34a723-6c9e-48f1-ba7d-924d542b2bc7\" (UID: \"fd34a723-6c9e-48f1-ba7d-924d542b2bc7\") "
Oct 30 21:28:57.201542 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:57.201517    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd34a723-6c9e-48f1-ba7d-924d542b2bc7-service-ca-bundle" pod "fd34a723-6c9e-48f1-ba7d-924d542b2bc7" (UID: "fd34a723-6c9e-48f1-ba7d-924d542b2bc7"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:28:57.201586 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:57.201513    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd34a723-6c9e-48f1-ba7d-924d542b2bc7-archives-path" pod "fd34a723-6c9e-48f1-ba7d-924d542b2bc7" (UID: "fd34a723-6c9e-48f1-ba7d-924d542b2bc7"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:28:57.201586 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:57.201574    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd34a723-6c9e-48f1-ba7d-924d542b2bc7-trusted-ca-bundle" pod "fd34a723-6c9e-48f1-ba7d-924d542b2bc7" (UID: "fd34a723-6c9e-48f1-ba7d-924d542b2bc7"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:28:57.203603 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:57.203584    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd34a723-6c9e-48f1-ba7d-924d542b2bc7-kube-api-access-fnx5t" pod "fd34a723-6c9e-48f1-ba7d-924d542b2bc7" (UID: "fd34a723-6c9e-48f1-ba7d-924d542b2bc7"). InnerVolumeSpecName "kube-api-access-fnx5t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:28:57.302344 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:57.302303    5032 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd34a723-6c9e-48f1-ba7d-924d542b2bc7-trusted-ca-bundle\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:28:57.302344 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:57.302336    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fnx5t\" (UniqueName: \"kubernetes.io/projected/fd34a723-6c9e-48f1-ba7d-924d542b2bc7-kube-api-access-fnx5t\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:28:57.302344 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:57.302349    5032 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd34a723-6c9e-48f1-ba7d-924d542b2bc7-service-ca-bundle\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:28:57.302344 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:57.302382    5032 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/fd34a723-6c9e-48f1-ba7d-924d542b2bc7-archives-path\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:28:57.728962 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:57.728927    5032 generic.go:356] "Generic (PLEG): container finished" podID="fd34a723-6c9e-48f1-ba7d-924d542b2bc7" containerID="891019481c213734fb14c99190ebe423c215bdaca160040e5852097370c6f0c2" exitCode=2
Oct 30 21:28:57.729154 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:57.728973    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-542ht-hjp6z"
Oct 30 21:28:57.729154 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:57.728987    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-542ht-hjp6z" event={"ID":"fd34a723-6c9e-48f1-ba7d-924d542b2bc7","Type":"ContainerDied","Data":"891019481c213734fb14c99190ebe423c215bdaca160040e5852097370c6f0c2"}
Oct 30 21:28:57.729154 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:57.729034    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-542ht-hjp6z" event={"ID":"fd34a723-6c9e-48f1-ba7d-924d542b2bc7","Type":"ContainerDied","Data":"1dca56b0734ec47a566bcaede4782a3c941f7018b609beabda556ecedc01fe47"}
Oct 30 21:28:57.729154 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:57.729061    5032 scope.go:122] "RemoveContainer" containerID="891019481c213734fb14c99190ebe423c215bdaca160040e5852097370c6f0c2"
Oct 30 21:28:57.737497 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:57.737475    5032 scope.go:122] "RemoveContainer" containerID="891019481c213734fb14c99190ebe423c215bdaca160040e5852097370c6f0c2"
Oct 30 21:28:57.737781 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:28:57.737755    5032 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"891019481c213734fb14c99190ebe423c215bdaca160040e5852097370c6f0c2\": container with ID starting with 891019481c213734fb14c99190ebe423c215bdaca160040e5852097370c6f0c2 not found: ID does not exist" containerID="891019481c213734fb14c99190ebe423c215bdaca160040e5852097370c6f0c2"
Oct 30 21:28:57.737844 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:57.737794    5032 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"891019481c213734fb14c99190ebe423c215bdaca160040e5852097370c6f0c2"} err="failed to get container status \"891019481c213734fb14c99190ebe423c215bdaca160040e5852097370c6f0c2\": rpc error: code = NotFound desc = could not find container \"891019481c213734fb14c99190ebe423c215bdaca160040e5852097370c6f0c2\": container with ID starting with 891019481c213734fb14c99190ebe423c215bdaca160040e5852097370c6f0c2 not found: ID does not exist"
Oct 30 21:28:57.747072 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:57.747045    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-542ht-hjp6z"]
Oct 30 21:28:57.751218 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:57.751167    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-insights/periodic-gathering-542ht-hjp6z"]
Oct 30 21:28:59.347478 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:28:59.347443    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fd34a723-6c9e-48f1-ba7d-924d542b2bc7" path="/var/lib/kubelet/pods/fd34a723-6c9e-48f1-ba7d-924d542b2bc7/volumes"
Oct 30 21:29:12.642383 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:29:12.642311    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-76679b6db7-ltmt6" podUID="039cf439-d12d-450a-80c1-eb3efeb50e8a" containerName="registry" containerID="cri-o://b9d509b1b6463a1b3dd6efc1021cb82cdd6139c5d78f759ed3a0d3ced055ebe5" gracePeriod=30
Oct 30 21:29:12.793641 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:29:12.793608    5032 generic.go:356] "Generic (PLEG): container finished" podID="039cf439-d12d-450a-80c1-eb3efeb50e8a" containerID="b9d509b1b6463a1b3dd6efc1021cb82cdd6139c5d78f759ed3a0d3ced055ebe5" exitCode=0
Oct 30 21:29:12.793815 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:29:12.793678    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76679b6db7-ltmt6" event={"ID":"039cf439-d12d-450a-80c1-eb3efeb50e8a","Type":"ContainerDied","Data":"b9d509b1b6463a1b3dd6efc1021cb82cdd6139c5d78f759ed3a0d3ced055ebe5"}
Oct 30 21:29:12.930503 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:29:12.930472    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76679b6db7-ltmt6"
Oct 30 21:29:13.041737 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:29:13.041696    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/039cf439-d12d-450a-80c1-eb3efeb50e8a-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/039cf439-d12d-450a-80c1-eb3efeb50e8a-trusted-ca\") pod \"039cf439-d12d-450a-80c1-eb3efeb50e8a\" (UID: \"039cf439-d12d-450a-80c1-eb3efeb50e8a\") "
Oct 30 21:29:13.041919 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:29:13.041748    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/039cf439-d12d-450a-80c1-eb3efeb50e8a-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/039cf439-d12d-450a-80c1-eb3efeb50e8a-ca-trust-extracted\") pod \"039cf439-d12d-450a-80c1-eb3efeb50e8a\" (UID: \"039cf439-d12d-450a-80c1-eb3efeb50e8a\") "
Oct 30 21:29:13.041919 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:29:13.041781    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/039cf439-d12d-450a-80c1-eb3efeb50e8a-registry-tls\" (UniqueName: \"kubernetes.io/projected/039cf439-d12d-450a-80c1-eb3efeb50e8a-registry-tls\") pod \"039cf439-d12d-450a-80c1-eb3efeb50e8a\" (UID: \"039cf439-d12d-450a-80c1-eb3efeb50e8a\") "
Oct 30 21:29:13.041919 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:29:13.041800    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/039cf439-d12d-450a-80c1-eb3efeb50e8a-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/039cf439-d12d-450a-80c1-eb3efeb50e8a-installation-pull-secrets\") pod \"039cf439-d12d-450a-80c1-eb3efeb50e8a\" (UID: \"039cf439-d12d-450a-80c1-eb3efeb50e8a\") "
Oct 30 21:29:13.041919 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:29:13.041821    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/039cf439-d12d-450a-80c1-eb3efeb50e8a-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/039cf439-d12d-450a-80c1-eb3efeb50e8a-bound-sa-token\") pod \"039cf439-d12d-450a-80c1-eb3efeb50e8a\" (UID: \"039cf439-d12d-450a-80c1-eb3efeb50e8a\") "
Oct 30 21:29:13.041919 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:29:13.041857    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/039cf439-d12d-450a-80c1-eb3efeb50e8a-kube-api-access-zrrdm\" (UniqueName: \"kubernetes.io/projected/039cf439-d12d-450a-80c1-eb3efeb50e8a-kube-api-access-zrrdm\") pod \"039cf439-d12d-450a-80c1-eb3efeb50e8a\" (UID: \"039cf439-d12d-450a-80c1-eb3efeb50e8a\") "
Oct 30 21:29:13.041919 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:29:13.041894    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/039cf439-d12d-450a-80c1-eb3efeb50e8a-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/039cf439-d12d-450a-80c1-eb3efeb50e8a-registry-certificates\") pod \"039cf439-d12d-450a-80c1-eb3efeb50e8a\" (UID: \"039cf439-d12d-450a-80c1-eb3efeb50e8a\") "
Oct 30 21:29:13.042430 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:29:13.041922    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/039cf439-d12d-450a-80c1-eb3efeb50e8a-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/039cf439-d12d-450a-80c1-eb3efeb50e8a-image-registry-private-configuration\") pod \"039cf439-d12d-450a-80c1-eb3efeb50e8a\" (UID: \"039cf439-d12d-450a-80c1-eb3efeb50e8a\") "
Oct 30 21:29:13.042430 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:29:13.042207    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/039cf439-d12d-450a-80c1-eb3efeb50e8a-trusted-ca" pod "039cf439-d12d-450a-80c1-eb3efeb50e8a" (UID: "039cf439-d12d-450a-80c1-eb3efeb50e8a"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:29:13.042742 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:29:13.042715    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/039cf439-d12d-450a-80c1-eb3efeb50e8a-registry-certificates" pod "039cf439-d12d-450a-80c1-eb3efeb50e8a" (UID: "039cf439-d12d-450a-80c1-eb3efeb50e8a"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:29:13.044760 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:29:13.044703    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/039cf439-d12d-450a-80c1-eb3efeb50e8a-registry-tls" pod "039cf439-d12d-450a-80c1-eb3efeb50e8a" (UID: "039cf439-d12d-450a-80c1-eb3efeb50e8a"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:29:13.045003 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:29:13.044967    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/039cf439-d12d-450a-80c1-eb3efeb50e8a-image-registry-private-configuration" pod "039cf439-d12d-450a-80c1-eb3efeb50e8a" (UID: "039cf439-d12d-450a-80c1-eb3efeb50e8a"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:29:13.045078 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:29:13.045033    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/039cf439-d12d-450a-80c1-eb3efeb50e8a-installation-pull-secrets" pod "039cf439-d12d-450a-80c1-eb3efeb50e8a" (UID: "039cf439-d12d-450a-80c1-eb3efeb50e8a"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:29:13.045078 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:29:13.045028    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/039cf439-d12d-450a-80c1-eb3efeb50e8a-bound-sa-token" pod "039cf439-d12d-450a-80c1-eb3efeb50e8a" (UID: "039cf439-d12d-450a-80c1-eb3efeb50e8a"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:29:13.045238 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:29:13.045221    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/039cf439-d12d-450a-80c1-eb3efeb50e8a-kube-api-access-zrrdm" pod "039cf439-d12d-450a-80c1-eb3efeb50e8a" (UID: "039cf439-d12d-450a-80c1-eb3efeb50e8a"). InnerVolumeSpecName "kube-api-access-zrrdm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:29:13.049100 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:29:13.049077    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/039cf439-d12d-450a-80c1-eb3efeb50e8a-ca-trust-extracted" pod "039cf439-d12d-450a-80c1-eb3efeb50e8a" (UID: "039cf439-d12d-450a-80c1-eb3efeb50e8a"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:29:13.142976 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:29:13.142946    5032 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/039cf439-d12d-450a-80c1-eb3efeb50e8a-trusted-ca\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:29:13.142976 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:29:13.142974    5032 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/039cf439-d12d-450a-80c1-eb3efeb50e8a-ca-trust-extracted\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:29:13.142976 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:29:13.142984    5032 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/039cf439-d12d-450a-80c1-eb3efeb50e8a-registry-tls\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:29:13.143204 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:29:13.142995    5032 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/039cf439-d12d-450a-80c1-eb3efeb50e8a-installation-pull-secrets\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:29:13.143204 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:29:13.143005    5032 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/039cf439-d12d-450a-80c1-eb3efeb50e8a-bound-sa-token\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:29:13.143204 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:29:13.143014    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zrrdm\" (UniqueName: \"kubernetes.io/projected/039cf439-d12d-450a-80c1-eb3efeb50e8a-kube-api-access-zrrdm\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:29:13.143204 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:29:13.143023    5032 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/039cf439-d12d-450a-80c1-eb3efeb50e8a-registry-certificates\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:29:13.143204 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:29:13.143033    5032 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/039cf439-d12d-450a-80c1-eb3efeb50e8a-image-registry-private-configuration\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:29:13.798803 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:29:13.798771    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76679b6db7-ltmt6"
Oct 30 21:29:13.799225 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:29:13.798809    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76679b6db7-ltmt6" event={"ID":"039cf439-d12d-450a-80c1-eb3efeb50e8a","Type":"ContainerDied","Data":"7ce3e4dac809a9eee2dd50a2b0facda4fa1b1b7342abc9a3eed42cbf9920fb9d"}
Oct 30 21:29:13.799225 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:29:13.798870    5032 scope.go:122] "RemoveContainer" containerID="b9d509b1b6463a1b3dd6efc1021cb82cdd6139c5d78f759ed3a0d3ced055ebe5"
Oct 30 21:29:13.819285 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:29:13.819254    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-76679b6db7-ltmt6"]
Oct 30 21:29:13.823720 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:29:13.823693    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-76679b6db7-ltmt6"]
Oct 30 21:29:15.343703 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:29:15.343671    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="039cf439-d12d-450a-80c1-eb3efeb50e8a" path="/var/lib/kubelet/pods/039cf439-d12d-450a-80c1-eb3efeb50e8a/volumes"
Oct 30 21:29:16.813352 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:29:16.813318    5032 generic.go:356] "Generic (PLEG): container finished" podID="8bb24a0d-3af0-4a05-bba4-4f9630b04bf3" containerID="4b323db09cec9d37c1aea20e981dcafa4bed4684d8d34a1c16d709bb371445dc" exitCode=0
Oct 30 21:29:16.813872 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:29:16.813395    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-9p8mb" event={"ID":"8bb24a0d-3af0-4a05-bba4-4f9630b04bf3","Type":"ContainerDied","Data":"4b323db09cec9d37c1aea20e981dcafa4bed4684d8d34a1c16d709bb371445dc"}
Oct 30 21:29:16.813872 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:29:16.813800    5032 scope.go:122] "RemoveContainer" containerID="4b323db09cec9d37c1aea20e981dcafa4bed4684d8d34a1c16d709bb371445dc"
Oct 30 21:29:17.818908 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:29:17.818874    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-9p8mb" event={"ID":"8bb24a0d-3af0-4a05-bba4-4f9630b04bf3","Type":"ContainerStarted","Data":"6540d43c572220bb1f3ce8b93a49e8cf048bf7d3818a44ebae27a0d06d15311e"}
Oct 30 21:29:18.344038 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:29:18.344002    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 21:29:36.898766 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:29:36.898726    5032 generic.go:356] "Generic (PLEG): container finished" podID="8873f9b9-2ac5-409c-9299-fa024e00189c" containerID="e332f7c86412ccecce5e050bcc3cbe8ebb64334a0ce1c1d94cca07e381a2cb5f" exitCode=0
Oct 30 21:29:36.899247 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:29:36.898802    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-9r5sb" event={"ID":"8873f9b9-2ac5-409c-9299-fa024e00189c","Type":"ContainerDied","Data":"e332f7c86412ccecce5e050bcc3cbe8ebb64334a0ce1c1d94cca07e381a2cb5f"}
Oct 30 21:29:36.899247 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:29:36.899210    5032 scope.go:122] "RemoveContainer" containerID="e332f7c86412ccecce5e050bcc3cbe8ebb64334a0ce1c1d94cca07e381a2cb5f"
Oct 30 21:29:37.904738 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:29:37.904696    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-9r5sb" event={"ID":"8873f9b9-2ac5-409c-9299-fa024e00189c","Type":"ContainerStarted","Data":"a22a16e5305d5d6531532a6d4e93f765197ea7b4cbfc7984da3c94f7a3350fb8"}
Oct 30 21:29:48.343471 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:29:48.343435    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 21:29:51.962314 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:29:51.962281    5032 generic.go:356] "Generic (PLEG): container finished" podID="2495cf48-596f-4ff0-afa7-4e5a9521d6ab" containerID="398bac5f4f91d3bd75182331b2a6af8040644db3b37d7453d641f3fd5e924ef5" exitCode=0
Oct 30 21:29:51.962706 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:29:51.962375    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-nssh6-9f5sq" event={"ID":"2495cf48-596f-4ff0-afa7-4e5a9521d6ab","Type":"ContainerDied","Data":"398bac5f4f91d3bd75182331b2a6af8040644db3b37d7453d641f3fd5e924ef5"}
Oct 30 21:29:53.092651 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:29:53.092626    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-nssh6-9f5sq"
Oct 30 21:29:53.161422 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:29:53.161387    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2495cf48-596f-4ff0-afa7-4e5a9521d6ab-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2495cf48-596f-4ff0-afa7-4e5a9521d6ab-service-ca-bundle\") pod \"2495cf48-596f-4ff0-afa7-4e5a9521d6ab\" (UID: \"2495cf48-596f-4ff0-afa7-4e5a9521d6ab\") "
Oct 30 21:29:53.161618 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:29:53.161456    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2495cf48-596f-4ff0-afa7-4e5a9521d6ab-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2495cf48-596f-4ff0-afa7-4e5a9521d6ab-trusted-ca-bundle\") pod \"2495cf48-596f-4ff0-afa7-4e5a9521d6ab\" (UID: \"2495cf48-596f-4ff0-afa7-4e5a9521d6ab\") "
Oct 30 21:29:53.161618 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:29:53.161480    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2495cf48-596f-4ff0-afa7-4e5a9521d6ab-kube-api-access-hkfsh\" (UniqueName: \"kubernetes.io/projected/2495cf48-596f-4ff0-afa7-4e5a9521d6ab-kube-api-access-hkfsh\") pod \"2495cf48-596f-4ff0-afa7-4e5a9521d6ab\" (UID: \"2495cf48-596f-4ff0-afa7-4e5a9521d6ab\") "
Oct 30 21:29:53.161618 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:29:53.161509    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2495cf48-596f-4ff0-afa7-4e5a9521d6ab-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/2495cf48-596f-4ff0-afa7-4e5a9521d6ab-archives-path\") pod \"2495cf48-596f-4ff0-afa7-4e5a9521d6ab\" (UID: \"2495cf48-596f-4ff0-afa7-4e5a9521d6ab\") "
Oct 30 21:29:53.161901 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:29:53.161871    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2495cf48-596f-4ff0-afa7-4e5a9521d6ab-service-ca-bundle" pod "2495cf48-596f-4ff0-afa7-4e5a9521d6ab" (UID: "2495cf48-596f-4ff0-afa7-4e5a9521d6ab"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:29:53.162022 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:29:53.161920    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2495cf48-596f-4ff0-afa7-4e5a9521d6ab-trusted-ca-bundle" pod "2495cf48-596f-4ff0-afa7-4e5a9521d6ab" (UID: "2495cf48-596f-4ff0-afa7-4e5a9521d6ab"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:29:53.162022 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:29:53.161953    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2495cf48-596f-4ff0-afa7-4e5a9521d6ab-archives-path" pod "2495cf48-596f-4ff0-afa7-4e5a9521d6ab" (UID: "2495cf48-596f-4ff0-afa7-4e5a9521d6ab"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:29:53.163689 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:29:53.163667    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2495cf48-596f-4ff0-afa7-4e5a9521d6ab-kube-api-access-hkfsh" pod "2495cf48-596f-4ff0-afa7-4e5a9521d6ab" (UID: "2495cf48-596f-4ff0-afa7-4e5a9521d6ab"). InnerVolumeSpecName "kube-api-access-hkfsh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:29:53.262438 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:29:53.262318    5032 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/2495cf48-596f-4ff0-afa7-4e5a9521d6ab-archives-path\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:29:53.262438 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:29:53.262354    5032 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2495cf48-596f-4ff0-afa7-4e5a9521d6ab-service-ca-bundle\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:29:53.262438 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:29:53.262393    5032 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2495cf48-596f-4ff0-afa7-4e5a9521d6ab-trusted-ca-bundle\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:29:53.262438 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:29:53.262407    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hkfsh\" (UniqueName: \"kubernetes.io/projected/2495cf48-596f-4ff0-afa7-4e5a9521d6ab-kube-api-access-hkfsh\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:29:53.971905 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:29:53.971870    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-nssh6-9f5sq"
Oct 30 21:29:53.972154 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:29:53.971875    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-nssh6-9f5sq" event={"ID":"2495cf48-596f-4ff0-afa7-4e5a9521d6ab","Type":"ContainerDied","Data":"167d6f9e13b104e2a5c5a4ca9b52839bdbd8f737aef9fad3ddba99ac7c189801"}
Oct 30 21:29:53.972154 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:29:53.972005    5032 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="167d6f9e13b104e2a5c5a4ca9b52839bdbd8f737aef9fad3ddba99ac7c189801"
Oct 30 21:30:18.343934 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:30:18.343834    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 21:30:48.344121 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:30:48.344077    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 21:31:06.525013 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:06.524967    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj"]
Oct 30 21:31:06.525671 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:06.525296    5032 cpu_manager.go:414] "containerMap: removing container" podUID="fd34a723-6c9e-48f1-ba7d-924d542b2bc7" containerName="insights-gathering"
Oct 30 21:31:06.525671 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:06.525308    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fd34a723-6c9e-48f1-ba7d-924d542b2bc7" containerName="insights-gathering"
Oct 30 21:31:06.525671 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:06.525332    5032 cpu_manager.go:414] "containerMap: removing container" podUID="94ee3757-5440-42e1-8617-2a68744ef3f4" containerName="registry"
Oct 30 21:31:06.525671 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:06.525338    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="94ee3757-5440-42e1-8617-2a68744ef3f4" containerName="registry"
Oct 30 21:31:06.525671 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:06.525350    5032 cpu_manager.go:414] "containerMap: removing container" podUID="2495cf48-596f-4ff0-afa7-4e5a9521d6ab" containerName="insights-gathering"
Oct 30 21:31:06.525671 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:06.525355    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2495cf48-596f-4ff0-afa7-4e5a9521d6ab" containerName="insights-gathering"
Oct 30 21:31:06.525671 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:06.525382    5032 cpu_manager.go:414] "containerMap: removing container" podUID="039cf439-d12d-450a-80c1-eb3efeb50e8a" containerName="registry"
Oct 30 21:31:06.525671 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:06.525387    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="039cf439-d12d-450a-80c1-eb3efeb50e8a" containerName="registry"
Oct 30 21:31:06.525671 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:06.525448    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="2495cf48-596f-4ff0-afa7-4e5a9521d6ab" containerName="insights-gathering"
Oct 30 21:31:06.525671 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:06.525460    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="039cf439-d12d-450a-80c1-eb3efeb50e8a" containerName="registry"
Oct 30 21:31:06.525671 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:06.525469    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="fd34a723-6c9e-48f1-ba7d-924d542b2bc7" containerName="insights-gathering"
Oct 30 21:31:06.525671 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:06.525476    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="94ee3757-5440-42e1-8617-2a68744ef3f4" containerName="registry"
Oct 30 21:31:06.531131 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:06.531104    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj"
Oct 30 21:31:06.543555 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:06.543527    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj"]
Oct 30 21:31:06.546443 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:06.546421    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"openshift-ai-inference-openshift-ai-inference-dockercfg-778ts\""
Oct 30 21:31:06.550342 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:06.550317    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Oct 30 21:31:06.671001 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:06.670962    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b114f93-6494-464c-bfec-b2f7386f353f-istio-token\") pod \"openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj\" (UID: \"6b114f93-6494-464c-bfec-b2f7386f353f\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj"
Oct 30 21:31:06.671001 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:06.671001    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsh4t\" (UniqueName: \"kubernetes.io/projected/6b114f93-6494-464c-bfec-b2f7386f353f-kube-api-access-lsh4t\") pod \"openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj\" (UID: \"6b114f93-6494-464c-bfec-b2f7386f353f\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj"
Oct 30 21:31:06.671216 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:06.671036    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b114f93-6494-464c-bfec-b2f7386f353f-workload-socket\") pod \"openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj\" (UID: \"6b114f93-6494-464c-bfec-b2f7386f353f\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj"
Oct 30 21:31:06.671216 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:06.671135    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b114f93-6494-464c-bfec-b2f7386f353f-istio-data\") pod \"openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj\" (UID: \"6b114f93-6494-464c-bfec-b2f7386f353f\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj"
Oct 30 21:31:06.671216 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:06.671171    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b114f93-6494-464c-bfec-b2f7386f353f-credential-socket\") pod \"openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj\" (UID: \"6b114f93-6494-464c-bfec-b2f7386f353f\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj"
Oct 30 21:31:06.671216 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:06.671190    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b114f93-6494-464c-bfec-b2f7386f353f-workload-certs\") pod \"openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj\" (UID: \"6b114f93-6494-464c-bfec-b2f7386f353f\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj"
Oct 30 21:31:06.671339 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:06.671275    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b114f93-6494-464c-bfec-b2f7386f353f-istio-envoy\") pod \"openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj\" (UID: \"6b114f93-6494-464c-bfec-b2f7386f353f\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj"
Oct 30 21:31:06.671339 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:06.671314    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b114f93-6494-464c-bfec-b2f7386f353f-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj\" (UID: \"6b114f93-6494-464c-bfec-b2f7386f353f\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj"
Oct 30 21:31:06.671434 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:06.671390    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b114f93-6494-464c-bfec-b2f7386f353f-istio-podinfo\") pod \"openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj\" (UID: \"6b114f93-6494-464c-bfec-b2f7386f353f\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj"
Oct 30 21:31:06.772677 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:06.772634    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b114f93-6494-464c-bfec-b2f7386f353f-workload-socket\") pod \"openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj\" (UID: \"6b114f93-6494-464c-bfec-b2f7386f353f\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj"
Oct 30 21:31:06.772677 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:06.772676    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b114f93-6494-464c-bfec-b2f7386f353f-istio-data\") pod \"openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj\" (UID: \"6b114f93-6494-464c-bfec-b2f7386f353f\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj"
Oct 30 21:31:06.772929 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:06.772705    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b114f93-6494-464c-bfec-b2f7386f353f-credential-socket\") pod \"openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj\" (UID: \"6b114f93-6494-464c-bfec-b2f7386f353f\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj"
Oct 30 21:31:06.772929 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:06.772730    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b114f93-6494-464c-bfec-b2f7386f353f-workload-certs\") pod \"openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj\" (UID: \"6b114f93-6494-464c-bfec-b2f7386f353f\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj"
Oct 30 21:31:06.772929 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:06.772779    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b114f93-6494-464c-bfec-b2f7386f353f-istio-envoy\") pod \"openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj\" (UID: \"6b114f93-6494-464c-bfec-b2f7386f353f\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj"
Oct 30 21:31:06.772929 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:06.772806    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b114f93-6494-464c-bfec-b2f7386f353f-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj\" (UID: \"6b114f93-6494-464c-bfec-b2f7386f353f\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj"
Oct 30 21:31:06.772929 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:06.772852    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b114f93-6494-464c-bfec-b2f7386f353f-istio-podinfo\") pod \"openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj\" (UID: \"6b114f93-6494-464c-bfec-b2f7386f353f\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj"
Oct 30 21:31:06.772929 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:06.772879    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b114f93-6494-464c-bfec-b2f7386f353f-istio-token\") pod \"openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj\" (UID: \"6b114f93-6494-464c-bfec-b2f7386f353f\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj"
Oct 30 21:31:06.772929 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:06.772900    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsh4t\" (UniqueName: \"kubernetes.io/projected/6b114f93-6494-464c-bfec-b2f7386f353f-kube-api-access-lsh4t\") pod \"openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj\" (UID: \"6b114f93-6494-464c-bfec-b2f7386f353f\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj"
Oct 30 21:31:06.773280 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:06.773102    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b114f93-6494-464c-bfec-b2f7386f353f-workload-socket\") pod \"openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj\" (UID: \"6b114f93-6494-464c-bfec-b2f7386f353f\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj"
Oct 30 21:31:06.773280 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:06.773153    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b114f93-6494-464c-bfec-b2f7386f353f-credential-socket\") pod \"openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj\" (UID: \"6b114f93-6494-464c-bfec-b2f7386f353f\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj"
Oct 30 21:31:06.773280 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:06.773206    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b114f93-6494-464c-bfec-b2f7386f353f-istio-data\") pod \"openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj\" (UID: \"6b114f93-6494-464c-bfec-b2f7386f353f\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj"
Oct 30 21:31:06.773606 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:06.773580    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b114f93-6494-464c-bfec-b2f7386f353f-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj\" (UID: \"6b114f93-6494-464c-bfec-b2f7386f353f\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj"
Oct 30 21:31:06.773674 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:06.773601    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b114f93-6494-464c-bfec-b2f7386f353f-workload-certs\") pod \"openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj\" (UID: \"6b114f93-6494-464c-bfec-b2f7386f353f\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj"
Oct 30 21:31:06.775299 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:06.775252    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b114f93-6494-464c-bfec-b2f7386f353f-istio-envoy\") pod \"openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj\" (UID: \"6b114f93-6494-464c-bfec-b2f7386f353f\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj"
Oct 30 21:31:06.775397 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:06.775382    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b114f93-6494-464c-bfec-b2f7386f353f-istio-podinfo\") pod \"openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj\" (UID: \"6b114f93-6494-464c-bfec-b2f7386f353f\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj"
Oct 30 21:31:06.785870 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:06.785845    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b114f93-6494-464c-bfec-b2f7386f353f-istio-token\") pod \"openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj\" (UID: \"6b114f93-6494-464c-bfec-b2f7386f353f\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj"
Oct 30 21:31:06.786078 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:06.786055    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsh4t\" (UniqueName: \"kubernetes.io/projected/6b114f93-6494-464c-bfec-b2f7386f353f-kube-api-access-lsh4t\") pod \"openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj\" (UID: \"6b114f93-6494-464c-bfec-b2f7386f353f\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj"
Oct 30 21:31:06.848062 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:06.848009    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj"
Oct 30 21:31:07.128254 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:07.128156    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj"]
Oct 30 21:31:07.131093 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:31:07.131069    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b114f93_6494_464c_bfec_b2f7386f353f.slice/crio-3cd68a0cec0b4a885c6e3c3639e65e7a3319e265b4a1f87810b7058d0d10de26 WatchSource:0}: Error finding container 3cd68a0cec0b4a885c6e3c3639e65e7a3319e265b4a1f87810b7058d0d10de26: Status 404 returned error can't find the container with id 3cd68a0cec0b4a885c6e3c3639e65e7a3319e265b4a1f87810b7058d0d10de26
Oct 30 21:31:07.273252 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:07.273216    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj" event={"ID":"6b114f93-6494-464c-bfec-b2f7386f353f","Type":"ContainerStarted","Data":"3cd68a0cec0b4a885c6e3c3639e65e7a3319e265b4a1f87810b7058d0d10de26"}
Oct 30 21:31:10.536418 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:10.536384    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:31:10.536801 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:10.536465    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:31:10.536801 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:10.536550    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:31:11.292622 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:11.292587    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj" event={"ID":"6b114f93-6494-464c-bfec-b2f7386f353f","Type":"ContainerStarted","Data":"ab36cca46caa8fa13ceab367bc7603a74d9876d9f3c9ecc8fdf49e29028b44b8"}
Oct 30 21:31:11.320241 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:11.320190    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj" podStartSLOduration=1.917703264 podStartE2EDuration="5.320176789s" totalImagesPullingTime="3.402473525s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:31:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:31:11.317843904 +0000 UTC m=+184.119656392" watchObservedRunningTime="2026-10-30 21:31:11.320176789 +0000 UTC m=+184.121989253"
Oct 30 21:31:11.849071 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:11.849032    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj"
Oct 30 21:31:11.853726 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:11.853701    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj"
Oct 30 21:31:12.297023 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:12.296991    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj"
Oct 30 21:31:12.297886 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:12.297866    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj"
Oct 30 21:31:18.343544 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:31:18.343507    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 21:31:41.304253 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:41.304219    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-6bbc596d7f-fq45v"]
Oct 30 21:31:41.316319 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:41.314794    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6bbc596d7f-fq45v"
Oct 30 21:31:41.337643 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:41.337583    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-6bbc596d7f-fq45v"]
Oct 30 21:31:41.368488 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:41.368453    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ed1ba0e9-d432-4674-b52e-7110f348e98a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6bbc596d7f-fq45v\" (UID: \"ed1ba0e9-d432-4674-b52e-7110f348e98a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6bbc596d7f-fq45v"
Oct 30 21:31:41.368667 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:41.368497    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ed1ba0e9-d432-4674-b52e-7110f348e98a-tmp\") pod \"opendatahub-operator-controller-manager-6bbc596d7f-fq45v\" (UID: \"ed1ba0e9-d432-4674-b52e-7110f348e98a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6bbc596d7f-fq45v"
Oct 30 21:31:41.368667 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:41.368518    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ed1ba0e9-d432-4674-b52e-7110f348e98a-metrics-cert\") pod \"opendatahub-operator-controller-manager-6bbc596d7f-fq45v\" (UID: \"ed1ba0e9-d432-4674-b52e-7110f348e98a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6bbc596d7f-fq45v"
Oct 30 21:31:41.368738 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:41.368673    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ed1ba0e9-d432-4674-b52e-7110f348e98a-webhook-cert\") pod \"opendatahub-operator-controller-manager-6bbc596d7f-fq45v\" (UID: \"ed1ba0e9-d432-4674-b52e-7110f348e98a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6bbc596d7f-fq45v"
Oct 30 21:31:41.368738 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:41.368715    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pj2cj\" (UniqueName: \"kubernetes.io/projected/ed1ba0e9-d432-4674-b52e-7110f348e98a-kube-api-access-pj2cj\") pod \"opendatahub-operator-controller-manager-6bbc596d7f-fq45v\" (UID: \"ed1ba0e9-d432-4674-b52e-7110f348e98a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6bbc596d7f-fq45v"
Oct 30 21:31:41.368809 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:41.368775    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ed1ba0e9-d432-4674-b52e-7110f348e98a-manifests\") pod \"opendatahub-operator-controller-manager-6bbc596d7f-fq45v\" (UID: \"ed1ba0e9-d432-4674-b52e-7110f348e98a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6bbc596d7f-fq45v"
Oct 30 21:31:41.393785 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:41.393756    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\""
Oct 30 21:31:41.394031 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:41.394014    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\""
Oct 30 21:31:41.395949 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:41.395921    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\""
Oct 30 21:31:41.395949 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:41.395935    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\""
Oct 30 21:31:41.396222 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:41.396205    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-jbnqm\""
Oct 30 21:31:41.396438 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:41.396423    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\""
Oct 30 21:31:41.469488 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:41.469453    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ed1ba0e9-d432-4674-b52e-7110f348e98a-tmp\") pod \"opendatahub-operator-controller-manager-6bbc596d7f-fq45v\" (UID: \"ed1ba0e9-d432-4674-b52e-7110f348e98a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6bbc596d7f-fq45v"
Oct 30 21:31:41.469488 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:41.469488    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ed1ba0e9-d432-4674-b52e-7110f348e98a-metrics-cert\") pod \"opendatahub-operator-controller-manager-6bbc596d7f-fq45v\" (UID: \"ed1ba0e9-d432-4674-b52e-7110f348e98a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6bbc596d7f-fq45v"
Oct 30 21:31:41.469736 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:41.469711    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ed1ba0e9-d432-4674-b52e-7110f348e98a-webhook-cert\") pod \"opendatahub-operator-controller-manager-6bbc596d7f-fq45v\" (UID: \"ed1ba0e9-d432-4674-b52e-7110f348e98a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6bbc596d7f-fq45v"
Oct 30 21:31:41.469801 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:41.469782    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pj2cj\" (UniqueName: \"kubernetes.io/projected/ed1ba0e9-d432-4674-b52e-7110f348e98a-kube-api-access-pj2cj\") pod \"opendatahub-operator-controller-manager-6bbc596d7f-fq45v\" (UID: \"ed1ba0e9-d432-4674-b52e-7110f348e98a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6bbc596d7f-fq45v"
Oct 30 21:31:41.469869 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:41.469838    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ed1ba0e9-d432-4674-b52e-7110f348e98a-manifests\") pod \"opendatahub-operator-controller-manager-6bbc596d7f-fq45v\" (UID: \"ed1ba0e9-d432-4674-b52e-7110f348e98a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6bbc596d7f-fq45v"
Oct 30 21:31:41.469979 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:41.469878    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ed1ba0e9-d432-4674-b52e-7110f348e98a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6bbc596d7f-fq45v\" (UID: \"ed1ba0e9-d432-4674-b52e-7110f348e98a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6bbc596d7f-fq45v"
Oct 30 21:31:41.470380 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:41.470340    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ed1ba0e9-d432-4674-b52e-7110f348e98a-manifests\") pod \"opendatahub-operator-controller-manager-6bbc596d7f-fq45v\" (UID: \"ed1ba0e9-d432-4674-b52e-7110f348e98a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6bbc596d7f-fq45v"
Oct 30 21:31:41.471958 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:41.471932    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ed1ba0e9-d432-4674-b52e-7110f348e98a-tmp\") pod \"opendatahub-operator-controller-manager-6bbc596d7f-fq45v\" (UID: \"ed1ba0e9-d432-4674-b52e-7110f348e98a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6bbc596d7f-fq45v"
Oct 30 21:31:41.472281 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:41.472261    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ed1ba0e9-d432-4674-b52e-7110f348e98a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6bbc596d7f-fq45v\" (UID: \"ed1ba0e9-d432-4674-b52e-7110f348e98a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6bbc596d7f-fq45v"
Oct 30 21:31:41.472281 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:41.472271    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ed1ba0e9-d432-4674-b52e-7110f348e98a-metrics-cert\") pod \"opendatahub-operator-controller-manager-6bbc596d7f-fq45v\" (UID: \"ed1ba0e9-d432-4674-b52e-7110f348e98a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6bbc596d7f-fq45v"
Oct 30 21:31:41.472430 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:41.472409    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ed1ba0e9-d432-4674-b52e-7110f348e98a-webhook-cert\") pod \"opendatahub-operator-controller-manager-6bbc596d7f-fq45v\" (UID: \"ed1ba0e9-d432-4674-b52e-7110f348e98a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6bbc596d7f-fq45v"
Oct 30 21:31:41.485767 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:41.485740    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pj2cj\" (UniqueName: \"kubernetes.io/projected/ed1ba0e9-d432-4674-b52e-7110f348e98a-kube-api-access-pj2cj\") pod \"opendatahub-operator-controller-manager-6bbc596d7f-fq45v\" (UID: \"ed1ba0e9-d432-4674-b52e-7110f348e98a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6bbc596d7f-fq45v"
Oct 30 21:31:41.636840 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:41.636755    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6bbc596d7f-fq45v"
Oct 30 21:31:41.938898 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:41.938870    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-6bbc596d7f-fq45v"]
Oct 30 21:31:41.940287 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:31:41.940261    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded1ba0e9_d432_4674_b52e_7110f348e98a.slice/crio-737fbccdd6affbc7850086381bb0f80429d9e10ef4e81471612f73f823daf030 WatchSource:0}: Error finding container 737fbccdd6affbc7850086381bb0f80429d9e10ef4e81471612f73f823daf030: Status 404 returned error can't find the container with id 737fbccdd6affbc7850086381bb0f80429d9e10ef4e81471612f73f823daf030
Oct 30 21:31:42.420275 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:42.420242    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6bbc596d7f-fq45v" event={"ID":"ed1ba0e9-d432-4674-b52e-7110f348e98a","Type":"ContainerStarted","Data":"737fbccdd6affbc7850086381bb0f80429d9e10ef4e81471612f73f823daf030"}
Oct 30 21:31:45.436179 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:45.436144    5032 generic.go:356] "Generic (PLEG): container finished" podID="ed1ba0e9-d432-4674-b52e-7110f348e98a" containerID="e5dfae87c02b7d548a4e66a3fb3d375dc438eb091c64fe5709a6a7681483b19e" exitCode=0
Oct 30 21:31:45.436587 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:45.436193    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6bbc596d7f-fq45v" event={"ID":"ed1ba0e9-d432-4674-b52e-7110f348e98a","Type":"ContainerDied","Data":"e5dfae87c02b7d548a4e66a3fb3d375dc438eb091c64fe5709a6a7681483b19e"}
Oct 30 21:31:46.442875 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:46.442838    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6bbc596d7f-fq45v" event={"ID":"ed1ba0e9-d432-4674-b52e-7110f348e98a","Type":"ContainerStarted","Data":"bd9338743bf86701db59d71215f15e3c3d619ce37d849190d1304ce529621ed6"}
Oct 30 21:31:46.443422 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:46.442962    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6bbc596d7f-fq45v"
Oct 30 21:31:46.491946 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:46.491885    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6bbc596d7f-fq45v" podStartSLOduration=1.632970653 podStartE2EDuration="5.49186922s" totalImagesPullingTime="3.306010057s" totalInitContainerRuntime="552.88851ms" isStatefulPod=false podCreationTimestamp="2026-10-30 21:31:41 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:31:46.490112988 +0000 UTC m=+219.291925452" watchObservedRunningTime="2026-10-30 21:31:46.49186922 +0000 UTC m=+219.293681683"
Oct 30 21:31:48.343765 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:31:48.343682    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 21:31:51.803414 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:51.803382    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-pctfn"]
Oct 30 21:31:51.808890 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:51.808866    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-pctfn"
Oct 30 21:31:51.816917 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:51.816895    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-c2pv4\""
Oct 30 21:31:51.822265 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:51.822241    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Oct 30 21:31:51.822417 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:51.822280    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:31:51.822948 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:51.822927    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-pctfn"]
Oct 30 21:31:51.960296 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:51.960255    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjw5x\" (UniqueName: \"kubernetes.io/projected/2e05c908-4707-42e5-9e1e-cb36732f5a6a-kube-api-access-mjw5x\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-pctfn\" (UID: \"2e05c908-4707-42e5-9e1e-cb36732f5a6a\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-pctfn"
Oct 30 21:31:51.960504 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:51.960405    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2e05c908-4707-42e5-9e1e-cb36732f5a6a-tmp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-pctfn\" (UID: \"2e05c908-4707-42e5-9e1e-cb36732f5a6a\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-pctfn"
Oct 30 21:31:52.061542 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:52.061464    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2e05c908-4707-42e5-9e1e-cb36732f5a6a-tmp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-pctfn\" (UID: \"2e05c908-4707-42e5-9e1e-cb36732f5a6a\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-pctfn"
Oct 30 21:31:52.061542 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:52.061508    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjw5x\" (UniqueName: \"kubernetes.io/projected/2e05c908-4707-42e5-9e1e-cb36732f5a6a-kube-api-access-mjw5x\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-pctfn\" (UID: \"2e05c908-4707-42e5-9e1e-cb36732f5a6a\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-pctfn"
Oct 30 21:31:52.061917 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:52.061896    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2e05c908-4707-42e5-9e1e-cb36732f5a6a-tmp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-pctfn\" (UID: \"2e05c908-4707-42e5-9e1e-cb36732f5a6a\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-pctfn"
Oct 30 21:31:52.074855 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:52.074832    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjw5x\" (UniqueName: \"kubernetes.io/projected/2e05c908-4707-42e5-9e1e-cb36732f5a6a-kube-api-access-mjw5x\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-pctfn\" (UID: \"2e05c908-4707-42e5-9e1e-cb36732f5a6a\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-pctfn"
Oct 30 21:31:52.120831 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:52.120797    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-pctfn"
Oct 30 21:31:52.406813 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:52.406784    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-pctfn"]
Oct 30 21:31:52.408584 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:31:52.408554    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e05c908_4707_42e5_9e1e_cb36732f5a6a.slice/crio-521572f39106b4be52c7599d07d84fff5618f8901af5123af50f2431a93767ac WatchSource:0}: Error finding container 521572f39106b4be52c7599d07d84fff5618f8901af5123af50f2431a93767ac: Status 404 returned error can't find the container with id 521572f39106b4be52c7599d07d84fff5618f8901af5123af50f2431a93767ac
Oct 30 21:31:52.467998 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:52.467962    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-pctfn" event={"ID":"2e05c908-4707-42e5-9e1e-cb36732f5a6a","Type":"ContainerStarted","Data":"521572f39106b4be52c7599d07d84fff5618f8901af5123af50f2431a93767ac"}
Oct 30 21:31:55.483283 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:55.483247    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-pctfn" event={"ID":"2e05c908-4707-42e5-9e1e-cb36732f5a6a","Type":"ContainerStarted","Data":"7bf48992c8af553b1359376ecc0fa5b2176cebfa9f2871bff9525b63ba3ba4a1"}
Oct 30 21:31:55.516432 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:55.516332    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-pctfn" podStartSLOduration=1.518635771 podStartE2EDuration="4.516313627s" totalImagesPullingTime="2.997677856s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:31:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:31:55.512943876 +0000 UTC m=+228.314756340" watchObservedRunningTime="2026-10-30 21:31:55.516313627 +0000 UTC m=+228.318126091"
Oct 30 21:31:57.449616 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:31:57.449583    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6bbc596d7f-fq45v"
Oct 30 21:32:01.269965 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:01.269931    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-8557cbdb84-xtcms"]
Oct 30 21:32:01.273410 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:01.273392    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-xtcms"
Oct 30 21:32:01.283453 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:01.283404    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Oct 30 21:32:01.283564 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:01.283461    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-qnc75\""
Oct 30 21:32:01.283564 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:01.283441    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Oct 30 21:32:01.283564 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:01.283512    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Oct 30 21:32:01.290054 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:01.290029    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-8557cbdb84-xtcms"]
Oct 30 21:32:01.445912 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:01.445858    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dknth\" (UniqueName: \"kubernetes.io/projected/bd9151d1-084d-4737-b9ce-cbac709e23c1-kube-api-access-dknth\") pod \"dns-operator-controller-manager-8557cbdb84-xtcms\" (UID: \"bd9151d1-084d-4737-b9ce-cbac709e23c1\") " pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-xtcms"
Oct 30 21:32:01.546758 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:01.546671    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dknth\" (UniqueName: \"kubernetes.io/projected/bd9151d1-084d-4737-b9ce-cbac709e23c1-kube-api-access-dknth\") pod \"dns-operator-controller-manager-8557cbdb84-xtcms\" (UID: \"bd9151d1-084d-4737-b9ce-cbac709e23c1\") " pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-xtcms"
Oct 30 21:32:01.564250 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:01.564213    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dknth\" (UniqueName: \"kubernetes.io/projected/bd9151d1-084d-4737-b9ce-cbac709e23c1-kube-api-access-dknth\") pod \"dns-operator-controller-manager-8557cbdb84-xtcms\" (UID: \"bd9151d1-084d-4737-b9ce-cbac709e23c1\") " pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-xtcms"
Oct 30 21:32:01.586106 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:01.586071    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-xtcms"
Oct 30 21:32:01.865994 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:01.865905    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-8557cbdb84-xtcms"]
Oct 30 21:32:01.869623 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:32:01.869593    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd9151d1_084d_4737_b9ce_cbac709e23c1.slice/crio-69b01f540827df6f8ca1819f84e9e48ed97e28cdf6537fa27ab77a1e29860003 WatchSource:0}: Error finding container 69b01f540827df6f8ca1819f84e9e48ed97e28cdf6537fa27ab77a1e29860003: Status 404 returned error can't find the container with id 69b01f540827df6f8ca1819f84e9e48ed97e28cdf6537fa27ab77a1e29860003
Oct 30 21:32:02.514215 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:02.514177    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-xtcms" event={"ID":"bd9151d1-084d-4737-b9ce-cbac709e23c1","Type":"ContainerStarted","Data":"69b01f540827df6f8ca1819f84e9e48ed97e28cdf6537fa27ab77a1e29860003"}
Oct 30 21:32:05.530611 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:05.530574    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-xtcms" event={"ID":"bd9151d1-084d-4737-b9ce-cbac709e23c1","Type":"ContainerStarted","Data":"faf069600fdee48955b1853abe5866b676e33f5098e92b9c081baf6e19e9a851"}
Oct 30 21:32:05.531038 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:05.530660    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-xtcms"
Oct 30 21:32:05.554159 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:05.554098    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-xtcms" podStartSLOduration=1.558072873 podStartE2EDuration="4.554083596s" totalImagesPullingTime="2.996010723s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:32:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:32:05.552773245 +0000 UTC m=+238.354585709" watchObservedRunningTime="2026-10-30 21:32:05.554083596 +0000 UTC m=+238.355896054"
Oct 30 21:32:07.372387 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:07.372345    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-597b95cb84-s5vsn"]
Oct 30 21:32:07.377088 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:07.377064    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-s5vsn"
Oct 30 21:32:07.386576 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:07.386555    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Oct 30 21:32:07.386731 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:07.386604    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-wtw92\""
Oct 30 21:32:07.386832 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:07.386802    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Oct 30 21:32:07.388197 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:07.388178    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-s5vsn"]
Oct 30 21:32:07.499302 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:07.499268    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8be8ce8b-7166-4530-8a24-c13f067797c8-bound-sa-token\") pod \"cert-manager-597b95cb84-s5vsn\" (UID: \"8be8ce8b-7166-4530-8a24-c13f067797c8\") " pod="cert-manager/cert-manager-597b95cb84-s5vsn"
Oct 30 21:32:07.499486 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:07.499315    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnt66\" (UniqueName: \"kubernetes.io/projected/8be8ce8b-7166-4530-8a24-c13f067797c8-kube-api-access-xnt66\") pod \"cert-manager-597b95cb84-s5vsn\" (UID: \"8be8ce8b-7166-4530-8a24-c13f067797c8\") " pod="cert-manager/cert-manager-597b95cb84-s5vsn"
Oct 30 21:32:07.600320 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:07.600285    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8be8ce8b-7166-4530-8a24-c13f067797c8-bound-sa-token\") pod \"cert-manager-597b95cb84-s5vsn\" (UID: \"8be8ce8b-7166-4530-8a24-c13f067797c8\") " pod="cert-manager/cert-manager-597b95cb84-s5vsn"
Oct 30 21:32:07.600528 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:07.600332    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnt66\" (UniqueName: \"kubernetes.io/projected/8be8ce8b-7166-4530-8a24-c13f067797c8-kube-api-access-xnt66\") pod \"cert-manager-597b95cb84-s5vsn\" (UID: \"8be8ce8b-7166-4530-8a24-c13f067797c8\") " pod="cert-manager/cert-manager-597b95cb84-s5vsn"
Oct 30 21:32:07.613342 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:07.613306    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8be8ce8b-7166-4530-8a24-c13f067797c8-bound-sa-token\") pod \"cert-manager-597b95cb84-s5vsn\" (UID: \"8be8ce8b-7166-4530-8a24-c13f067797c8\") " pod="cert-manager/cert-manager-597b95cb84-s5vsn"
Oct 30 21:32:07.613537 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:07.613512    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnt66\" (UniqueName: \"kubernetes.io/projected/8be8ce8b-7166-4530-8a24-c13f067797c8-kube-api-access-xnt66\") pod \"cert-manager-597b95cb84-s5vsn\" (UID: \"8be8ce8b-7166-4530-8a24-c13f067797c8\") " pod="cert-manager/cert-manager-597b95cb84-s5vsn"
Oct 30 21:32:07.699615 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:07.699523    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-s5vsn"
Oct 30 21:32:07.974458 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:07.974433    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-s5vsn"]
Oct 30 21:32:07.976388 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:32:07.976334    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8be8ce8b_7166_4530_8a24_c13f067797c8.slice/crio-acdb37fdb8b08be9c72dbcff69e2897478472fac642185fff935dddc70114163 WatchSource:0}: Error finding container acdb37fdb8b08be9c72dbcff69e2897478472fac642185fff935dddc70114163: Status 404 returned error can't find the container with id acdb37fdb8b08be9c72dbcff69e2897478472fac642185fff935dddc70114163
Oct 30 21:32:08.524261 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:08.524228    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-z4j7c"]
Oct 30 21:32:08.528983 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:08.528965    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-z4j7c"
Oct 30 21:32:08.536195 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:08.536172    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-xn5tn\""
Oct 30 21:32:08.541006 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:08.540982    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-z4j7c"]
Oct 30 21:32:08.545626 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:08.545602    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-s5vsn" event={"ID":"8be8ce8b-7166-4530-8a24-c13f067797c8","Type":"ContainerStarted","Data":"acdb37fdb8b08be9c72dbcff69e2897478472fac642185fff935dddc70114163"}
Oct 30 21:32:08.710630 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:08.710587    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/14250b14-8ba8-4e5d-a20d-09fb6a0b3b9e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-cf8469d7b-z4j7c\" (UID: \"14250b14-8ba8-4e5d-a20d-09fb6a0b3b9e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-z4j7c"
Oct 30 21:32:08.710807 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:08.710715    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kz4lj\" (UniqueName: \"kubernetes.io/projected/14250b14-8ba8-4e5d-a20d-09fb6a0b3b9e-kube-api-access-kz4lj\") pod \"kuadrant-operator-controller-manager-cf8469d7b-z4j7c\" (UID: \"14250b14-8ba8-4e5d-a20d-09fb6a0b3b9e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-z4j7c"
Oct 30 21:32:08.811215 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:08.811122    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kz4lj\" (UniqueName: \"kubernetes.io/projected/14250b14-8ba8-4e5d-a20d-09fb6a0b3b9e-kube-api-access-kz4lj\") pod \"kuadrant-operator-controller-manager-cf8469d7b-z4j7c\" (UID: \"14250b14-8ba8-4e5d-a20d-09fb6a0b3b9e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-z4j7c"
Oct 30 21:32:08.811215 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:08.811185    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/14250b14-8ba8-4e5d-a20d-09fb6a0b3b9e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-cf8469d7b-z4j7c\" (UID: \"14250b14-8ba8-4e5d-a20d-09fb6a0b3b9e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-z4j7c"
Oct 30 21:32:08.811559 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:08.811541    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/14250b14-8ba8-4e5d-a20d-09fb6a0b3b9e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-cf8469d7b-z4j7c\" (UID: \"14250b14-8ba8-4e5d-a20d-09fb6a0b3b9e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-z4j7c"
Oct 30 21:32:08.826797 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:08.826770    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kz4lj\" (UniqueName: \"kubernetes.io/projected/14250b14-8ba8-4e5d-a20d-09fb6a0b3b9e-kube-api-access-kz4lj\") pod \"kuadrant-operator-controller-manager-cf8469d7b-z4j7c\" (UID: \"14250b14-8ba8-4e5d-a20d-09fb6a0b3b9e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-z4j7c"
Oct 30 21:32:08.841658 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:08.841628    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-z4j7c"
Oct 30 21:32:09.140275 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:09.140167    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-z4j7c"]
Oct 30 21:32:09.553026 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:09.552991    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-z4j7c" event={"ID":"14250b14-8ba8-4e5d-a20d-09fb6a0b3b9e","Type":"ContainerStarted","Data":"8751c5831339a6a8f145d37b587cd282f359b37add4998b18f56755a4148876b"}
Oct 30 21:32:11.565379 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:11.565321    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-s5vsn" event={"ID":"8be8ce8b-7166-4530-8a24-c13f067797c8","Type":"ContainerStarted","Data":"f03e5e7ef77c97ce677fec7e67bb176fc9afbafa57b9f96ef9568cae484e48ce"}
Oct 30 21:32:11.589635 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:11.589577    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-597b95cb84-s5vsn" podStartSLOduration=1.4070572129999999 podStartE2EDuration="4.589562662s" totalImagesPullingTime="3.182505449s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:32:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:32:11.58839619 +0000 UTC m=+244.390208673" watchObservedRunningTime="2026-10-30 21:32:11.589562662 +0000 UTC m=+244.391375126"
Oct 30 21:32:12.083905 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:12.083875    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-5778bb4449-m85ql"]
Oct 30 21:32:12.087618 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:12.087593    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-m85ql"
Oct 30 21:32:12.100738 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:12.099830    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-5778bb4449-m85ql"]
Oct 30 21:32:12.103326 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:12.103299    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-c57kx\""
Oct 30 21:32:12.243745 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:12.243699    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbfjr\" (UniqueName: \"kubernetes.io/projected/0900d338-ad69-49e6-b23d-185def51a7cb-kube-api-access-vbfjr\") pod \"limitador-operator-controller-manager-5778bb4449-m85ql\" (UID: \"0900d338-ad69-49e6-b23d-185def51a7cb\") " pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-m85ql"
Oct 30 21:32:12.345147 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:12.345069    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbfjr\" (UniqueName: \"kubernetes.io/projected/0900d338-ad69-49e6-b23d-185def51a7cb-kube-api-access-vbfjr\") pod \"limitador-operator-controller-manager-5778bb4449-m85ql\" (UID: \"0900d338-ad69-49e6-b23d-185def51a7cb\") " pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-m85ql"
Oct 30 21:32:12.360119 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:12.360090    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbfjr\" (UniqueName: \"kubernetes.io/projected/0900d338-ad69-49e6-b23d-185def51a7cb-kube-api-access-vbfjr\") pod \"limitador-operator-controller-manager-5778bb4449-m85ql\" (UID: \"0900d338-ad69-49e6-b23d-185def51a7cb\") " pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-m85ql"
Oct 30 21:32:12.403897 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:12.403863    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-m85ql"
Oct 30 21:32:12.702085 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:12.702056    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-5778bb4449-m85ql"]
Oct 30 21:32:12.709042 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:32:12.709008    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0900d338_ad69_49e6_b23d_185def51a7cb.slice/crio-6c88c6dfe878c5babeb0cb072f26c8e90118a1f69ad70ee95826e563e593fd8a WatchSource:0}: Error finding container 6c88c6dfe878c5babeb0cb072f26c8e90118a1f69ad70ee95826e563e593fd8a: Status 404 returned error can't find the container with id 6c88c6dfe878c5babeb0cb072f26c8e90118a1f69ad70ee95826e563e593fd8a
Oct 30 21:32:13.580000 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:13.579961    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-z4j7c" event={"ID":"14250b14-8ba8-4e5d-a20d-09fb6a0b3b9e","Type":"ContainerStarted","Data":"08ee4f975ce5911c93a66b483b0038a64e99a654eba739c2570f04817e2fb791"}
Oct 30 21:32:13.580243 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:13.580067    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-z4j7c"
Oct 30 21:32:13.581505 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:13.581471    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-m85ql" event={"ID":"0900d338-ad69-49e6-b23d-185def51a7cb","Type":"ContainerStarted","Data":"6c88c6dfe878c5babeb0cb072f26c8e90118a1f69ad70ee95826e563e593fd8a"}
Oct 30 21:32:13.609860 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:13.609806    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-z4j7c" podStartSLOduration=1.36838324 podStartE2EDuration="5.609788691s" totalImagesPullingTime="4.241405451s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:32:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:32:13.608451586 +0000 UTC m=+246.410264050" watchObservedRunningTime="2026-10-30 21:32:13.609788691 +0000 UTC m=+246.411601159"
Oct 30 21:32:14.182326 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:14.182291    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6b748b65852wbfw"]
Oct 30 21:32:14.186137 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:14.186110    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6b748b65852wbfw"
Oct 30 21:32:14.199975 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:14.199945    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-nf9d2\""
Oct 30 21:32:14.213185 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:14.213151    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6b748b65852wbfw"]
Oct 30 21:32:14.362844 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:14.362807    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f48d971-e68f-4bb2-8c83-0f2acf541950-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6b748b65852wbfw\" (UID: \"3f48d971-e68f-4bb2-8c83-0f2acf541950\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6b748b65852wbfw"
Oct 30 21:32:14.363084 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:14.362853    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f48d971-e68f-4bb2-8c83-0f2acf541950-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6b748b65852wbfw\" (UID: \"3f48d971-e68f-4bb2-8c83-0f2acf541950\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6b748b65852wbfw"
Oct 30 21:32:14.363084 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:14.362961    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f48d971-e68f-4bb2-8c83-0f2acf541950-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6b748b65852wbfw\" (UID: \"3f48d971-e68f-4bb2-8c83-0f2acf541950\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6b748b65852wbfw"
Oct 30 21:32:14.363084 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:14.363007    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f48d971-e68f-4bb2-8c83-0f2acf541950-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6b748b65852wbfw\" (UID: \"3f48d971-e68f-4bb2-8c83-0f2acf541950\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6b748b65852wbfw"
Oct 30 21:32:14.363231 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:14.363139    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w74xr\" (UniqueName: \"kubernetes.io/projected/3f48d971-e68f-4bb2-8c83-0f2acf541950-kube-api-access-w74xr\") pod \"data-science-gateway-data-science-gateway-class-6b748b65852wbfw\" (UID: \"3f48d971-e68f-4bb2-8c83-0f2acf541950\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6b748b65852wbfw"
Oct 30 21:32:14.363231 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:14.363177    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f48d971-e68f-4bb2-8c83-0f2acf541950-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6b748b65852wbfw\" (UID: \"3f48d971-e68f-4bb2-8c83-0f2acf541950\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6b748b65852wbfw"
Oct 30 21:32:14.363231 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:14.363208    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f48d971-e68f-4bb2-8c83-0f2acf541950-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6b748b65852wbfw\" (UID: \"3f48d971-e68f-4bb2-8c83-0f2acf541950\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6b748b65852wbfw"
Oct 30 21:32:14.363353 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:14.363265    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f48d971-e68f-4bb2-8c83-0f2acf541950-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6b748b65852wbfw\" (UID: \"3f48d971-e68f-4bb2-8c83-0f2acf541950\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6b748b65852wbfw"
Oct 30 21:32:14.363353 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:14.363347    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f48d971-e68f-4bb2-8c83-0f2acf541950-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6b748b65852wbfw\" (UID: \"3f48d971-e68f-4bb2-8c83-0f2acf541950\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6b748b65852wbfw"
Oct 30 21:32:14.464111 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:14.464079    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f48d971-e68f-4bb2-8c83-0f2acf541950-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6b748b65852wbfw\" (UID: \"3f48d971-e68f-4bb2-8c83-0f2acf541950\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6b748b65852wbfw"
Oct 30 21:32:14.464302 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:14.464137    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f48d971-e68f-4bb2-8c83-0f2acf541950-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6b748b65852wbfw\" (UID: \"3f48d971-e68f-4bb2-8c83-0f2acf541950\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6b748b65852wbfw"
Oct 30 21:32:14.464302 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:14.464174    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f48d971-e68f-4bb2-8c83-0f2acf541950-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6b748b65852wbfw\" (UID: \"3f48d971-e68f-4bb2-8c83-0f2acf541950\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6b748b65852wbfw"
Oct 30 21:32:14.464302 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:14.464240    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f48d971-e68f-4bb2-8c83-0f2acf541950-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6b748b65852wbfw\" (UID: \"3f48d971-e68f-4bb2-8c83-0f2acf541950\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6b748b65852wbfw"
Oct 30 21:32:14.464302 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:14.464263    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f48d971-e68f-4bb2-8c83-0f2acf541950-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6b748b65852wbfw\" (UID: \"3f48d971-e68f-4bb2-8c83-0f2acf541950\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6b748b65852wbfw"
Oct 30 21:32:14.464302 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:14.464284    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f48d971-e68f-4bb2-8c83-0f2acf541950-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6b748b65852wbfw\" (UID: \"3f48d971-e68f-4bb2-8c83-0f2acf541950\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6b748b65852wbfw"
Oct 30 21:32:14.464302 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:14.464302    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f48d971-e68f-4bb2-8c83-0f2acf541950-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6b748b65852wbfw\" (UID: \"3f48d971-e68f-4bb2-8c83-0f2acf541950\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6b748b65852wbfw"
Oct 30 21:32:14.464649 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:14.464341    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w74xr\" (UniqueName: \"kubernetes.io/projected/3f48d971-e68f-4bb2-8c83-0f2acf541950-kube-api-access-w74xr\") pod \"data-science-gateway-data-science-gateway-class-6b748b65852wbfw\" (UID: \"3f48d971-e68f-4bb2-8c83-0f2acf541950\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6b748b65852wbfw"
Oct 30 21:32:14.464649 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:14.464387    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f48d971-e68f-4bb2-8c83-0f2acf541950-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6b748b65852wbfw\" (UID: \"3f48d971-e68f-4bb2-8c83-0f2acf541950\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6b748b65852wbfw"
Oct 30 21:32:14.464649 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:14.464598    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f48d971-e68f-4bb2-8c83-0f2acf541950-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6b748b65852wbfw\" (UID: \"3f48d971-e68f-4bb2-8c83-0f2acf541950\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6b748b65852wbfw"
Oct 30 21:32:14.464791 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:14.464708    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f48d971-e68f-4bb2-8c83-0f2acf541950-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6b748b65852wbfw\" (UID: \"3f48d971-e68f-4bb2-8c83-0f2acf541950\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6b748b65852wbfw"
Oct 30 21:32:14.464791 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:14.464744    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f48d971-e68f-4bb2-8c83-0f2acf541950-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6b748b65852wbfw\" (UID: \"3f48d971-e68f-4bb2-8c83-0f2acf541950\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6b748b65852wbfw"
Oct 30 21:32:14.464922 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:14.464893    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f48d971-e68f-4bb2-8c83-0f2acf541950-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6b748b65852wbfw\" (UID: \"3f48d971-e68f-4bb2-8c83-0f2acf541950\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6b748b65852wbfw"
Oct 30 21:32:14.465328 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:14.465307    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f48d971-e68f-4bb2-8c83-0f2acf541950-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6b748b65852wbfw\" (UID: \"3f48d971-e68f-4bb2-8c83-0f2acf541950\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6b748b65852wbfw"
Oct 30 21:32:14.467603 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:14.467578    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f48d971-e68f-4bb2-8c83-0f2acf541950-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6b748b65852wbfw\" (UID: \"3f48d971-e68f-4bb2-8c83-0f2acf541950\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6b748b65852wbfw"
Oct 30 21:32:14.475177 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:14.475145    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f48d971-e68f-4bb2-8c83-0f2acf541950-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6b748b65852wbfw\" (UID: \"3f48d971-e68f-4bb2-8c83-0f2acf541950\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6b748b65852wbfw"
Oct 30 21:32:14.498623 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:14.498569    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f48d971-e68f-4bb2-8c83-0f2acf541950-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6b748b65852wbfw\" (UID: \"3f48d971-e68f-4bb2-8c83-0f2acf541950\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6b748b65852wbfw"
Oct 30 21:32:14.499229 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:14.499206    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w74xr\" (UniqueName: \"kubernetes.io/projected/3f48d971-e68f-4bb2-8c83-0f2acf541950-kube-api-access-w74xr\") pod \"data-science-gateway-data-science-gateway-class-6b748b65852wbfw\" (UID: \"3f48d971-e68f-4bb2-8c83-0f2acf541950\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6b748b65852wbfw"
Oct 30 21:32:14.505048 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:14.505020    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6b748b65852wbfw"
Oct 30 21:32:14.833817 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:14.833792    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6b748b65852wbfw"]
Oct 30 21:32:14.836089 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:32:14.836062    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f48d971_e68f_4bb2_8c83_0f2acf541950.slice/crio-af2127734cb2a415af8fee0ed983c142a28777c9b7759b407527f1ed7bc52de5 WatchSource:0}: Error finding container af2127734cb2a415af8fee0ed983c142a28777c9b7759b407527f1ed7bc52de5: Status 404 returned error can't find the container with id af2127734cb2a415af8fee0ed983c142a28777c9b7759b407527f1ed7bc52de5
Oct 30 21:32:14.838453 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:14.838418    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:32:14.838546 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:14.838487    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:32:14.838546 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:14.838518    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:32:15.592737 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:15.592693    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-m85ql" event={"ID":"0900d338-ad69-49e6-b23d-185def51a7cb","Type":"ContainerStarted","Data":"0a3793ea82e37455621d6350f4d1f59ac61208ce7b6d525af3bdccf3a0df0e9e"}
Oct 30 21:32:15.593433 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:15.593409    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-m85ql"
Oct 30 21:32:15.600213 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:15.600187    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6b748b65852wbfw" event={"ID":"3f48d971-e68f-4bb2-8c83-0f2acf541950","Type":"ContainerStarted","Data":"e13692c63ac542c240d96d54fdd781afcc323b1a6da3f76bc39e48eb28c3b274"}
Oct 30 21:32:15.600342 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:15.600234    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6b748b65852wbfw" event={"ID":"3f48d971-e68f-4bb2-8c83-0f2acf541950","Type":"ContainerStarted","Data":"af2127734cb2a415af8fee0ed983c142a28777c9b7759b407527f1ed7bc52de5"}
Oct 30 21:32:15.623024 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:15.622974    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-m85ql" podStartSLOduration=1.7144515820000001 podStartE2EDuration="3.622961457s" totalImagesPullingTime="1.908509875s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:32:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:32:15.62169193 +0000 UTC m=+248.423504398" watchObservedRunningTime="2026-10-30 21:32:15.622961457 +0000 UTC m=+248.424773921"
Oct 30 21:32:15.651483 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:15.651436    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6b748b65852wbfw" podStartSLOduration=1.651421986 podStartE2EDuration="1.651421986s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:32:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:32:15.650580033 +0000 UTC m=+248.452392499" watchObservedRunningTime="2026-10-30 21:32:15.651421986 +0000 UTC m=+248.453234444"
Oct 30 21:32:16.505620 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:16.505589    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6b748b65852wbfw"
Oct 30 21:32:16.510382 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:16.510338    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6b748b65852wbfw"
Oct 30 21:32:16.537266 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:16.537239    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-xtcms"
Oct 30 21:32:16.605570 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:16.605539    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6b748b65852wbfw"
Oct 30 21:32:16.606605 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:16.606579    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6b748b65852wbfw"
Oct 30 21:32:18.346326 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:32:18.346281    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 21:32:18.585122 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:18.585087    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-6nnzj"]
Oct 30 21:32:18.632083 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:18.632010    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-6nnzj"]
Oct 30 21:32:18.632223 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:18.632104    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-6nnzj"
Oct 30 21:32:18.638552 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:18.638527    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Oct 30 21:32:18.643823 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:18.643797    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"odh-trusted-ca-bundle\""
Oct 30 21:32:18.643823 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:18.643822    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-module-controller-manager-dockercfg-6r4c5\""
Oct 30 21:32:18.739544 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:18.739514    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Oct 30 21:32:18.805146 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:18.805105    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c369fb18-590a-4da3-93ff-f0da6ae518e2-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-6nnzj\" (UID: \"c369fb18-590a-4da3-93ff-f0da6ae518e2\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-6nnzj"
Oct 30 21:32:18.805353 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:18.805167    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c369fb18-590a-4da3-93ff-f0da6ae518e2-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-6nnzj\" (UID: \"c369fb18-590a-4da3-93ff-f0da6ae518e2\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-6nnzj"
Oct 30 21:32:18.805353 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:18.805293    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c369fb18-590a-4da3-93ff-f0da6ae518e2-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-6nnzj\" (UID: \"c369fb18-590a-4da3-93ff-f0da6ae518e2\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-6nnzj"
Oct 30 21:32:18.805477 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:18.805398    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcl8x\" (UniqueName: \"kubernetes.io/projected/c369fb18-590a-4da3-93ff-f0da6ae518e2-kube-api-access-gcl8x\") pod \"kserve-module-controller-manager-7566cf5fc8-6nnzj\" (UID: \"c369fb18-590a-4da3-93ff-f0da6ae518e2\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-6nnzj"
Oct 30 21:32:18.906596 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:18.906515    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c369fb18-590a-4da3-93ff-f0da6ae518e2-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-6nnzj\" (UID: \"c369fb18-590a-4da3-93ff-f0da6ae518e2\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-6nnzj"
Oct 30 21:32:18.906596 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:18.906561    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c369fb18-590a-4da3-93ff-f0da6ae518e2-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-6nnzj\" (UID: \"c369fb18-590a-4da3-93ff-f0da6ae518e2\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-6nnzj"
Oct 30 21:32:18.906596 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:18.906580    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcl8x\" (UniqueName: \"kubernetes.io/projected/c369fb18-590a-4da3-93ff-f0da6ae518e2-kube-api-access-gcl8x\") pod \"kserve-module-controller-manager-7566cf5fc8-6nnzj\" (UID: \"c369fb18-590a-4da3-93ff-f0da6ae518e2\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-6nnzj"
Oct 30 21:32:18.906830 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:18.906650    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c369fb18-590a-4da3-93ff-f0da6ae518e2-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-6nnzj\" (UID: \"c369fb18-590a-4da3-93ff-f0da6ae518e2\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-6nnzj"
Oct 30 21:32:18.907072 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:18.907056    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c369fb18-590a-4da3-93ff-f0da6ae518e2-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-6nnzj\" (UID: \"c369fb18-590a-4da3-93ff-f0da6ae518e2\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-6nnzj"
Oct 30 21:32:18.907114 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:18.907094    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c369fb18-590a-4da3-93ff-f0da6ae518e2-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-6nnzj\" (UID: \"c369fb18-590a-4da3-93ff-f0da6ae518e2\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-6nnzj"
Oct 30 21:32:18.907729 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:18.907711    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c369fb18-590a-4da3-93ff-f0da6ae518e2-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-6nnzj\" (UID: \"c369fb18-590a-4da3-93ff-f0da6ae518e2\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-6nnzj"
Oct 30 21:32:18.949540 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:18.949503    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcl8x\" (UniqueName: \"kubernetes.io/projected/c369fb18-590a-4da3-93ff-f0da6ae518e2-kube-api-access-gcl8x\") pod \"kserve-module-controller-manager-7566cf5fc8-6nnzj\" (UID: \"c369fb18-590a-4da3-93ff-f0da6ae518e2\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-6nnzj"
Oct 30 21:32:18.967875 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:18.967847    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-6nnzj"
Oct 30 21:32:19.282413 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:19.282386    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-6nnzj"]
Oct 30 21:32:19.285002 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:32:19.284975    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc369fb18_590a_4da3_93ff_f0da6ae518e2.slice/crio-6ff0e78a5c7a15106da2677a05154e3e92063f8fccdb478a6219658daa398e89 WatchSource:0}: Error finding container 6ff0e78a5c7a15106da2677a05154e3e92063f8fccdb478a6219658daa398e89: Status 404 returned error can't find the container with id 6ff0e78a5c7a15106da2677a05154e3e92063f8fccdb478a6219658daa398e89
Oct 30 21:32:19.623558 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:19.623477    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-6nnzj" event={"ID":"c369fb18-590a-4da3-93ff-f0da6ae518e2","Type":"ContainerStarted","Data":"6ff0e78a5c7a15106da2677a05154e3e92063f8fccdb478a6219658daa398e89"}
Oct 30 21:32:22.640794 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:22.640749    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-6nnzj" event={"ID":"c369fb18-590a-4da3-93ff-f0da6ae518e2","Type":"ContainerStarted","Data":"da9948e8a2fc88cef420150cdc77d29984165e0e685e1cfa42674444652f57ea"}
Oct 30 21:32:22.641583 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:22.641552    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-6nnzj"
Oct 30 21:32:22.663355 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:22.663301    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-6nnzj" podStartSLOduration=1.476338984 podStartE2EDuration="4.663289806s" totalImagesPullingTime="3.186950822s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:32:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:32:22.662442561 +0000 UTC m=+255.464255025" watchObservedRunningTime="2026-10-30 21:32:22.663289806 +0000 UTC m=+255.465102269"
Oct 30 21:32:24.588149 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:24.588119    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-z4j7c"
Oct 30 21:32:26.607476 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:26.607449    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-m85ql"
Oct 30 21:32:41.949555 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:41.949514    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-z4j7c"]
Oct 30 21:32:41.950083 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:41.949748    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-z4j7c" podUID="14250b14-8ba8-4e5d-a20d-09fb6a0b3b9e" containerName="manager" containerID="cri-o://08ee4f975ce5911c93a66b483b0038a64e99a654eba739c2570f04817e2fb791" gracePeriod=10
Oct 30 21:32:42.194920 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:42.194897    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-z4j7c"
Oct 30 21:32:42.309483 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:42.309450    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/14250b14-8ba8-4e5d-a20d-09fb6a0b3b9e-extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/14250b14-8ba8-4e5d-a20d-09fb6a0b3b9e-extensions-socket-volume\") pod \"14250b14-8ba8-4e5d-a20d-09fb6a0b3b9e\" (UID: \"14250b14-8ba8-4e5d-a20d-09fb6a0b3b9e\") "
Oct 30 21:32:42.309687 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:42.309532    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/14250b14-8ba8-4e5d-a20d-09fb6a0b3b9e-kube-api-access-kz4lj\" (UniqueName: \"kubernetes.io/projected/14250b14-8ba8-4e5d-a20d-09fb6a0b3b9e-kube-api-access-kz4lj\") pod \"14250b14-8ba8-4e5d-a20d-09fb6a0b3b9e\" (UID: \"14250b14-8ba8-4e5d-a20d-09fb6a0b3b9e\") "
Oct 30 21:32:42.309944 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:42.309919    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14250b14-8ba8-4e5d-a20d-09fb6a0b3b9e-extensions-socket-volume" pod "14250b14-8ba8-4e5d-a20d-09fb6a0b3b9e" (UID: "14250b14-8ba8-4e5d-a20d-09fb6a0b3b9e"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:42.311805 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:42.311782    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14250b14-8ba8-4e5d-a20d-09fb6a0b3b9e-kube-api-access-kz4lj" pod "14250b14-8ba8-4e5d-a20d-09fb6a0b3b9e" (UID: "14250b14-8ba8-4e5d-a20d-09fb6a0b3b9e"). InnerVolumeSpecName "kube-api-access-kz4lj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:32:42.411017 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:42.410973    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kz4lj\" (UniqueName: \"kubernetes.io/projected/14250b14-8ba8-4e5d-a20d-09fb6a0b3b9e-kube-api-access-kz4lj\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:42.411017 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:42.411012    5032 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/14250b14-8ba8-4e5d-a20d-09fb6a0b3b9e-extensions-socket-volume\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:42.739956 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:42.739915    5032 generic.go:356] "Generic (PLEG): container finished" podID="14250b14-8ba8-4e5d-a20d-09fb6a0b3b9e" containerID="08ee4f975ce5911c93a66b483b0038a64e99a654eba739c2570f04817e2fb791" exitCode=0
Oct 30 21:32:42.740158 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:42.739966    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-z4j7c"
Oct 30 21:32:42.740158 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:42.739998    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-z4j7c" event={"ID":"14250b14-8ba8-4e5d-a20d-09fb6a0b3b9e","Type":"ContainerDied","Data":"08ee4f975ce5911c93a66b483b0038a64e99a654eba739c2570f04817e2fb791"}
Oct 30 21:32:42.740158 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:42.740060    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-z4j7c" event={"ID":"14250b14-8ba8-4e5d-a20d-09fb6a0b3b9e","Type":"ContainerDied","Data":"8751c5831339a6a8f145d37b587cd282f359b37add4998b18f56755a4148876b"}
Oct 30 21:32:42.740158 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:42.740088    5032 scope.go:122] "RemoveContainer" containerID="08ee4f975ce5911c93a66b483b0038a64e99a654eba739c2570f04817e2fb791"
Oct 30 21:32:42.749933 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:42.749916    5032 scope.go:122] "RemoveContainer" containerID="08ee4f975ce5911c93a66b483b0038a64e99a654eba739c2570f04817e2fb791"
Oct 30 21:32:42.750181 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:32:42.750164    5032 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08ee4f975ce5911c93a66b483b0038a64e99a654eba739c2570f04817e2fb791\": container with ID starting with 08ee4f975ce5911c93a66b483b0038a64e99a654eba739c2570f04817e2fb791 not found: ID does not exist" containerID="08ee4f975ce5911c93a66b483b0038a64e99a654eba739c2570f04817e2fb791"
Oct 30 21:32:42.750230 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:42.750190    5032 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08ee4f975ce5911c93a66b483b0038a64e99a654eba739c2570f04817e2fb791"} err="failed to get container status \"08ee4f975ce5911c93a66b483b0038a64e99a654eba739c2570f04817e2fb791\": rpc error: code = NotFound desc = could not find container \"08ee4f975ce5911c93a66b483b0038a64e99a654eba739c2570f04817e2fb791\": container with ID starting with 08ee4f975ce5911c93a66b483b0038a64e99a654eba739c2570f04817e2fb791 not found: ID does not exist"
Oct 30 21:32:42.775736 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:42.775705    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-z4j7c"]
Oct 30 21:32:42.790954 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:42.790926    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-z4j7c"]
Oct 30 21:32:43.343426 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:43.343387    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="14250b14-8ba8-4e5d-a20d-09fb6a0b3b9e" path="/var/lib/kubelet/pods/14250b14-8ba8-4e5d-a20d-09fb6a0b3b9e/volumes"
Oct 30 21:32:44.652115 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:44.652079    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-6nnzj"
Oct 30 21:32:48.344212 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:32:48.344176    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 21:32:58.987916 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:58.987873    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-9md4d"]
Oct 30 21:32:58.988481 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:58.988462    5032 cpu_manager.go:414] "containerMap: removing container" podUID="14250b14-8ba8-4e5d-a20d-09fb6a0b3b9e" containerName="manager"
Oct 30 21:32:58.988565 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:58.988483    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="14250b14-8ba8-4e5d-a20d-09fb6a0b3b9e" containerName="manager"
Oct 30 21:32:58.988620 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:58.988593    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="14250b14-8ba8-4e5d-a20d-09fb6a0b3b9e" containerName="manager"
Oct 30 21:32:58.996843 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:58.996817    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-9md4d"
Oct 30 21:32:59.002941 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:59.002914    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-9md4d"]
Oct 30 21:32:59.005088 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:59.004698    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Oct 30 21:32:59.005716 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:59.005696    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-955c4\""
Oct 30 21:32:59.067440 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:59.067405    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-zh266"]
Oct 30 21:32:59.071808 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:59.071781    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-zh266"
Oct 30 21:32:59.084733 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:59.084702    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-9md4d"]
Oct 30 21:32:59.089691 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:59.089661    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-zh266"]
Oct 30 21:32:59.104889 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:59.104861    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Oct 30 21:32:59.105070 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:59.104922    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Oct 30 21:32:59.167572 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:59.167535    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcm7t\" (UniqueName: \"kubernetes.io/projected/a70b450b-81a2-4059-aed2-6beebb186a95-kube-api-access-vcm7t\") pod \"kuadrant-console-plugin-66c4fb575d-zh266\" (UID: \"a70b450b-81a2-4059-aed2-6beebb186a95\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-zh266"
Oct 30 21:32:59.167762 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:59.167597    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nh2s4\" (UniqueName: \"kubernetes.io/projected/1cbea422-0fd0-47bc-9e33-b84f96cd3697-kube-api-access-nh2s4\") pod \"limitador-limitador-74dfcfb894-9md4d\" (UID: \"1cbea422-0fd0-47bc-9e33-b84f96cd3697\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-9md4d"
Oct 30 21:32:59.167762 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:59.167704    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a70b450b-81a2-4059-aed2-6beebb186a95-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-zh266\" (UID: \"a70b450b-81a2-4059-aed2-6beebb186a95\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-zh266"
Oct 30 21:32:59.167762 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:59.167750    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1cbea422-0fd0-47bc-9e33-b84f96cd3697-config-file\") pod \"limitador-limitador-74dfcfb894-9md4d\" (UID: \"1cbea422-0fd0-47bc-9e33-b84f96cd3697\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-9md4d"
Oct 30 21:32:59.167879 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:59.167811    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a70b450b-81a2-4059-aed2-6beebb186a95-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-zh266\" (UID: \"a70b450b-81a2-4059-aed2-6beebb186a95\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-zh266"
Oct 30 21:32:59.268926 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:59.268836    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a70b450b-81a2-4059-aed2-6beebb186a95-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-zh266\" (UID: \"a70b450b-81a2-4059-aed2-6beebb186a95\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-zh266"
Oct 30 21:32:59.268926 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:59.268911    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1cbea422-0fd0-47bc-9e33-b84f96cd3697-config-file\") pod \"limitador-limitador-74dfcfb894-9md4d\" (UID: \"1cbea422-0fd0-47bc-9e33-b84f96cd3697\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-9md4d"
Oct 30 21:32:59.269157 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:59.268936    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a70b450b-81a2-4059-aed2-6beebb186a95-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-zh266\" (UID: \"a70b450b-81a2-4059-aed2-6beebb186a95\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-zh266"
Oct 30 21:32:59.269157 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:59.268982    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcm7t\" (UniqueName: \"kubernetes.io/projected/a70b450b-81a2-4059-aed2-6beebb186a95-kube-api-access-vcm7t\") pod \"kuadrant-console-plugin-66c4fb575d-zh266\" (UID: \"a70b450b-81a2-4059-aed2-6beebb186a95\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-zh266"
Oct 30 21:32:59.269157 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:59.269041    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nh2s4\" (UniqueName: \"kubernetes.io/projected/1cbea422-0fd0-47bc-9e33-b84f96cd3697-kube-api-access-nh2s4\") pod \"limitador-limitador-74dfcfb894-9md4d\" (UID: \"1cbea422-0fd0-47bc-9e33-b84f96cd3697\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-9md4d"
Oct 30 21:32:59.269834 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:59.269802    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1cbea422-0fd0-47bc-9e33-b84f96cd3697-config-file\") pod \"limitador-limitador-74dfcfb894-9md4d\" (UID: \"1cbea422-0fd0-47bc-9e33-b84f96cd3697\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-9md4d"
Oct 30 21:32:59.269938 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:59.269799    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a70b450b-81a2-4059-aed2-6beebb186a95-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-zh266\" (UID: \"a70b450b-81a2-4059-aed2-6beebb186a95\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-zh266"
Oct 30 21:32:59.271941 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:59.271916    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a70b450b-81a2-4059-aed2-6beebb186a95-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-zh266\" (UID: \"a70b450b-81a2-4059-aed2-6beebb186a95\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-zh266"
Oct 30 21:32:59.282022 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:59.281995    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nh2s4\" (UniqueName: \"kubernetes.io/projected/1cbea422-0fd0-47bc-9e33-b84f96cd3697-kube-api-access-nh2s4\") pod \"limitador-limitador-74dfcfb894-9md4d\" (UID: \"1cbea422-0fd0-47bc-9e33-b84f96cd3697\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-9md4d"
Oct 30 21:32:59.287150 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:59.287124    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcm7t\" (UniqueName: \"kubernetes.io/projected/a70b450b-81a2-4059-aed2-6beebb186a95-kube-api-access-vcm7t\") pod \"kuadrant-console-plugin-66c4fb575d-zh266\" (UID: \"a70b450b-81a2-4059-aed2-6beebb186a95\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-zh266"
Oct 30 21:32:59.313321 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:59.313289    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-9md4d"
Oct 30 21:32:59.389473 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:59.389435    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-zh266"
Oct 30 21:32:59.442063 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:59.440648    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 21:32:59.444790 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:59.444658    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7c9b87d574-jxn67"
Oct 30 21:32:59.460314 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:59.459924    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 21:32:59.510225 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:59.510195    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 21:32:59.572602 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:59.572519    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4b0e32e7-f487-4b95-8557-5302fa36d72b-config-file\") pod \"limitador-limitador-7c9b87d574-jxn67\" (UID: \"4b0e32e7-f487-4b95-8557-5302fa36d72b\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-jxn67"
Oct 30 21:32:59.572752 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:59.572648    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qzvr\" (UniqueName: \"kubernetes.io/projected/4b0e32e7-f487-4b95-8557-5302fa36d72b-kube-api-access-8qzvr\") pod \"limitador-limitador-7c9b87d574-jxn67\" (UID: \"4b0e32e7-f487-4b95-8557-5302fa36d72b\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-jxn67"
Oct 30 21:32:59.605806 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:59.605779    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-9md4d"]
Oct 30 21:32:59.607861 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:32:59.607832    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cbea422_0fd0_47bc_9e33_b84f96cd3697.slice/crio-0f64d3d549e46990c003532d3df73e971b1b5ac28a0340285f63bbffc3b764f9 WatchSource:0}: Error finding container 0f64d3d549e46990c003532d3df73e971b1b5ac28a0340285f63bbffc3b764f9: Status 404 returned error can't find the container with id 0f64d3d549e46990c003532d3df73e971b1b5ac28a0340285f63bbffc3b764f9
Oct 30 21:32:59.673221 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:59.673186    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qzvr\" (UniqueName: \"kubernetes.io/projected/4b0e32e7-f487-4b95-8557-5302fa36d72b-kube-api-access-8qzvr\") pod \"limitador-limitador-7c9b87d574-jxn67\" (UID: \"4b0e32e7-f487-4b95-8557-5302fa36d72b\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-jxn67"
Oct 30 21:32:59.673444 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:59.673245    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4b0e32e7-f487-4b95-8557-5302fa36d72b-config-file\") pod \"limitador-limitador-7c9b87d574-jxn67\" (UID: \"4b0e32e7-f487-4b95-8557-5302fa36d72b\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-jxn67"
Oct 30 21:32:59.673949 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:59.673928    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-zh266"]
Oct 30 21:32:59.674135 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:59.674096    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4b0e32e7-f487-4b95-8557-5302fa36d72b-config-file\") pod \"limitador-limitador-7c9b87d574-jxn67\" (UID: \"4b0e32e7-f487-4b95-8557-5302fa36d72b\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-jxn67"
Oct 30 21:32:59.676398 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:32:59.676370    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda70b450b_81a2_4059_aed2_6beebb186a95.slice/crio-331d10dafa0e4e2d7d63f75fd537be595ba4a5ec9cd4298a3d193c45a38e1773 WatchSource:0}: Error finding container 331d10dafa0e4e2d7d63f75fd537be595ba4a5ec9cd4298a3d193c45a38e1773: Status 404 returned error can't find the container with id 331d10dafa0e4e2d7d63f75fd537be595ba4a5ec9cd4298a3d193c45a38e1773
Oct 30 21:32:59.686729 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:59.686702    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qzvr\" (UniqueName: \"kubernetes.io/projected/4b0e32e7-f487-4b95-8557-5302fa36d72b-kube-api-access-8qzvr\") pod \"limitador-limitador-7c9b87d574-jxn67\" (UID: \"4b0e32e7-f487-4b95-8557-5302fa36d72b\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-jxn67"
Oct 30 21:32:59.765647 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:59.765611    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7c9b87d574-jxn67"
Oct 30 21:32:59.831408 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:59.829452    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-zh266" event={"ID":"a70b450b-81a2-4059-aed2-6beebb186a95","Type":"ContainerStarted","Data":"331d10dafa0e4e2d7d63f75fd537be595ba4a5ec9cd4298a3d193c45a38e1773"}
Oct 30 21:32:59.831553 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:32:59.831432    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-9md4d" event={"ID":"1cbea422-0fd0-47bc-9e33-b84f96cd3697","Type":"ContainerStarted","Data":"0f64d3d549e46990c003532d3df73e971b1b5ac28a0340285f63bbffc3b764f9"}
Oct 30 21:33:00.044692 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:00.044665    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 21:33:00.047083 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:33:00.047052    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b0e32e7_f487_4b95_8557_5302fa36d72b.slice/crio-4e6264b669219a97deb591e6678ac8b93900490ed85ea77c14c6a00129e4ac3f WatchSource:0}: Error finding container 4e6264b669219a97deb591e6678ac8b93900490ed85ea77c14c6a00129e4ac3f: Status 404 returned error can't find the container with id 4e6264b669219a97deb591e6678ac8b93900490ed85ea77c14c6a00129e4ac3f
Oct 30 21:33:00.841527 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:00.841483    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7c9b87d574-jxn67" event={"ID":"4b0e32e7-f487-4b95-8557-5302fa36d72b","Type":"ContainerStarted","Data":"4e6264b669219a97deb591e6678ac8b93900490ed85ea77c14c6a00129e4ac3f"}
Oct 30 21:33:01.852092 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:01.852045    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-9md4d" event={"ID":"1cbea422-0fd0-47bc-9e33-b84f96cd3697","Type":"ContainerStarted","Data":"888a45797801498d39d534454700fab717492aa1d6dc977f7cae28dd87109c2b"}
Oct 30 21:33:01.852837 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:01.852819    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-9md4d"
Oct 30 21:33:01.854643 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:01.854616    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7c9b87d574-jxn67" event={"ID":"4b0e32e7-f487-4b95-8557-5302fa36d72b","Type":"ContainerStarted","Data":"f1f34cc03fe7dc596e1de50f7ee7ee5e99afff30e8102004aea31f6d6585ba11"}
Oct 30 21:33:01.854804 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:01.854748    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-7c9b87d574-jxn67"
Oct 30 21:33:01.874304 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:01.874251    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-74dfcfb894-9md4d" podStartSLOduration=2.481901986 podStartE2EDuration="3.874236228s" totalImagesPullingTime="1.392334242s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:32:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:33:01.872787184 +0000 UTC m=+294.674599649" watchObservedRunningTime="2026-10-30 21:33:01.874236228 +0000 UTC m=+294.676048692"
Oct 30 21:33:04.872378 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:04.872331    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-zh266" event={"ID":"a70b450b-81a2-4059-aed2-6beebb186a95","Type":"ContainerStarted","Data":"8506d3eadf5bfea70cfde73e68cb0acd3566ef6164d75c1a27cdddc64e58a5ac"}
Oct 30 21:33:05.906933 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:05.906872    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-7c9b87d574-jxn67" podStartSLOduration=5.878576135 podStartE2EDuration="6.906857706s" totalImagesPullingTime="1.028281571s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:32:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:33:01.903344077 +0000 UTC m=+294.705156547" watchObservedRunningTime="2026-10-30 21:33:05.906857706 +0000 UTC m=+298.708670172"
Oct 30 21:33:07.374316 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:07.374285    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-sklxd_67292bdc-7fe1-449d-af75-2baa90c16ea9/console-operator/1.log"
Oct 30 21:33:07.374723 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:07.374327    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-sklxd_67292bdc-7fe1-449d-af75-2baa90c16ea9/console-operator/1.log"
Oct 30 21:33:12.862143 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:12.862109    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-7c9b87d574-jxn67"
Oct 30 21:33:12.884335 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:12.884287    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-zh266" podStartSLOduration=8.759621651 podStartE2EDuration="13.88427308s" totalImagesPullingTime="5.124651429s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:32:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:33:05.907611769 +0000 UTC m=+298.709424233" watchObservedRunningTime="2026-10-30 21:33:12.88427308 +0000 UTC m=+305.686085569"
Oct 30 21:33:12.920049 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:12.920018    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-9md4d"]
Oct 30 21:33:12.920222 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:12.920204    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-74dfcfb894-9md4d" podUID="1cbea422-0fd0-47bc-9e33-b84f96cd3697" containerName="limitador" containerID="cri-o://888a45797801498d39d534454700fab717492aa1d6dc977f7cae28dd87109c2b" gracePeriod=30
Oct 30 21:33:12.921078 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:12.920992    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-9md4d"
Oct 30 21:33:13.465401 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:13.465375    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-9md4d"
Oct 30 21:33:13.502971 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:13.502890    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1cbea422-0fd0-47bc-9e33-b84f96cd3697-kube-api-access-nh2s4\" (UniqueName: \"kubernetes.io/projected/1cbea422-0fd0-47bc-9e33-b84f96cd3697-kube-api-access-nh2s4\") pod \"1cbea422-0fd0-47bc-9e33-b84f96cd3697\" (UID: \"1cbea422-0fd0-47bc-9e33-b84f96cd3697\") "
Oct 30 21:33:13.502971 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:13.502958    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1cbea422-0fd0-47bc-9e33-b84f96cd3697-config-file\" (UniqueName: \"kubernetes.io/configmap/1cbea422-0fd0-47bc-9e33-b84f96cd3697-config-file\") pod \"1cbea422-0fd0-47bc-9e33-b84f96cd3697\" (UID: \"1cbea422-0fd0-47bc-9e33-b84f96cd3697\") "
Oct 30 21:33:13.503735 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:13.503704    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1cbea422-0fd0-47bc-9e33-b84f96cd3697-config-file" pod "1cbea422-0fd0-47bc-9e33-b84f96cd3697" (UID: "1cbea422-0fd0-47bc-9e33-b84f96cd3697"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:33:13.505180 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:13.505157    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cbea422-0fd0-47bc-9e33-b84f96cd3697-kube-api-access-nh2s4" pod "1cbea422-0fd0-47bc-9e33-b84f96cd3697" (UID: "1cbea422-0fd0-47bc-9e33-b84f96cd3697"). InnerVolumeSpecName "kube-api-access-nh2s4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:33:13.603581 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:13.603548    5032 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1cbea422-0fd0-47bc-9e33-b84f96cd3697-config-file\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:33:13.603581 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:13.603575    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nh2s4\" (UniqueName: \"kubernetes.io/projected/1cbea422-0fd0-47bc-9e33-b84f96cd3697-kube-api-access-nh2s4\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:33:13.916496 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:13.916406    5032 generic.go:356] "Generic (PLEG): container finished" podID="1cbea422-0fd0-47bc-9e33-b84f96cd3697" containerID="888a45797801498d39d534454700fab717492aa1d6dc977f7cae28dd87109c2b" exitCode=0
Oct 30 21:33:13.916496 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:13.916449    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-9md4d"
Oct 30 21:33:13.916977 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:13.916484    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-9md4d" event={"ID":"1cbea422-0fd0-47bc-9e33-b84f96cd3697","Type":"ContainerDied","Data":"888a45797801498d39d534454700fab717492aa1d6dc977f7cae28dd87109c2b"}
Oct 30 21:33:13.916977 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:13.916545    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-9md4d" event={"ID":"1cbea422-0fd0-47bc-9e33-b84f96cd3697","Type":"ContainerDied","Data":"0f64d3d549e46990c003532d3df73e971b1b5ac28a0340285f63bbffc3b764f9"}
Oct 30 21:33:13.916977 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:13.916575    5032 scope.go:122] "RemoveContainer" containerID="888a45797801498d39d534454700fab717492aa1d6dc977f7cae28dd87109c2b"
Oct 30 21:33:13.925824 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:13.925644    5032 scope.go:122] "RemoveContainer" containerID="888a45797801498d39d534454700fab717492aa1d6dc977f7cae28dd87109c2b"
Oct 30 21:33:13.925984 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:33:13.925959    5032 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"888a45797801498d39d534454700fab717492aa1d6dc977f7cae28dd87109c2b\": container with ID starting with 888a45797801498d39d534454700fab717492aa1d6dc977f7cae28dd87109c2b not found: ID does not exist" containerID="888a45797801498d39d534454700fab717492aa1d6dc977f7cae28dd87109c2b"
Oct 30 21:33:13.926040 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:13.925990    5032 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"888a45797801498d39d534454700fab717492aa1d6dc977f7cae28dd87109c2b"} err="failed to get container status \"888a45797801498d39d534454700fab717492aa1d6dc977f7cae28dd87109c2b\": rpc error: code = NotFound desc = could not find container \"888a45797801498d39d534454700fab717492aa1d6dc977f7cae28dd87109c2b\": container with ID starting with 888a45797801498d39d534454700fab717492aa1d6dc977f7cae28dd87109c2b not found: ID does not exist"
Oct 30 21:33:13.947931 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:13.947900    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-9md4d"]
Oct 30 21:33:13.961731 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:13.961705    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-9md4d"]
Oct 30 21:33:15.344076 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:15.344039    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1cbea422-0fd0-47bc-9e33-b84f96cd3697" path="/var/lib/kubelet/pods/1cbea422-0fd0-47bc-9e33-b84f96cd3697/volumes"
Oct 30 21:33:15.636793 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:15.636714    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-5mx94"]
Oct 30 21:33:15.637252 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:15.637232    5032 cpu_manager.go:414] "containerMap: removing container" podUID="1cbea422-0fd0-47bc-9e33-b84f96cd3697" containerName="limitador"
Oct 30 21:33:15.637252 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:15.637249    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1cbea422-0fd0-47bc-9e33-b84f96cd3697" containerName="limitador"
Oct 30 21:33:15.637457 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:15.637402    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="1cbea422-0fd0-47bc-9e33-b84f96cd3697" containerName="limitador"
Oct 30 21:33:15.642389 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:15.642352    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-776cc66d47-5mx94"
Oct 30 21:33:15.655766 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:15.655725    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Oct 30 21:33:15.660878 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:15.660852    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-5mx94"]
Oct 30 21:33:15.666324 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:15.666303    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-hq4hv\""
Oct 30 21:33:15.713712 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:15.713677    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-wrl7s"]
Oct 30 21:33:15.717981 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:15.717956    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-75bd89fdfc-wrl7s"
Oct 30 21:33:15.725355 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:15.725326    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hj67g\" (UniqueName: \"kubernetes.io/projected/7a16205c-fa5e-4c19-bf95-122f14d59b1d-kube-api-access-hj67g\") pod \"kserve-controller-manager-776cc66d47-5mx94\" (UID: \"7a16205c-fa5e-4c19-bf95-122f14d59b1d\") " pod="opendatahub/kserve-controller-manager-776cc66d47-5mx94"
Oct 30 21:33:15.725652 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:15.725568    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7a16205c-fa5e-4c19-bf95-122f14d59b1d-cert\") pod \"kserve-controller-manager-776cc66d47-5mx94\" (UID: \"7a16205c-fa5e-4c19-bf95-122f14d59b1d\") " pod="opendatahub/kserve-controller-manager-776cc66d47-5mx94"
Oct 30 21:33:15.739001 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:15.738975    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-wrl7s"]
Oct 30 21:33:15.752589 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:15.752563    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-769875db56-gs699"]
Oct 30 21:33:15.756145 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:15.756111    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-769875db56-gs699"
Oct 30 21:33:15.758240 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:15.758201    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Oct 30 21:33:15.758648 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:15.758505    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-xdpx8\""
Oct 30 21:33:15.780513 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:15.780484    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-769875db56-gs699"]
Oct 30 21:33:15.826579 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:15.826543    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/60408246-d50f-492a-9ade-9f28c3f9ad2c-cert\") pod \"odh-model-controller-769875db56-gs699\" (UID: \"60408246-d50f-492a-9ade-9f28c3f9ad2c\") " pod="opendatahub/odh-model-controller-769875db56-gs699"
Oct 30 21:33:15.826757 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:15.826586    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7a16205c-fa5e-4c19-bf95-122f14d59b1d-cert\") pod \"kserve-controller-manager-776cc66d47-5mx94\" (UID: \"7a16205c-fa5e-4c19-bf95-122f14d59b1d\") " pod="opendatahub/kserve-controller-manager-776cc66d47-5mx94"
Oct 30 21:33:15.826757 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:15.826622    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8frz5\" (UniqueName: \"kubernetes.io/projected/9ef57dfb-240c-42e2-bc9d-c68c63f0a11c-kube-api-access-8frz5\") pod \"model-serving-api-75bd89fdfc-wrl7s\" (UID: \"9ef57dfb-240c-42e2-bc9d-c68c63f0a11c\") " pod="opendatahub/model-serving-api-75bd89fdfc-wrl7s"
Oct 30 21:33:15.826839 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:15.826814    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ef57dfb-240c-42e2-bc9d-c68c63f0a11c-tls-certs\") pod \"model-serving-api-75bd89fdfc-wrl7s\" (UID: \"9ef57dfb-240c-42e2-bc9d-c68c63f0a11c\") " pod="opendatahub/model-serving-api-75bd89fdfc-wrl7s"
Oct 30 21:33:15.826921 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:15.826906    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6rrt\" (UniqueName: \"kubernetes.io/projected/60408246-d50f-492a-9ade-9f28c3f9ad2c-kube-api-access-x6rrt\") pod \"odh-model-controller-769875db56-gs699\" (UID: \"60408246-d50f-492a-9ade-9f28c3f9ad2c\") " pod="opendatahub/odh-model-controller-769875db56-gs699"
Oct 30 21:33:15.826967 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:15.826936    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hj67g\" (UniqueName: \"kubernetes.io/projected/7a16205c-fa5e-4c19-bf95-122f14d59b1d-kube-api-access-hj67g\") pod \"kserve-controller-manager-776cc66d47-5mx94\" (UID: \"7a16205c-fa5e-4c19-bf95-122f14d59b1d\") " pod="opendatahub/kserve-controller-manager-776cc66d47-5mx94"
Oct 30 21:33:15.827007 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:15.826992    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60408246-d50f-492a-9ade-9f28c3f9ad2c-metrics-certs\") pod \"odh-model-controller-769875db56-gs699\" (UID: \"60408246-d50f-492a-9ade-9f28c3f9ad2c\") " pod="opendatahub/odh-model-controller-769875db56-gs699"
Oct 30 21:33:15.829301 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:15.829274    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7a16205c-fa5e-4c19-bf95-122f14d59b1d-cert\") pod \"kserve-controller-manager-776cc66d47-5mx94\" (UID: \"7a16205c-fa5e-4c19-bf95-122f14d59b1d\") " pod="opendatahub/kserve-controller-manager-776cc66d47-5mx94"
Oct 30 21:33:15.858530 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:15.858501    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hj67g\" (UniqueName: \"kubernetes.io/projected/7a16205c-fa5e-4c19-bf95-122f14d59b1d-kube-api-access-hj67g\") pod \"kserve-controller-manager-776cc66d47-5mx94\" (UID: \"7a16205c-fa5e-4c19-bf95-122f14d59b1d\") " pod="opendatahub/kserve-controller-manager-776cc66d47-5mx94"
Oct 30 21:33:15.870402 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:15.861602    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Oct 30 21:33:15.870402 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:15.861813    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-metrics-tls\""
Oct 30 21:33:15.870402 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:15.861813    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-888jx\""
Oct 30 21:33:15.927788 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:15.927703    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6rrt\" (UniqueName: \"kubernetes.io/projected/60408246-d50f-492a-9ade-9f28c3f9ad2c-kube-api-access-x6rrt\") pod \"odh-model-controller-769875db56-gs699\" (UID: \"60408246-d50f-492a-9ade-9f28c3f9ad2c\") " pod="opendatahub/odh-model-controller-769875db56-gs699"
Oct 30 21:33:15.927788 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:15.927743    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60408246-d50f-492a-9ade-9f28c3f9ad2c-metrics-certs\") pod \"odh-model-controller-769875db56-gs699\" (UID: \"60408246-d50f-492a-9ade-9f28c3f9ad2c\") " pod="opendatahub/odh-model-controller-769875db56-gs699"
Oct 30 21:33:15.927973 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:15.927920    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/60408246-d50f-492a-9ade-9f28c3f9ad2c-cert\") pod \"odh-model-controller-769875db56-gs699\" (UID: \"60408246-d50f-492a-9ade-9f28c3f9ad2c\") " pod="opendatahub/odh-model-controller-769875db56-gs699"
Oct 30 21:33:15.928009 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:15.927990    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8frz5\" (UniqueName: \"kubernetes.io/projected/9ef57dfb-240c-42e2-bc9d-c68c63f0a11c-kube-api-access-8frz5\") pod \"model-serving-api-75bd89fdfc-wrl7s\" (UID: \"9ef57dfb-240c-42e2-bc9d-c68c63f0a11c\") " pod="opendatahub/model-serving-api-75bd89fdfc-wrl7s"
Oct 30 21:33:15.928066 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:15.928051    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ef57dfb-240c-42e2-bc9d-c68c63f0a11c-tls-certs\") pod \"model-serving-api-75bd89fdfc-wrl7s\" (UID: \"9ef57dfb-240c-42e2-bc9d-c68c63f0a11c\") " pod="opendatahub/model-serving-api-75bd89fdfc-wrl7s"
Oct 30 21:33:15.930503 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:15.930467    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60408246-d50f-492a-9ade-9f28c3f9ad2c-metrics-certs\") pod \"odh-model-controller-769875db56-gs699\" (UID: \"60408246-d50f-492a-9ade-9f28c3f9ad2c\") " pod="opendatahub/odh-model-controller-769875db56-gs699"
Oct 30 21:33:15.930503 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:15.930485    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/60408246-d50f-492a-9ade-9f28c3f9ad2c-cert\") pod \"odh-model-controller-769875db56-gs699\" (UID: \"60408246-d50f-492a-9ade-9f28c3f9ad2c\") " pod="opendatahub/odh-model-controller-769875db56-gs699"
Oct 30 21:33:15.930676 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:15.930557    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ef57dfb-240c-42e2-bc9d-c68c63f0a11c-tls-certs\") pod \"model-serving-api-75bd89fdfc-wrl7s\" (UID: \"9ef57dfb-240c-42e2-bc9d-c68c63f0a11c\") " pod="opendatahub/model-serving-api-75bd89fdfc-wrl7s"
Oct 30 21:33:15.957344 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:15.957317    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-776cc66d47-5mx94"
Oct 30 21:33:15.987596 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:15.987567    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8frz5\" (UniqueName: \"kubernetes.io/projected/9ef57dfb-240c-42e2-bc9d-c68c63f0a11c-kube-api-access-8frz5\") pod \"model-serving-api-75bd89fdfc-wrl7s\" (UID: \"9ef57dfb-240c-42e2-bc9d-c68c63f0a11c\") " pod="opendatahub/model-serving-api-75bd89fdfc-wrl7s"
Oct 30 21:33:15.991626 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:15.991595    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6rrt\" (UniqueName: \"kubernetes.io/projected/60408246-d50f-492a-9ade-9f28c3f9ad2c-kube-api-access-x6rrt\") pod \"odh-model-controller-769875db56-gs699\" (UID: \"60408246-d50f-492a-9ade-9f28c3f9ad2c\") " pod="opendatahub/odh-model-controller-769875db56-gs699"
Oct 30 21:33:16.032769 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:16.032543    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-75bd89fdfc-wrl7s"
Oct 30 21:33:16.070929 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:16.070895    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-769875db56-gs699"
Oct 30 21:33:16.266245 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:16.266217    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-5mx94"]
Oct 30 21:33:16.268465 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:33:16.268430    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a16205c_fa5e_4c19_bf95_122f14d59b1d.slice/crio-8ae4090514a323f42e38ac493776e543463d1ec18c5ef05077e372805d1cc5f6 WatchSource:0}: Error finding container 8ae4090514a323f42e38ac493776e543463d1ec18c5ef05077e372805d1cc5f6: Status 404 returned error can't find the container with id 8ae4090514a323f42e38ac493776e543463d1ec18c5ef05077e372805d1cc5f6
Oct 30 21:33:16.269922 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:16.269900    5032 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 21:33:16.379502 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:16.379469    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-769875db56-gs699"]
Oct 30 21:33:16.385312 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:16.385200    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-wrl7s"]
Oct 30 21:33:16.387531 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:33:16.387496    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60408246_d50f_492a_9ade_9f28c3f9ad2c.slice/crio-d6a52dfe6dc92971b7ed414c0466e27412840cb850d3b6779915e88d2868456a WatchSource:0}: Error finding container d6a52dfe6dc92971b7ed414c0466e27412840cb850d3b6779915e88d2868456a: Status 404 returned error can't find the container with id d6a52dfe6dc92971b7ed414c0466e27412840cb850d3b6779915e88d2868456a
Oct 30 21:33:16.388298 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:33:16.388268    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ef57dfb_240c_42e2_bc9d_c68c63f0a11c.slice/crio-83da6662b8e23b0c449bf0e1cfb03e0f90d73f40d66d46853788553f06abc3c0 WatchSource:0}: Error finding container 83da6662b8e23b0c449bf0e1cfb03e0f90d73f40d66d46853788553f06abc3c0: Status 404 returned error can't find the container with id 83da6662b8e23b0c449bf0e1cfb03e0f90d73f40d66d46853788553f06abc3c0
Oct 30 21:33:16.936834 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:16.936792    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-776cc66d47-5mx94" event={"ID":"7a16205c-fa5e-4c19-bf95-122f14d59b1d","Type":"ContainerStarted","Data":"8ae4090514a323f42e38ac493776e543463d1ec18c5ef05077e372805d1cc5f6"}
Oct 30 21:33:16.938161 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:16.938122    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-769875db56-gs699" event={"ID":"60408246-d50f-492a-9ade-9f28c3f9ad2c","Type":"ContainerStarted","Data":"d6a52dfe6dc92971b7ed414c0466e27412840cb850d3b6779915e88d2868456a"}
Oct 30 21:33:16.939430 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:16.939400    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-75bd89fdfc-wrl7s" event={"ID":"9ef57dfb-240c-42e2-bc9d-c68c63f0a11c","Type":"ContainerStarted","Data":"83da6662b8e23b0c449bf0e1cfb03e0f90d73f40d66d46853788553f06abc3c0"}
Oct 30 21:33:18.355758 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:33:18.355536    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 21:33:18.595285 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:18.595250    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-mlq7r"]
Oct 30 21:33:18.599536 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:18.599498    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-mlq7r"
Oct 30 21:33:18.610734 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:18.609572    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 21:33:18.610734 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:18.609682    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 21:33:18.610734 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:18.609755    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Oct 30 21:33:18.610734 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:18.609845    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-252gq\""
Oct 30 21:33:18.612937 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:18.612907    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-mlq7r"]
Oct 30 21:33:18.654599 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:18.654557    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vf97x\" (UniqueName: \"kubernetes.io/projected/5d55ecfe-deb9-40d1-bdbb-2ad4c0d67f4b-kube-api-access-vf97x\") pod \"postgres-7cdbf6dc44-mlq7r\" (UID: \"5d55ecfe-deb9-40d1-bdbb-2ad4c0d67f4b\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-mlq7r"
Oct 30 21:33:18.654761 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:18.654733    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5d55ecfe-deb9-40d1-bdbb-2ad4c0d67f4b-data\") pod \"postgres-7cdbf6dc44-mlq7r\" (UID: \"5d55ecfe-deb9-40d1-bdbb-2ad4c0d67f4b\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-mlq7r"
Oct 30 21:33:18.756121 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:18.755927    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vf97x\" (UniqueName: \"kubernetes.io/projected/5d55ecfe-deb9-40d1-bdbb-2ad4c0d67f4b-kube-api-access-vf97x\") pod \"postgres-7cdbf6dc44-mlq7r\" (UID: \"5d55ecfe-deb9-40d1-bdbb-2ad4c0d67f4b\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-mlq7r"
Oct 30 21:33:18.756121 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:18.756037    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5d55ecfe-deb9-40d1-bdbb-2ad4c0d67f4b-data\") pod \"postgres-7cdbf6dc44-mlq7r\" (UID: \"5d55ecfe-deb9-40d1-bdbb-2ad4c0d67f4b\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-mlq7r"
Oct 30 21:33:18.756549 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:18.756521    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5d55ecfe-deb9-40d1-bdbb-2ad4c0d67f4b-data\") pod \"postgres-7cdbf6dc44-mlq7r\" (UID: \"5d55ecfe-deb9-40d1-bdbb-2ad4c0d67f4b\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-mlq7r"
Oct 30 21:33:18.785219 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:18.785182    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vf97x\" (UniqueName: \"kubernetes.io/projected/5d55ecfe-deb9-40d1-bdbb-2ad4c0d67f4b-kube-api-access-vf97x\") pod \"postgres-7cdbf6dc44-mlq7r\" (UID: \"5d55ecfe-deb9-40d1-bdbb-2ad4c0d67f4b\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-mlq7r"
Oct 30 21:33:18.922020 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:18.921989    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-mlq7r"
Oct 30 21:33:18.972396 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:18.972048    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-75bd89fdfc-wrl7s" event={"ID":"9ef57dfb-240c-42e2-bc9d-c68c63f0a11c","Type":"ContainerStarted","Data":"dd6d3651ca0bb913f061c992d6d6de7318f1e0bbc994af55bc0e274c4d7940f3"}
Oct 30 21:33:18.972396 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:18.972217    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-75bd89fdfc-wrl7s"
Oct 30 21:33:18.995305 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:18.994285    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/model-serving-api-75bd89fdfc-wrl7s" podStartSLOduration=1.521600661 podStartE2EDuration="3.99426844s" totalImagesPullingTime="2.472667779s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:33:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:33:18.993488201 +0000 UTC m=+311.795300666" watchObservedRunningTime="2026-10-30 21:33:18.99426844 +0000 UTC m=+311.796080934"
Oct 30 21:33:19.256612 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:19.256546    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-mlq7r"]
Oct 30 21:33:19.259004 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:33:19.258971    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d55ecfe_deb9_40d1_bdbb_2ad4c0d67f4b.slice/crio-6c01a3ec66850625390329c67d76e56e208d765359184c57e45a764d8817885a WatchSource:0}: Error finding container 6c01a3ec66850625390329c67d76e56e208d765359184c57e45a764d8817885a: Status 404 returned error can't find the container with id 6c01a3ec66850625390329c67d76e56e208d765359184c57e45a764d8817885a
Oct 30 21:33:19.978542 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:19.978499    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-769875db56-gs699" event={"ID":"60408246-d50f-492a-9ade-9f28c3f9ad2c","Type":"ContainerStarted","Data":"9c7c3fa8010119cc97c17491a82662fb45ae2434c0594c7730604d053fa44b2a"}
Oct 30 21:33:19.979041 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:19.978577    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-769875db56-gs699"
Oct 30 21:33:19.980004 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:19.979978    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-mlq7r" event={"ID":"5d55ecfe-deb9-40d1-bdbb-2ad4c0d67f4b","Type":"ContainerStarted","Data":"6c01a3ec66850625390329c67d76e56e208d765359184c57e45a764d8817885a"}
Oct 30 21:33:19.981497 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:19.981474    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-776cc66d47-5mx94" event={"ID":"7a16205c-fa5e-4c19-bf95-122f14d59b1d","Type":"ContainerStarted","Data":"0f33d8bce95fb9f546a2004673a55f30c56a5418e9a38f87c0674f40b63c9c55"}
Oct 30 21:33:19.981660 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:19.981636    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-776cc66d47-5mx94"
Oct 30 21:33:20.001073 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:20.001014    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/odh-model-controller-769875db56-gs699" podStartSLOduration=1.6177310280000001 podStartE2EDuration="5.000996799s" totalImagesPullingTime="3.383265771s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:33:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:33:20.000073138 +0000 UTC m=+312.801885627" watchObservedRunningTime="2026-10-30 21:33:20.000996799 +0000 UTC m=+312.802809263"
Oct 30 21:33:20.030243 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:20.030128    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-776cc66d47-5mx94" podStartSLOduration=2.145030936 podStartE2EDuration="5.030108646s" totalImagesPullingTime="2.88507771s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:33:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:33:20.027302334 +0000 UTC m=+312.829114799" watchObservedRunningTime="2026-10-30 21:33:20.030108646 +0000 UTC m=+312.831921111"
Oct 30 21:33:26.020122 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:26.020080    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-mlq7r" event={"ID":"5d55ecfe-deb9-40d1-bdbb-2ad4c0d67f4b","Type":"ContainerStarted","Data":"81c6029b85f0ca270ebbb9e8605c70ed0b1313c140265d0ad9a1b54847a32610"}
Oct 30 21:33:26.020619 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:26.020187    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-mlq7r"
Oct 30 21:33:26.039123 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:26.039066    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-mlq7r" podStartSLOduration=2.156750419 podStartE2EDuration="8.039049422s" totalImagesPullingTime="5.882299003s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:33:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:33:26.037262441 +0000 UTC m=+318.839074907" watchObservedRunningTime="2026-10-30 21:33:26.039049422 +0000 UTC m=+318.840861877"
Oct 30 21:33:29.985792 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:29.985764    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-75bd89fdfc-wrl7s"
Oct 30 21:33:30.991895 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:30.991859    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-769875db56-gs699"
Oct 30 21:33:32.055536 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:32.055501    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-mlq7r"
Oct 30 21:33:39.367303 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:39.367267    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-579cf7c6d-pg474"]
Oct 30 21:33:39.372403 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:39.372383    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579cf7c6d-pg474"
Oct 30 21:33:39.391632 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:39.391601    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Oct 30 21:33:39.391799 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:39.391674    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Oct 30 21:33:39.391927 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:39.391909    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Oct 30 21:33:39.393021 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:39.391787    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-4v6cn\""
Oct 30 21:33:39.394266 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:39.394237    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-579cf7c6d-pg474"]
Oct 30 21:33:39.460438 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:39.460391    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ad17299-8009-4a0a-bd8c-eb3f29b3a736-metrics-certs\") pod \"maas-controller-579cf7c6d-pg474\" (UID: \"5ad17299-8009-4a0a-bd8c-eb3f29b3a736\") " pod="opendatahub/maas-controller-579cf7c6d-pg474"
Oct 30 21:33:39.460615 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:39.460502    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5ad17299-8009-4a0a-bd8c-eb3f29b3a736-webhook-cert\") pod \"maas-controller-579cf7c6d-pg474\" (UID: \"5ad17299-8009-4a0a-bd8c-eb3f29b3a736\") " pod="opendatahub/maas-controller-579cf7c6d-pg474"
Oct 30 21:33:39.460615 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:39.460546    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2x4cd\" (UniqueName: \"kubernetes.io/projected/5ad17299-8009-4a0a-bd8c-eb3f29b3a736-kube-api-access-2x4cd\") pod \"maas-controller-579cf7c6d-pg474\" (UID: \"5ad17299-8009-4a0a-bd8c-eb3f29b3a736\") " pod="opendatahub/maas-controller-579cf7c6d-pg474"
Oct 30 21:33:39.561758 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:39.561722    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5ad17299-8009-4a0a-bd8c-eb3f29b3a736-webhook-cert\") pod \"maas-controller-579cf7c6d-pg474\" (UID: \"5ad17299-8009-4a0a-bd8c-eb3f29b3a736\") " pod="opendatahub/maas-controller-579cf7c6d-pg474"
Oct 30 21:33:39.561965 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:39.561768    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2x4cd\" (UniqueName: \"kubernetes.io/projected/5ad17299-8009-4a0a-bd8c-eb3f29b3a736-kube-api-access-2x4cd\") pod \"maas-controller-579cf7c6d-pg474\" (UID: \"5ad17299-8009-4a0a-bd8c-eb3f29b3a736\") " pod="opendatahub/maas-controller-579cf7c6d-pg474"
Oct 30 21:33:39.561965 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:39.561811    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ad17299-8009-4a0a-bd8c-eb3f29b3a736-metrics-certs\") pod \"maas-controller-579cf7c6d-pg474\" (UID: \"5ad17299-8009-4a0a-bd8c-eb3f29b3a736\") " pod="opendatahub/maas-controller-579cf7c6d-pg474"
Oct 30 21:33:39.564417 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:39.564386    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5ad17299-8009-4a0a-bd8c-eb3f29b3a736-webhook-cert\") pod \"maas-controller-579cf7c6d-pg474\" (UID: \"5ad17299-8009-4a0a-bd8c-eb3f29b3a736\") " pod="opendatahub/maas-controller-579cf7c6d-pg474"
Oct 30 21:33:39.564539 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:39.564443    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ad17299-8009-4a0a-bd8c-eb3f29b3a736-metrics-certs\") pod \"maas-controller-579cf7c6d-pg474\" (UID: \"5ad17299-8009-4a0a-bd8c-eb3f29b3a736\") " pod="opendatahub/maas-controller-579cf7c6d-pg474"
Oct 30 21:33:39.575731 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:39.575701    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2x4cd\" (UniqueName: \"kubernetes.io/projected/5ad17299-8009-4a0a-bd8c-eb3f29b3a736-kube-api-access-2x4cd\") pod \"maas-controller-579cf7c6d-pg474\" (UID: \"5ad17299-8009-4a0a-bd8c-eb3f29b3a736\") " pod="opendatahub/maas-controller-579cf7c6d-pg474"
Oct 30 21:33:39.690217 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:39.690131    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579cf7c6d-pg474"
Oct 30 21:33:39.715611 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:39.715580    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7bff7d9b4d-tzcgw"]
Oct 30 21:33:39.720285 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:39.720259    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7bff7d9b4d-tzcgw"
Oct 30 21:33:39.730949 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:39.730915    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7bff7d9b4d-tzcgw"]
Oct 30 21:33:39.865303 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:39.865263    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9cr42\" (UniqueName: \"kubernetes.io/projected/ca2836d0-0021-4c48-9b62-03e24eb03ac8-kube-api-access-9cr42\") pod \"maas-controller-7bff7d9b4d-tzcgw\" (UID: \"ca2836d0-0021-4c48-9b62-03e24eb03ac8\") " pod="opendatahub/maas-controller-7bff7d9b4d-tzcgw"
Oct 30 21:33:39.865512 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:39.865314    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ca2836d0-0021-4c48-9b62-03e24eb03ac8-webhook-cert\") pod \"maas-controller-7bff7d9b4d-tzcgw\" (UID: \"ca2836d0-0021-4c48-9b62-03e24eb03ac8\") " pod="opendatahub/maas-controller-7bff7d9b4d-tzcgw"
Oct 30 21:33:39.865512 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:39.865333    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca2836d0-0021-4c48-9b62-03e24eb03ac8-metrics-certs\") pod \"maas-controller-7bff7d9b4d-tzcgw\" (UID: \"ca2836d0-0021-4c48-9b62-03e24eb03ac8\") " pod="opendatahub/maas-controller-7bff7d9b4d-tzcgw"
Oct 30 21:33:39.966639 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:39.966603    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9cr42\" (UniqueName: \"kubernetes.io/projected/ca2836d0-0021-4c48-9b62-03e24eb03ac8-kube-api-access-9cr42\") pod \"maas-controller-7bff7d9b4d-tzcgw\" (UID: \"ca2836d0-0021-4c48-9b62-03e24eb03ac8\") " pod="opendatahub/maas-controller-7bff7d9b4d-tzcgw"
Oct 30 21:33:39.966835 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:39.966654    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ca2836d0-0021-4c48-9b62-03e24eb03ac8-webhook-cert\") pod \"maas-controller-7bff7d9b4d-tzcgw\" (UID: \"ca2836d0-0021-4c48-9b62-03e24eb03ac8\") " pod="opendatahub/maas-controller-7bff7d9b4d-tzcgw"
Oct 30 21:33:39.966835 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:39.966671    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca2836d0-0021-4c48-9b62-03e24eb03ac8-metrics-certs\") pod \"maas-controller-7bff7d9b4d-tzcgw\" (UID: \"ca2836d0-0021-4c48-9b62-03e24eb03ac8\") " pod="opendatahub/maas-controller-7bff7d9b4d-tzcgw"
Oct 30 21:33:39.969467 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:39.969438    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca2836d0-0021-4c48-9b62-03e24eb03ac8-metrics-certs\") pod \"maas-controller-7bff7d9b4d-tzcgw\" (UID: \"ca2836d0-0021-4c48-9b62-03e24eb03ac8\") " pod="opendatahub/maas-controller-7bff7d9b4d-tzcgw"
Oct 30 21:33:39.969608 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:39.969484    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ca2836d0-0021-4c48-9b62-03e24eb03ac8-webhook-cert\") pod \"maas-controller-7bff7d9b4d-tzcgw\" (UID: \"ca2836d0-0021-4c48-9b62-03e24eb03ac8\") " pod="opendatahub/maas-controller-7bff7d9b4d-tzcgw"
Oct 30 21:33:39.972067 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:39.972046    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-579cf7c6d-pg474"]
Oct 30 21:33:39.973889 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:33:39.973861    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ad17299_8009_4a0a_bd8c_eb3f29b3a736.slice/crio-fa650bd66ced5cb70686c0e57395ab8fc8584a56b464c6927fb4ee30f3fa809b WatchSource:0}: Error finding container fa650bd66ced5cb70686c0e57395ab8fc8584a56b464c6927fb4ee30f3fa809b: Status 404 returned error can't find the container with id fa650bd66ced5cb70686c0e57395ab8fc8584a56b464c6927fb4ee30f3fa809b
Oct 30 21:33:39.979252 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:39.979229    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9cr42\" (UniqueName: \"kubernetes.io/projected/ca2836d0-0021-4c48-9b62-03e24eb03ac8-kube-api-access-9cr42\") pod \"maas-controller-7bff7d9b4d-tzcgw\" (UID: \"ca2836d0-0021-4c48-9b62-03e24eb03ac8\") " pod="opendatahub/maas-controller-7bff7d9b4d-tzcgw"
Oct 30 21:33:40.036859 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:40.036826    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7bff7d9b4d-tzcgw"
Oct 30 21:33:40.093127 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:40.093092    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579cf7c6d-pg474" event={"ID":"5ad17299-8009-4a0a-bd8c-eb3f29b3a736","Type":"ContainerStarted","Data":"fa650bd66ced5cb70686c0e57395ab8fc8584a56b464c6927fb4ee30f3fa809b"}
Oct 30 21:33:40.317580 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:40.317553    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7bff7d9b4d-tzcgw"]
Oct 30 21:33:40.319379 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:33:40.319333    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca2836d0_0021_4c48_9b62_03e24eb03ac8.slice/crio-d3a4c5af9d24de6098fc6b43f71099c6062f11922e9cf51561ef638d6abc8c2b WatchSource:0}: Error finding container d3a4c5af9d24de6098fc6b43f71099c6062f11922e9cf51561ef638d6abc8c2b: Status 404 returned error can't find the container with id d3a4c5af9d24de6098fc6b43f71099c6062f11922e9cf51561ef638d6abc8c2b
Oct 30 21:33:41.109743 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:41.108656    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7bff7d9b4d-tzcgw" event={"ID":"ca2836d0-0021-4c48-9b62-03e24eb03ac8","Type":"ContainerStarted","Data":"d3a4c5af9d24de6098fc6b43f71099c6062f11922e9cf51561ef638d6abc8c2b"}
Oct 30 21:33:43.123283 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:43.123250    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7bff7d9b4d-tzcgw" event={"ID":"ca2836d0-0021-4c48-9b62-03e24eb03ac8","Type":"ContainerStarted","Data":"e6759070c57fb32641b40c1505a100acd50d78ada8ff82fadbdd1fc52da36871"}
Oct 30 21:33:43.123702 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:43.123518    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7bff7d9b4d-tzcgw"
Oct 30 21:33:43.148453 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:43.148392    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-7bff7d9b4d-tzcgw" podStartSLOduration=1.427957727 podStartE2EDuration="4.148347503s" totalImagesPullingTime="2.720389776s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:33:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:33:43.146557011 +0000 UTC m=+335.948369487" watchObservedRunningTime="2026-10-30 21:33:43.148347503 +0000 UTC m=+335.950159967"
Oct 30 21:33:44.130742 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:44.130702    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579cf7c6d-pg474" event={"ID":"5ad17299-8009-4a0a-bd8c-eb3f29b3a736","Type":"ContainerStarted","Data":"91cf04f9166ec62c7aa8ad3680842ea4f1d03e442dc120cfac145ca7a25e343d"}
Oct 30 21:33:44.131243 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:44.130794    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-579cf7c6d-pg474"
Oct 30 21:33:44.167841 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:44.167782    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-579cf7c6d-pg474" podStartSLOduration=1.953570083 podStartE2EDuration="5.167762461s" totalImagesPullingTime="3.214192378s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:33:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:33:44.166653346 +0000 UTC m=+336.968465819" watchObservedRunningTime="2026-10-30 21:33:44.167762461 +0000 UTC m=+336.969574925"
Oct 30 21:33:48.347815 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:33:48.347773    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 21:33:50.994255 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:50.994226    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-776cc66d47-5mx94"
Oct 30 21:33:54.133113 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:54.133085    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7bff7d9b4d-tzcgw"
Oct 30 21:33:54.188266 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:54.188235    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-579cf7c6d-pg474"]
Oct 30 21:33:54.188591 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:54.188539    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-579cf7c6d-pg474" podUID="5ad17299-8009-4a0a-bd8c-eb3f29b3a736" containerName="manager" containerID="cri-o://91cf04f9166ec62c7aa8ad3680842ea4f1d03e442dc120cfac145ca7a25e343d" gracePeriod=30
Oct 30 21:33:54.190521 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:54.190462    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-579cf7c6d-pg474"
Oct 30 21:33:54.448828 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:54.448796    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579cf7c6d-pg474"
Oct 30 21:33:54.496903 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:54.496867    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5ad17299-8009-4a0a-bd8c-eb3f29b3a736-metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ad17299-8009-4a0a-bd8c-eb3f29b3a736-metrics-certs\") pod \"5ad17299-8009-4a0a-bd8c-eb3f29b3a736\" (UID: \"5ad17299-8009-4a0a-bd8c-eb3f29b3a736\") "
Oct 30 21:33:54.497096 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:54.496927    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5ad17299-8009-4a0a-bd8c-eb3f29b3a736-webhook-cert\" (UniqueName: \"kubernetes.io/secret/5ad17299-8009-4a0a-bd8c-eb3f29b3a736-webhook-cert\") pod \"5ad17299-8009-4a0a-bd8c-eb3f29b3a736\" (UID: \"5ad17299-8009-4a0a-bd8c-eb3f29b3a736\") "
Oct 30 21:33:54.497096 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:54.496971    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5ad17299-8009-4a0a-bd8c-eb3f29b3a736-kube-api-access-2x4cd\" (UniqueName: \"kubernetes.io/projected/5ad17299-8009-4a0a-bd8c-eb3f29b3a736-kube-api-access-2x4cd\") pod \"5ad17299-8009-4a0a-bd8c-eb3f29b3a736\" (UID: \"5ad17299-8009-4a0a-bd8c-eb3f29b3a736\") "
Oct 30 21:33:54.499242 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:54.499202    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ad17299-8009-4a0a-bd8c-eb3f29b3a736-webhook-cert" pod "5ad17299-8009-4a0a-bd8c-eb3f29b3a736" (UID: "5ad17299-8009-4a0a-bd8c-eb3f29b3a736"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:33:54.499414 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:54.499251    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ad17299-8009-4a0a-bd8c-eb3f29b3a736-kube-api-access-2x4cd" pod "5ad17299-8009-4a0a-bd8c-eb3f29b3a736" (UID: "5ad17299-8009-4a0a-bd8c-eb3f29b3a736"). InnerVolumeSpecName "kube-api-access-2x4cd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:33:54.499414 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:54.499281    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ad17299-8009-4a0a-bd8c-eb3f29b3a736-metrics-certs" pod "5ad17299-8009-4a0a-bd8c-eb3f29b3a736" (UID: "5ad17299-8009-4a0a-bd8c-eb3f29b3a736"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:33:54.597846 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:54.597812    5032 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ad17299-8009-4a0a-bd8c-eb3f29b3a736-metrics-certs\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:33:54.597846 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:54.597839    5032 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5ad17299-8009-4a0a-bd8c-eb3f29b3a736-webhook-cert\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:33:54.597846 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:54.597849    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2x4cd\" (UniqueName: \"kubernetes.io/projected/5ad17299-8009-4a0a-bd8c-eb3f29b3a736-kube-api-access-2x4cd\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:33:55.186743 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:55.186706    5032 generic.go:356] "Generic (PLEG): container finished" podID="5ad17299-8009-4a0a-bd8c-eb3f29b3a736" containerID="91cf04f9166ec62c7aa8ad3680842ea4f1d03e442dc120cfac145ca7a25e343d" exitCode=0
Oct 30 21:33:55.187209 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:55.186760    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579cf7c6d-pg474"
Oct 30 21:33:55.187209 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:55.186794    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579cf7c6d-pg474" event={"ID":"5ad17299-8009-4a0a-bd8c-eb3f29b3a736","Type":"ContainerDied","Data":"91cf04f9166ec62c7aa8ad3680842ea4f1d03e442dc120cfac145ca7a25e343d"}
Oct 30 21:33:55.187209 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:55.186850    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579cf7c6d-pg474" event={"ID":"5ad17299-8009-4a0a-bd8c-eb3f29b3a736","Type":"ContainerDied","Data":"fa650bd66ced5cb70686c0e57395ab8fc8584a56b464c6927fb4ee30f3fa809b"}
Oct 30 21:33:55.187209 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:55.186874    5032 scope.go:122] "RemoveContainer" containerID="91cf04f9166ec62c7aa8ad3680842ea4f1d03e442dc120cfac145ca7a25e343d"
Oct 30 21:33:55.197047 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:55.196926    5032 scope.go:122] "RemoveContainer" containerID="91cf04f9166ec62c7aa8ad3680842ea4f1d03e442dc120cfac145ca7a25e343d"
Oct 30 21:33:55.197253 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:33:55.197235    5032 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91cf04f9166ec62c7aa8ad3680842ea4f1d03e442dc120cfac145ca7a25e343d\": container with ID starting with 91cf04f9166ec62c7aa8ad3680842ea4f1d03e442dc120cfac145ca7a25e343d not found: ID does not exist" containerID="91cf04f9166ec62c7aa8ad3680842ea4f1d03e442dc120cfac145ca7a25e343d"
Oct 30 21:33:55.197296 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:55.197262    5032 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91cf04f9166ec62c7aa8ad3680842ea4f1d03e442dc120cfac145ca7a25e343d"} err="failed to get container status \"91cf04f9166ec62c7aa8ad3680842ea4f1d03e442dc120cfac145ca7a25e343d\": rpc error: code = NotFound desc = could not find container \"91cf04f9166ec62c7aa8ad3680842ea4f1d03e442dc120cfac145ca7a25e343d\": container with ID starting with 91cf04f9166ec62c7aa8ad3680842ea4f1d03e442dc120cfac145ca7a25e343d not found: ID does not exist"
Oct 30 21:33:55.210810 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:55.210776    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-579cf7c6d-pg474"]
Oct 30 21:33:55.215146 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:55.215117    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-579cf7c6d-pg474"]
Oct 30 21:33:55.344996 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:33:55.344963    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5ad17299-8009-4a0a-bd8c-eb3f29b3a736" path="/var/lib/kubelet/pods/5ad17299-8009-4a0a-bd8c-eb3f29b3a736/volumes"
Oct 30 21:34:18.001870 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:34:18.001832    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 21:34:18.345274 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:34:18.345191    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 21:34:27.387194 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:34:27.387164    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6884d9c49d-kr2ch"]
Oct 30 21:34:27.387685 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:34:27.387626    5032 cpu_manager.go:414] "containerMap: removing container" podUID="5ad17299-8009-4a0a-bd8c-eb3f29b3a736" containerName="manager"
Oct 30 21:34:27.387685 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:34:27.387638    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5ad17299-8009-4a0a-bd8c-eb3f29b3a736" containerName="manager"
Oct 30 21:34:27.387770 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:34:27.387709    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="5ad17299-8009-4a0a-bd8c-eb3f29b3a736" containerName="manager"
Oct 30 21:34:27.391288 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:34:27.391265    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6884d9c49d-kr2ch"
Oct 30 21:34:27.403068 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:34:27.403036    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6884d9c49d-kr2ch"]
Oct 30 21:34:27.467986 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:34:27.467959    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Oct 30 21:34:27.467986 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:34:27.467957    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Oct 30 21:34:27.468194 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:34:27.468032    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-4ttw5\""
Oct 30 21:34:27.495155 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:34:27.495124    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bb5pt\" (UniqueName: \"kubernetes.io/projected/928b268a-2870-48b8-ae93-2480013abe2a-kube-api-access-bb5pt\") pod \"maas-api-6884d9c49d-kr2ch\" (UID: \"928b268a-2870-48b8-ae93-2480013abe2a\") " pod="odh-ai-gateway-infra/maas-api-6884d9c49d-kr2ch"
Oct 30 21:34:27.495298 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:34:27.495178    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/928b268a-2870-48b8-ae93-2480013abe2a-metrics-certs\") pod \"maas-api-6884d9c49d-kr2ch\" (UID: \"928b268a-2870-48b8-ae93-2480013abe2a\") " pod="odh-ai-gateway-infra/maas-api-6884d9c49d-kr2ch"
Oct 30 21:34:27.495298 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:34:27.495204    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/928b268a-2870-48b8-ae93-2480013abe2a-maas-api-tls\") pod \"maas-api-6884d9c49d-kr2ch\" (UID: \"928b268a-2870-48b8-ae93-2480013abe2a\") " pod="odh-ai-gateway-infra/maas-api-6884d9c49d-kr2ch"
Oct 30 21:34:27.495439 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:34:27.495299    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/928b268a-2870-48b8-ae93-2480013abe2a-openshift-service-ca\") pod \"maas-api-6884d9c49d-kr2ch\" (UID: \"928b268a-2870-48b8-ae93-2480013abe2a\") " pod="odh-ai-gateway-infra/maas-api-6884d9c49d-kr2ch"
Oct 30 21:34:27.596452 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:34:27.596415    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bb5pt\" (UniqueName: \"kubernetes.io/projected/928b268a-2870-48b8-ae93-2480013abe2a-kube-api-access-bb5pt\") pod \"maas-api-6884d9c49d-kr2ch\" (UID: \"928b268a-2870-48b8-ae93-2480013abe2a\") " pod="odh-ai-gateway-infra/maas-api-6884d9c49d-kr2ch"
Oct 30 21:34:27.596662 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:34:27.596475    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/928b268a-2870-48b8-ae93-2480013abe2a-metrics-certs\") pod \"maas-api-6884d9c49d-kr2ch\" (UID: \"928b268a-2870-48b8-ae93-2480013abe2a\") " pod="odh-ai-gateway-infra/maas-api-6884d9c49d-kr2ch"
Oct 30 21:34:27.596662 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:34:27.596494    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/928b268a-2870-48b8-ae93-2480013abe2a-maas-api-tls\") pod \"maas-api-6884d9c49d-kr2ch\" (UID: \"928b268a-2870-48b8-ae93-2480013abe2a\") " pod="odh-ai-gateway-infra/maas-api-6884d9c49d-kr2ch"
Oct 30 21:34:27.596662 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:34:27.596510    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/928b268a-2870-48b8-ae93-2480013abe2a-openshift-service-ca\") pod \"maas-api-6884d9c49d-kr2ch\" (UID: \"928b268a-2870-48b8-ae93-2480013abe2a\") " pod="odh-ai-gateway-infra/maas-api-6884d9c49d-kr2ch"
Oct 30 21:34:27.597419 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:34:27.597387    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/928b268a-2870-48b8-ae93-2480013abe2a-openshift-service-ca\") pod \"maas-api-6884d9c49d-kr2ch\" (UID: \"928b268a-2870-48b8-ae93-2480013abe2a\") " pod="odh-ai-gateway-infra/maas-api-6884d9c49d-kr2ch"
Oct 30 21:34:27.599080 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:34:27.599058    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/928b268a-2870-48b8-ae93-2480013abe2a-metrics-certs\") pod \"maas-api-6884d9c49d-kr2ch\" (UID: \"928b268a-2870-48b8-ae93-2480013abe2a\") " pod="odh-ai-gateway-infra/maas-api-6884d9c49d-kr2ch"
Oct 30 21:34:27.599172 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:34:27.599135    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/928b268a-2870-48b8-ae93-2480013abe2a-maas-api-tls\") pod \"maas-api-6884d9c49d-kr2ch\" (UID: \"928b268a-2870-48b8-ae93-2480013abe2a\") " pod="odh-ai-gateway-infra/maas-api-6884d9c49d-kr2ch"
Oct 30 21:34:27.613945 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:34:27.613913    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bb5pt\" (UniqueName: \"kubernetes.io/projected/928b268a-2870-48b8-ae93-2480013abe2a-kube-api-access-bb5pt\") pod \"maas-api-6884d9c49d-kr2ch\" (UID: \"928b268a-2870-48b8-ae93-2480013abe2a\") " pod="odh-ai-gateway-infra/maas-api-6884d9c49d-kr2ch"
Oct 30 21:34:27.704766 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:34:27.704736    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6884d9c49d-kr2ch"
Oct 30 21:34:27.985445 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:34:27.985415    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6884d9c49d-kr2ch"]
Oct 30 21:34:27.987532 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:34:27.987500    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod928b268a_2870_48b8_ae93_2480013abe2a.slice/crio-bac9246b81e6408741bb550f42a5c5579e4fe5de07bb6aa80cf554a840196157 WatchSource:0}: Error finding container bac9246b81e6408741bb550f42a5c5579e4fe5de07bb6aa80cf554a840196157: Status 404 returned error can't find the container with id bac9246b81e6408741bb550f42a5c5579e4fe5de07bb6aa80cf554a840196157
Oct 30 21:34:28.355659 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:34:28.355577    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6884d9c49d-kr2ch" event={"ID":"928b268a-2870-48b8-ae93-2480013abe2a","Type":"ContainerStarted","Data":"bac9246b81e6408741bb550f42a5c5579e4fe5de07bb6aa80cf554a840196157"}
Oct 30 21:34:30.365965 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:34:30.365931    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6884d9c49d-kr2ch" event={"ID":"928b268a-2870-48b8-ae93-2480013abe2a","Type":"ContainerStarted","Data":"bd7792d8044b5ca19eafc03e5cece417a42c635e9df6fc95a429d04d40ae1e2e"}
Oct 30 21:34:30.366421 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:34:30.366093    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6884d9c49d-kr2ch"
Oct 30 21:34:30.387372 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:34:30.387321    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6884d9c49d-kr2ch" podStartSLOduration=1.975948687 podStartE2EDuration="3.387307052s" totalImagesPullingTime="1.411358365s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:34:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:34:30.386345158 +0000 UTC m=+383.188157632" watchObservedRunningTime="2026-10-30 21:34:30.387307052 +0000 UTC m=+383.189119516"
Oct 30 21:34:36.376584 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:34:36.376548    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6884d9c49d-kr2ch"
Oct 30 21:34:48.344025 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:34:48.343937    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 21:35:18.343344 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:35:18.343306    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 21:35:22.853504 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:22.853472    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-xv7qv"]
Oct 30 21:35:22.857374 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:22.857339    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-xv7qv"
Oct 30 21:35:22.864218 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:22.864013    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Oct 30 21:35:22.864218 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:22.864030    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-k8tnn\""
Oct 30 21:35:22.864218 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:22.864026    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Oct 30 21:35:22.864218 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:22.864056    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Oct 30 21:35:22.871200 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:22.871173    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-xv7qv"]
Oct 30 21:35:23.004466 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:23.004422    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f6d23b8-1364-4b03-96c6-6649f3a0644a-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-xv7qv\" (UID: \"4f6d23b8-1364-4b03-96c6-6649f3a0644a\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-xv7qv"
Oct 30 21:35:23.004650 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:23.004486    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f6d23b8-1364-4b03-96c6-6649f3a0644a-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-xv7qv\" (UID: \"4f6d23b8-1364-4b03-96c6-6649f3a0644a\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-xv7qv"
Oct 30 21:35:23.004650 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:23.004567    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f6d23b8-1364-4b03-96c6-6649f3a0644a-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-xv7qv\" (UID: \"4f6d23b8-1364-4b03-96c6-6649f3a0644a\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-xv7qv"
Oct 30 21:35:23.004650 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:23.004593    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f6d23b8-1364-4b03-96c6-6649f3a0644a-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-xv7qv\" (UID: \"4f6d23b8-1364-4b03-96c6-6649f3a0644a\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-xv7qv"
Oct 30 21:35:23.004650 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:23.004631    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f6d23b8-1364-4b03-96c6-6649f3a0644a-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-xv7qv\" (UID: \"4f6d23b8-1364-4b03-96c6-6649f3a0644a\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-xv7qv"
Oct 30 21:35:23.004813 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:23.004750    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wc2nb\" (UniqueName: \"kubernetes.io/projected/4f6d23b8-1364-4b03-96c6-6649f3a0644a-kube-api-access-wc2nb\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-xv7qv\" (UID: \"4f6d23b8-1364-4b03-96c6-6649f3a0644a\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-xv7qv"
Oct 30 21:35:23.105780 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:23.105688    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wc2nb\" (UniqueName: \"kubernetes.io/projected/4f6d23b8-1364-4b03-96c6-6649f3a0644a-kube-api-access-wc2nb\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-xv7qv\" (UID: \"4f6d23b8-1364-4b03-96c6-6649f3a0644a\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-xv7qv"
Oct 30 21:35:23.105780 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:23.105741    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f6d23b8-1364-4b03-96c6-6649f3a0644a-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-xv7qv\" (UID: \"4f6d23b8-1364-4b03-96c6-6649f3a0644a\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-xv7qv"
Oct 30 21:35:23.105780 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:23.105773    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f6d23b8-1364-4b03-96c6-6649f3a0644a-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-xv7qv\" (UID: \"4f6d23b8-1364-4b03-96c6-6649f3a0644a\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-xv7qv"
Oct 30 21:35:23.106068 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:23.105789    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f6d23b8-1364-4b03-96c6-6649f3a0644a-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-xv7qv\" (UID: \"4f6d23b8-1364-4b03-96c6-6649f3a0644a\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-xv7qv"
Oct 30 21:35:23.106068 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:23.105804    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f6d23b8-1364-4b03-96c6-6649f3a0644a-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-xv7qv\" (UID: \"4f6d23b8-1364-4b03-96c6-6649f3a0644a\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-xv7qv"
Oct 30 21:35:23.106068 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:23.105834    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f6d23b8-1364-4b03-96c6-6649f3a0644a-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-xv7qv\" (UID: \"4f6d23b8-1364-4b03-96c6-6649f3a0644a\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-xv7qv"
Oct 30 21:35:23.106335 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:23.106291    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f6d23b8-1364-4b03-96c6-6649f3a0644a-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-xv7qv\" (UID: \"4f6d23b8-1364-4b03-96c6-6649f3a0644a\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-xv7qv"
Oct 30 21:35:23.106496 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:23.106344    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f6d23b8-1364-4b03-96c6-6649f3a0644a-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-xv7qv\" (UID: \"4f6d23b8-1364-4b03-96c6-6649f3a0644a\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-xv7qv"
Oct 30 21:35:23.106496 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:23.106314    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f6d23b8-1364-4b03-96c6-6649f3a0644a-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-xv7qv\" (UID: \"4f6d23b8-1364-4b03-96c6-6649f3a0644a\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-xv7qv"
Oct 30 21:35:23.108236 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:23.108215    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f6d23b8-1364-4b03-96c6-6649f3a0644a-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-xv7qv\" (UID: \"4f6d23b8-1364-4b03-96c6-6649f3a0644a\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-xv7qv"
Oct 30 21:35:23.108414 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:23.108397    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f6d23b8-1364-4b03-96c6-6649f3a0644a-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-xv7qv\" (UID: \"4f6d23b8-1364-4b03-96c6-6649f3a0644a\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-xv7qv"
Oct 30 21:35:23.122247 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:23.122206    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wc2nb\" (UniqueName: \"kubernetes.io/projected/4f6d23b8-1364-4b03-96c6-6649f3a0644a-kube-api-access-wc2nb\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-xv7qv\" (UID: \"4f6d23b8-1364-4b03-96c6-6649f3a0644a\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-xv7qv"
Oct 30 21:35:23.173536 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:23.173506    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-xv7qv"
Oct 30 21:35:23.365385 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:23.365284    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 21:35:23.451940 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:23.451913    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-xv7qv"]
Oct 30 21:35:23.454202 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:35:23.454167    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f6d23b8_1364_4b03_96c6_6649f3a0644a.slice/crio-5cd9789ae975e97bd7ea0f1e8d7598e8df67859a69ecae69dd61176e0dfa51cf WatchSource:0}: Error finding container 5cd9789ae975e97bd7ea0f1e8d7598e8df67859a69ecae69dd61176e0dfa51cf: Status 404 returned error can't find the container with id 5cd9789ae975e97bd7ea0f1e8d7598e8df67859a69ecae69dd61176e0dfa51cf
Oct 30 21:35:23.625614 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:23.625524    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-xv7qv" event={"ID":"4f6d23b8-1364-4b03-96c6-6649f3a0644a","Type":"ContainerStarted","Data":"5cd9789ae975e97bd7ea0f1e8d7598e8df67859a69ecae69dd61176e0dfa51cf"}
Oct 30 21:35:24.782331 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:24.782294    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-d9q4t"]
Oct 30 21:35:24.786502 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:24.786477    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-d9q4t"
Oct 30 21:35:24.796860 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:24.796833    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Oct 30 21:35:24.798543 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:24.798519    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-d9q4t"]
Oct 30 21:35:24.824389 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:24.824348    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/294414e8-4fc5-48ec-9c47-9660381fda80-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-d9q4t\" (UID: \"294414e8-4fc5-48ec-9c47-9660381fda80\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-d9q4t"
Oct 30 21:35:24.824389 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:24.824394    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/294414e8-4fc5-48ec-9c47-9660381fda80-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-d9q4t\" (UID: \"294414e8-4fc5-48ec-9c47-9660381fda80\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-d9q4t"
Oct 30 21:35:24.824644 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:24.824430    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/294414e8-4fc5-48ec-9c47-9660381fda80-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-d9q4t\" (UID: \"294414e8-4fc5-48ec-9c47-9660381fda80\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-d9q4t"
Oct 30 21:35:24.824644 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:24.824469    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/294414e8-4fc5-48ec-9c47-9660381fda80-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-d9q4t\" (UID: \"294414e8-4fc5-48ec-9c47-9660381fda80\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-d9q4t"
Oct 30 21:35:24.824644 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:24.824625    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvrhr\" (UniqueName: \"kubernetes.io/projected/294414e8-4fc5-48ec-9c47-9660381fda80-kube-api-access-cvrhr\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-d9q4t\" (UID: \"294414e8-4fc5-48ec-9c47-9660381fda80\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-d9q4t"
Oct 30 21:35:24.824798 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:24.824753    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/294414e8-4fc5-48ec-9c47-9660381fda80-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-d9q4t\" (UID: \"294414e8-4fc5-48ec-9c47-9660381fda80\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-d9q4t"
Oct 30 21:35:24.925467 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:24.925425    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/294414e8-4fc5-48ec-9c47-9660381fda80-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-d9q4t\" (UID: \"294414e8-4fc5-48ec-9c47-9660381fda80\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-d9q4t"
Oct 30 21:35:24.925662 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:24.925506    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/294414e8-4fc5-48ec-9c47-9660381fda80-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-d9q4t\" (UID: \"294414e8-4fc5-48ec-9c47-9660381fda80\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-d9q4t"
Oct 30 21:35:24.925662 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:24.925528    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/294414e8-4fc5-48ec-9c47-9660381fda80-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-d9q4t\" (UID: \"294414e8-4fc5-48ec-9c47-9660381fda80\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-d9q4t"
Oct 30 21:35:24.925662 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:24.925566    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/294414e8-4fc5-48ec-9c47-9660381fda80-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-d9q4t\" (UID: \"294414e8-4fc5-48ec-9c47-9660381fda80\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-d9q4t"
Oct 30 21:35:24.925662 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:24.925601    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/294414e8-4fc5-48ec-9c47-9660381fda80-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-d9q4t\" (UID: \"294414e8-4fc5-48ec-9c47-9660381fda80\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-d9q4t"
Oct 30 21:35:24.925886 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:24.925710    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvrhr\" (UniqueName: \"kubernetes.io/projected/294414e8-4fc5-48ec-9c47-9660381fda80-kube-api-access-cvrhr\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-d9q4t\" (UID: \"294414e8-4fc5-48ec-9c47-9660381fda80\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-d9q4t"
Oct 30 21:35:24.925970 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:24.925921    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/294414e8-4fc5-48ec-9c47-9660381fda80-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-d9q4t\" (UID: \"294414e8-4fc5-48ec-9c47-9660381fda80\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-d9q4t"
Oct 30 21:35:24.926393 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:24.926336    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/294414e8-4fc5-48ec-9c47-9660381fda80-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-d9q4t\" (UID: \"294414e8-4fc5-48ec-9c47-9660381fda80\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-d9q4t"
Oct 30 21:35:24.926536 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:24.926397    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/294414e8-4fc5-48ec-9c47-9660381fda80-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-d9q4t\" (UID: \"294414e8-4fc5-48ec-9c47-9660381fda80\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-d9q4t"
Oct 30 21:35:24.928572 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:24.928547    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/294414e8-4fc5-48ec-9c47-9660381fda80-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-d9q4t\" (UID: \"294414e8-4fc5-48ec-9c47-9660381fda80\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-d9q4t"
Oct 30 21:35:24.928701 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:24.928676    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/294414e8-4fc5-48ec-9c47-9660381fda80-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-d9q4t\" (UID: \"294414e8-4fc5-48ec-9c47-9660381fda80\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-d9q4t"
Oct 30 21:35:24.938508 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:24.938483    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvrhr\" (UniqueName: \"kubernetes.io/projected/294414e8-4fc5-48ec-9c47-9660381fda80-kube-api-access-cvrhr\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-d9q4t\" (UID: \"294414e8-4fc5-48ec-9c47-9660381fda80\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-d9q4t"
Oct 30 21:35:25.103124 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:25.103034    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-d9q4t"
Oct 30 21:35:25.381052 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:25.381023    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-d9q4t"]
Oct 30 21:35:25.383574 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:35:25.383546    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod294414e8_4fc5_48ec_9c47_9660381fda80.slice/crio-783d0c6f2ca1afa17ef21dd64f4abb75a71f2bfb9fb1cc9ef8afd76373f4b308 WatchSource:0}: Error finding container 783d0c6f2ca1afa17ef21dd64f4abb75a71f2bfb9fb1cc9ef8afd76373f4b308: Status 404 returned error can't find the container with id 783d0c6f2ca1afa17ef21dd64f4abb75a71f2bfb9fb1cc9ef8afd76373f4b308
Oct 30 21:35:25.636060 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:25.635983    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 21:35:25.640367 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:25.640337    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-d9q4t" event={"ID":"294414e8-4fc5-48ec-9c47-9660381fda80","Type":"ContainerStarted","Data":"783d0c6f2ca1afa17ef21dd64f4abb75a71f2bfb9fb1cc9ef8afd76373f4b308"}
Oct 30 21:35:25.641822 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:25.641796    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-xv7qv" event={"ID":"4f6d23b8-1364-4b03-96c6-6649f3a0644a","Type":"ContainerStarted","Data":"47bc7c50b5183576cc017da7cb69f3a85d65cead510371e4b11375b5b37ab6aa"}
Oct 30 21:35:25.662328 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:25.662105    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-xv7qv" podStartSLOduration=2.41333279 podStartE2EDuration="3.662089423s" totalImagesPullingTime="1.248756633s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:35:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:35:25.662079084 +0000 UTC m=+438.463891571" watchObservedRunningTime="2026-10-30 21:35:25.662089423 +0000 UTC m=+438.463901888"
Oct 30 21:35:26.174303 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:26.174260    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-xv7qv"
Oct 30 21:35:26.649780 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:26.649744    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-d9q4t" event={"ID":"294414e8-4fc5-48ec-9c47-9660381fda80","Type":"ContainerStarted","Data":"307e769db84f0b24cb00282d3b5017ba9817c293f5067f3a3f9b8c00994c700b"}
Oct 30 21:35:26.672743 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:26.672687    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-d9q4t" podStartSLOduration=2.382123945 podStartE2EDuration="2.672671963s" totalImagesPullingTime="290.548018ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:35:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:35:26.671229846 +0000 UTC m=+439.473042310" watchObservedRunningTime="2026-10-30 21:35:26.672671963 +0000 UTC m=+439.474484427"
Oct 30 21:35:27.103551 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:27.103509    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-d9q4t"
Oct 30 21:35:27.617720 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:27.617683    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 21:35:28.644668 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:28.644636    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-vg752"]
Oct 30 21:35:28.648346 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:28.648322    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/echo-simulator-kserve-64b6f8f96c-vg752"
Oct 30 21:35:28.660941 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:28.659386    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"echo-simulator-kserve-self-signed-certs\""
Oct 30 21:35:28.661768 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:28.661738    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-vg752"]
Oct 30 21:35:28.664318 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:28.664293    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8fa31603-e280-4566-bded-aaebc80c2179-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-vg752\" (UID: \"8fa31603-e280-4566-bded-aaebc80c2179\") " pod="llm/echo-simulator-kserve-64b6f8f96c-vg752"
Oct 30 21:35:28.664436 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:28.664330    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8fa31603-e280-4566-bded-aaebc80c2179-home\") pod \"echo-simulator-kserve-64b6f8f96c-vg752\" (UID: \"8fa31603-e280-4566-bded-aaebc80c2179\") " pod="llm/echo-simulator-kserve-64b6f8f96c-vg752"
Oct 30 21:35:28.664436 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:28.664398    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g29t4\" (UniqueName: \"kubernetes.io/projected/8fa31603-e280-4566-bded-aaebc80c2179-kube-api-access-g29t4\") pod \"echo-simulator-kserve-64b6f8f96c-vg752\" (UID: \"8fa31603-e280-4566-bded-aaebc80c2179\") " pod="llm/echo-simulator-kserve-64b6f8f96c-vg752"
Oct 30 21:35:28.664538 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:28.664508    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8fa31603-e280-4566-bded-aaebc80c2179-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-vg752\" (UID: \"8fa31603-e280-4566-bded-aaebc80c2179\") " pod="llm/echo-simulator-kserve-64b6f8f96c-vg752"
Oct 30 21:35:28.664616 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:28.664560    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8fa31603-e280-4566-bded-aaebc80c2179-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-vg752\" (UID: \"8fa31603-e280-4566-bded-aaebc80c2179\") " pod="llm/echo-simulator-kserve-64b6f8f96c-vg752"
Oct 30 21:35:28.664658 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:28.664629    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8fa31603-e280-4566-bded-aaebc80c2179-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-vg752\" (UID: \"8fa31603-e280-4566-bded-aaebc80c2179\") " pod="llm/echo-simulator-kserve-64b6f8f96c-vg752"
Oct 30 21:35:28.765956 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:28.765916    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8fa31603-e280-4566-bded-aaebc80c2179-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-vg752\" (UID: \"8fa31603-e280-4566-bded-aaebc80c2179\") " pod="llm/echo-simulator-kserve-64b6f8f96c-vg752"
Oct 30 21:35:28.765956 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:28.765961    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8fa31603-e280-4566-bded-aaebc80c2179-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-vg752\" (UID: \"8fa31603-e280-4566-bded-aaebc80c2179\") " pod="llm/echo-simulator-kserve-64b6f8f96c-vg752"
Oct 30 21:35:28.766184 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:28.766038    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8fa31603-e280-4566-bded-aaebc80c2179-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-vg752\" (UID: \"8fa31603-e280-4566-bded-aaebc80c2179\") " pod="llm/echo-simulator-kserve-64b6f8f96c-vg752"
Oct 30 21:35:28.766232 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:28.766202    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8fa31603-e280-4566-bded-aaebc80c2179-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-vg752\" (UID: \"8fa31603-e280-4566-bded-aaebc80c2179\") " pod="llm/echo-simulator-kserve-64b6f8f96c-vg752"
Oct 30 21:35:28.766278 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:28.766256    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8fa31603-e280-4566-bded-aaebc80c2179-home\") pod \"echo-simulator-kserve-64b6f8f96c-vg752\" (UID: \"8fa31603-e280-4566-bded-aaebc80c2179\") " pod="llm/echo-simulator-kserve-64b6f8f96c-vg752"
Oct 30 21:35:28.766393 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:28.766330    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g29t4\" (UniqueName: \"kubernetes.io/projected/8fa31603-e280-4566-bded-aaebc80c2179-kube-api-access-g29t4\") pod \"echo-simulator-kserve-64b6f8f96c-vg752\" (UID: \"8fa31603-e280-4566-bded-aaebc80c2179\") " pod="llm/echo-simulator-kserve-64b6f8f96c-vg752"
Oct 30 21:35:28.766464 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:28.766435    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8fa31603-e280-4566-bded-aaebc80c2179-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-vg752\" (UID: \"8fa31603-e280-4566-bded-aaebc80c2179\") " pod="llm/echo-simulator-kserve-64b6f8f96c-vg752"
Oct 30 21:35:28.766578 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:28.766551    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8fa31603-e280-4566-bded-aaebc80c2179-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-vg752\" (UID: \"8fa31603-e280-4566-bded-aaebc80c2179\") " pod="llm/echo-simulator-kserve-64b6f8f96c-vg752"
Oct 30 21:35:28.766794 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:28.766772    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8fa31603-e280-4566-bded-aaebc80c2179-home\") pod \"echo-simulator-kserve-64b6f8f96c-vg752\" (UID: \"8fa31603-e280-4566-bded-aaebc80c2179\") " pod="llm/echo-simulator-kserve-64b6f8f96c-vg752"
Oct 30 21:35:28.768409 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:28.768391    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8fa31603-e280-4566-bded-aaebc80c2179-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-vg752\" (UID: \"8fa31603-e280-4566-bded-aaebc80c2179\") " pod="llm/echo-simulator-kserve-64b6f8f96c-vg752"
Oct 30 21:35:28.768766 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:28.768747    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8fa31603-e280-4566-bded-aaebc80c2179-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-vg752\" (UID: \"8fa31603-e280-4566-bded-aaebc80c2179\") " pod="llm/echo-simulator-kserve-64b6f8f96c-vg752"
Oct 30 21:35:28.779154 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:28.779126    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g29t4\" (UniqueName: \"kubernetes.io/projected/8fa31603-e280-4566-bded-aaebc80c2179-kube-api-access-g29t4\") pod \"echo-simulator-kserve-64b6f8f96c-vg752\" (UID: \"8fa31603-e280-4566-bded-aaebc80c2179\") " pod="llm/echo-simulator-kserve-64b6f8f96c-vg752"
Oct 30 21:35:28.966605 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:28.966569    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/echo-simulator-kserve-64b6f8f96c-vg752"
Oct 30 21:35:29.245756 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:29.245731    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-vg752"]
Oct 30 21:35:29.248729 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:35:29.248700    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fa31603_e280_4566_bded_aaebc80c2179.slice/crio-b3e68c94a02e54164e2fc37a8da521df9b931c157fe6977e6a58afeb80d4ea7e WatchSource:0}: Error finding container b3e68c94a02e54164e2fc37a8da521df9b931c157fe6977e6a58afeb80d4ea7e: Status 404 returned error can't find the container with id b3e68c94a02e54164e2fc37a8da521df9b931c157fe6977e6a58afeb80d4ea7e
Oct 30 21:35:29.673933 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:29.673848    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/echo-simulator-kserve-64b6f8f96c-vg752" event={"ID":"8fa31603-e280-4566-bded-aaebc80c2179","Type":"ContainerStarted","Data":"b3e68c94a02e54164e2fc37a8da521df9b931c157fe6977e6a58afeb80d4ea7e"}
Oct 30 21:35:31.115137 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:31.115097    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 21:35:31.686879 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:31.686843    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/echo-simulator-kserve-64b6f8f96c-vg752" event={"ID":"8fa31603-e280-4566-bded-aaebc80c2179","Type":"ContainerStarted","Data":"01556fe4b2eeaf35946c8098f947284b9a435e4ed9549e96ce9713996a81f42d"}
Oct 30 21:35:31.709321 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:31.709265    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/echo-simulator-kserve-64b6f8f96c-vg752" podStartSLOduration=1.9250435430000001 podStartE2EDuration="3.709250568s" totalImagesPullingTime="1.784207025s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:35:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:35:31.707564695 +0000 UTC m=+444.509377160" watchObservedRunningTime="2026-10-30 21:35:31.709250568 +0000 UTC m=+444.511063043"
Oct 30 21:35:31.966902 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:31.966864    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/echo-simulator-kserve-64b6f8f96c-vg752"
Oct 30 21:35:33.174620 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:33.174585    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-xv7qv"
Oct 30 21:35:33.187220 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:33.187191    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-xv7qv"
Oct 30 21:35:33.707906 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:33.707877    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-xv7qv"
Oct 30 21:35:34.846418 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:34.846381    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 21:35:35.103618 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:35.103529    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-d9q4t"
Oct 30 21:35:35.116556 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:35.116526    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-d9q4t"
Oct 30 21:35:35.177795 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:35.177764    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-jx4qg"]
Oct 30 21:35:35.181735 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:35.181712    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-jx4qg"
Oct 30 21:35:35.193490 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:35.193465    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Oct 30 21:35:35.194387 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:35.194328    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-jx4qg"]
Oct 30 21:35:35.231737 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:35.231700    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c945ece-1ac0-4993-aea4-e853a23a4099-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-jx4qg\" (UID: \"0c945ece-1ac0-4993-aea4-e853a23a4099\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-jx4qg"
Oct 30 21:35:35.231948 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:35.231850    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0c945ece-1ac0-4993-aea4-e853a23a4099-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-jx4qg\" (UID: \"0c945ece-1ac0-4993-aea4-e853a23a4099\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-jx4qg"
Oct 30 21:35:35.231948 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:35.231905    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0c945ece-1ac0-4993-aea4-e853a23a4099-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-jx4qg\" (UID: \"0c945ece-1ac0-4993-aea4-e853a23a4099\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-jx4qg"
Oct 30 21:35:35.232638 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:35.231972    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kc5m8\" (UniqueName: \"kubernetes.io/projected/0c945ece-1ac0-4993-aea4-e853a23a4099-kube-api-access-kc5m8\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-jx4qg\" (UID: \"0c945ece-1ac0-4993-aea4-e853a23a4099\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-jx4qg"
Oct 30 21:35:35.232786 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:35.232658    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c945ece-1ac0-4993-aea4-e853a23a4099-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-jx4qg\" (UID: \"0c945ece-1ac0-4993-aea4-e853a23a4099\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-jx4qg"
Oct 30 21:35:35.232786 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:35.232705    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0c945ece-1ac0-4993-aea4-e853a23a4099-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-jx4qg\" (UID: \"0c945ece-1ac0-4993-aea4-e853a23a4099\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-jx4qg"
Oct 30 21:35:35.333838 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:35.333800    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0c945ece-1ac0-4993-aea4-e853a23a4099-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-jx4qg\" (UID: \"0c945ece-1ac0-4993-aea4-e853a23a4099\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-jx4qg"
Oct 30 21:35:35.334010 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:35.333859    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kc5m8\" (UniqueName: \"kubernetes.io/projected/0c945ece-1ac0-4993-aea4-e853a23a4099-kube-api-access-kc5m8\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-jx4qg\" (UID: \"0c945ece-1ac0-4993-aea4-e853a23a4099\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-jx4qg"
Oct 30 21:35:35.334010 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:35.333978    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c945ece-1ac0-4993-aea4-e853a23a4099-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-jx4qg\" (UID: \"0c945ece-1ac0-4993-aea4-e853a23a4099\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-jx4qg"
Oct 30 21:35:35.334116 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:35.334068    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0c945ece-1ac0-4993-aea4-e853a23a4099-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-jx4qg\" (UID: \"0c945ece-1ac0-4993-aea4-e853a23a4099\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-jx4qg"
Oct 30 21:35:35.334181 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:35.334129    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c945ece-1ac0-4993-aea4-e853a23a4099-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-jx4qg\" (UID: \"0c945ece-1ac0-4993-aea4-e853a23a4099\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-jx4qg"
Oct 30 21:35:35.334242 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:35.334198    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0c945ece-1ac0-4993-aea4-e853a23a4099-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-jx4qg\" (UID: \"0c945ece-1ac0-4993-aea4-e853a23a4099\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-jx4qg"
Oct 30 21:35:35.334462 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:35.334439    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c945ece-1ac0-4993-aea4-e853a23a4099-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-jx4qg\" (UID: \"0c945ece-1ac0-4993-aea4-e853a23a4099\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-jx4qg"
Oct 30 21:35:35.334498 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:35.334446    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0c945ece-1ac0-4993-aea4-e853a23a4099-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-jx4qg\" (UID: \"0c945ece-1ac0-4993-aea4-e853a23a4099\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-jx4qg"
Oct 30 21:35:35.334533 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:35.334504    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0c945ece-1ac0-4993-aea4-e853a23a4099-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-jx4qg\" (UID: \"0c945ece-1ac0-4993-aea4-e853a23a4099\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-jx4qg"
Oct 30 21:35:35.336133 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:35.336101    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0c945ece-1ac0-4993-aea4-e853a23a4099-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-jx4qg\" (UID: \"0c945ece-1ac0-4993-aea4-e853a23a4099\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-jx4qg"
Oct 30 21:35:35.336510 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:35.336491    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c945ece-1ac0-4993-aea4-e853a23a4099-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-jx4qg\" (UID: \"0c945ece-1ac0-4993-aea4-e853a23a4099\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-jx4qg"
Oct 30 21:35:35.346730 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:35.346709    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kc5m8\" (UniqueName: \"kubernetes.io/projected/0c945ece-1ac0-4993-aea4-e853a23a4099-kube-api-access-kc5m8\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-jx4qg\" (UID: \"0c945ece-1ac0-4993-aea4-e853a23a4099\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-jx4qg"
Oct 30 21:35:35.497442 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:35.497400    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-jx4qg"
Oct 30 21:35:35.717718 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:35.717692    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-d9q4t"
Oct 30 21:35:35.779208 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:35.779183    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-jx4qg"]
Oct 30 21:35:35.781281 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:35:35.781251    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c945ece_1ac0_4993_aea4_e853a23a4099.slice/crio-bbab0d647f3a4e0d602cc8e77bb5954bb8886999f6341ccfac4a052009ff4ab1 WatchSource:0}: Error finding container bbab0d647f3a4e0d602cc8e77bb5954bb8886999f6341ccfac4a052009ff4ab1: Status 404 returned error can't find the container with id bbab0d647f3a4e0d602cc8e77bb5954bb8886999f6341ccfac4a052009ff4ab1
Oct 30 21:35:36.714311 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:36.714270    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-jx4qg" event={"ID":"0c945ece-1ac0-4993-aea4-e853a23a4099","Type":"ContainerStarted","Data":"dcba9b198dbcc6a2171d473eb78b7f806224009abd568cca133d1e5fd752bd50"}
Oct 30 21:35:36.714769 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:36.714336    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-jx4qg" event={"ID":"0c945ece-1ac0-4993-aea4-e853a23a4099","Type":"ContainerStarted","Data":"bbab0d647f3a4e0d602cc8e77bb5954bb8886999f6341ccfac4a052009ff4ab1"}
Oct 30 21:35:36.743826 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:36.743765    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-jx4qg" podStartSLOduration=1.463338549 podStartE2EDuration="1.743751203s" totalImagesPullingTime="280.412654ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:35:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:35:36.741051811 +0000 UTC m=+449.542864285" watchObservedRunningTime="2026-10-30 21:35:36.743751203 +0000 UTC m=+449.545563664"
Oct 30 21:35:37.497803 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:37.497765    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-jx4qg"
Oct 30 21:35:38.538476 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:38.538447    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 21:35:38.967674 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:38.967643    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/echo-simulator-kserve-64b6f8f96c-vg752"
Oct 30 21:35:38.980168 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:38.980141    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/echo-simulator-kserve-64b6f8f96c-vg752"
Oct 30 21:35:39.741583 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:39.741556    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/echo-simulator-kserve-64b6f8f96c-vg752"
Oct 30 21:35:45.497850 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:45.497821    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-jx4qg"
Oct 30 21:35:45.510248 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:45.510222    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-jx4qg"
Oct 30 21:35:45.775897 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:35:45.775802    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-jx4qg"
Oct 30 21:35:48.344173 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:35:48.344128    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 21:36:06.975486 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:06.975446    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mxsln"]
Oct 30 21:36:06.984331 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:06.984304    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mxsln"
Oct 30 21:36:07.002176 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:07.002147    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-wb984\""
Oct 30 21:36:07.003090 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:07.003065    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mxsln"]
Oct 30 21:36:07.042118 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:07.041973    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f77afa6c-51f5-4bc2-b10e-89101bd067c8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-mxsln\" (UID: \"f77afa6c-51f5-4bc2-b10e-89101bd067c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mxsln"
Oct 30 21:36:07.042118 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:07.042035    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f77afa6c-51f5-4bc2-b10e-89101bd067c8-istio-token\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-mxsln\" (UID: \"f77afa6c-51f5-4bc2-b10e-89101bd067c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mxsln"
Oct 30 21:36:07.042400 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:07.042138    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f77afa6c-51f5-4bc2-b10e-89101bd067c8-workload-socket\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-mxsln\" (UID: \"f77afa6c-51f5-4bc2-b10e-89101bd067c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mxsln"
Oct 30 21:36:07.042400 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:07.042171    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f77afa6c-51f5-4bc2-b10e-89101bd067c8-workload-certs\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-mxsln\" (UID: \"f77afa6c-51f5-4bc2-b10e-89101bd067c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mxsln"
Oct 30 21:36:07.042400 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:07.042202    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f77afa6c-51f5-4bc2-b10e-89101bd067c8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-mxsln\" (UID: \"f77afa6c-51f5-4bc2-b10e-89101bd067c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mxsln"
Oct 30 21:36:07.042400 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:07.042237    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f77afa6c-51f5-4bc2-b10e-89101bd067c8-credential-socket\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-mxsln\" (UID: \"f77afa6c-51f5-4bc2-b10e-89101bd067c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mxsln"
Oct 30 21:36:07.042400 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:07.042261    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kc9sg\" (UniqueName: \"kubernetes.io/projected/f77afa6c-51f5-4bc2-b10e-89101bd067c8-kube-api-access-kc9sg\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-mxsln\" (UID: \"f77afa6c-51f5-4bc2-b10e-89101bd067c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mxsln"
Oct 30 21:36:07.042400 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:07.042304    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f77afa6c-51f5-4bc2-b10e-89101bd067c8-istio-envoy\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-mxsln\" (UID: \"f77afa6c-51f5-4bc2-b10e-89101bd067c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mxsln"
Oct 30 21:36:07.042659 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:07.042498    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f77afa6c-51f5-4bc2-b10e-89101bd067c8-istio-data\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-mxsln\" (UID: \"f77afa6c-51f5-4bc2-b10e-89101bd067c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mxsln"
Oct 30 21:36:07.143047 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:07.143004    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f77afa6c-51f5-4bc2-b10e-89101bd067c8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-mxsln\" (UID: \"f77afa6c-51f5-4bc2-b10e-89101bd067c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mxsln"
Oct 30 21:36:07.143047 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:07.143052    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f77afa6c-51f5-4bc2-b10e-89101bd067c8-credential-socket\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-mxsln\" (UID: \"f77afa6c-51f5-4bc2-b10e-89101bd067c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mxsln"
Oct 30 21:36:07.143330 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:07.143071    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kc9sg\" (UniqueName: \"kubernetes.io/projected/f77afa6c-51f5-4bc2-b10e-89101bd067c8-kube-api-access-kc9sg\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-mxsln\" (UID: \"f77afa6c-51f5-4bc2-b10e-89101bd067c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mxsln"
Oct 30 21:36:07.143330 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:07.143101    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f77afa6c-51f5-4bc2-b10e-89101bd067c8-istio-envoy\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-mxsln\" (UID: \"f77afa6c-51f5-4bc2-b10e-89101bd067c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mxsln"
Oct 30 21:36:07.143330 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:07.143141    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f77afa6c-51f5-4bc2-b10e-89101bd067c8-istio-data\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-mxsln\" (UID: \"f77afa6c-51f5-4bc2-b10e-89101bd067c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mxsln"
Oct 30 21:36:07.143330 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:07.143177    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f77afa6c-51f5-4bc2-b10e-89101bd067c8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-mxsln\" (UID: \"f77afa6c-51f5-4bc2-b10e-89101bd067c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mxsln"
Oct 30 21:36:07.143330 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:07.143213    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f77afa6c-51f5-4bc2-b10e-89101bd067c8-istio-token\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-mxsln\" (UID: \"f77afa6c-51f5-4bc2-b10e-89101bd067c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mxsln"
Oct 30 21:36:07.143330 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:07.143253    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f77afa6c-51f5-4bc2-b10e-89101bd067c8-workload-socket\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-mxsln\" (UID: \"f77afa6c-51f5-4bc2-b10e-89101bd067c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mxsln"
Oct 30 21:36:07.143330 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:07.143281    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f77afa6c-51f5-4bc2-b10e-89101bd067c8-workload-certs\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-mxsln\" (UID: \"f77afa6c-51f5-4bc2-b10e-89101bd067c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mxsln"
Oct 30 21:36:07.143716 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:07.143573    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f77afa6c-51f5-4bc2-b10e-89101bd067c8-credential-socket\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-mxsln\" (UID: \"f77afa6c-51f5-4bc2-b10e-89101bd067c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mxsln"
Oct 30 21:36:07.143716 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:07.143627    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f77afa6c-51f5-4bc2-b10e-89101bd067c8-istio-data\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-mxsln\" (UID: \"f77afa6c-51f5-4bc2-b10e-89101bd067c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mxsln"
Oct 30 21:36:07.143835 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:07.143813    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f77afa6c-51f5-4bc2-b10e-89101bd067c8-workload-certs\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-mxsln\" (UID: \"f77afa6c-51f5-4bc2-b10e-89101bd067c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mxsln"
Oct 30 21:36:07.143874 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:07.143828    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f77afa6c-51f5-4bc2-b10e-89101bd067c8-workload-socket\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-mxsln\" (UID: \"f77afa6c-51f5-4bc2-b10e-89101bd067c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mxsln"
Oct 30 21:36:07.144168 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:07.144141    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f77afa6c-51f5-4bc2-b10e-89101bd067c8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-mxsln\" (UID: \"f77afa6c-51f5-4bc2-b10e-89101bd067c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mxsln"
Oct 30 21:36:07.145583 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:07.145561    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f77afa6c-51f5-4bc2-b10e-89101bd067c8-istio-envoy\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-mxsln\" (UID: \"f77afa6c-51f5-4bc2-b10e-89101bd067c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mxsln"
Oct 30 21:36:07.145908 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:07.145892    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f77afa6c-51f5-4bc2-b10e-89101bd067c8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-mxsln\" (UID: \"f77afa6c-51f5-4bc2-b10e-89101bd067c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mxsln"
Oct 30 21:36:07.162028 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:07.161987    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f77afa6c-51f5-4bc2-b10e-89101bd067c8-istio-token\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-mxsln\" (UID: \"f77afa6c-51f5-4bc2-b10e-89101bd067c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mxsln"
Oct 30 21:36:07.162166 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:07.162104    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kc9sg\" (UniqueName: \"kubernetes.io/projected/f77afa6c-51f5-4bc2-b10e-89101bd067c8-kube-api-access-kc9sg\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-mxsln\" (UID: \"f77afa6c-51f5-4bc2-b10e-89101bd067c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mxsln"
Oct 30 21:36:07.302014 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:07.301924    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-wb984\""
Oct 30 21:36:07.309015 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:07.308975    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mxsln"
Oct 30 21:36:07.598566 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:07.598492    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mxsln"]
Oct 30 21:36:07.602281 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:36:07.602254    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf77afa6c_51f5_4bc2_b10e_89101bd067c8.slice/crio-1cd80a4fd7c80704bbbcec666b35ac386355749754dd04c54694ac5219cf7cd6 WatchSource:0}: Error finding container 1cd80a4fd7c80704bbbcec666b35ac386355749754dd04c54694ac5219cf7cd6: Status 404 returned error can't find the container with id 1cd80a4fd7c80704bbbcec666b35ac386355749754dd04c54694ac5219cf7cd6
Oct 30 21:36:07.604441 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:07.604410    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:36:07.604518 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:07.604475    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:36:07.604518 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:07.604503    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:36:07.883434 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:07.883311    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mxsln" event={"ID":"f77afa6c-51f5-4bc2-b10e-89101bd067c8","Type":"ContainerStarted","Data":"e5804f87e9f6a15b0702665596cdfc0c2907c0225c19189d9e95cb6ee8d680c6"}
Oct 30 21:36:07.883434 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:07.883408    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mxsln" event={"ID":"f77afa6c-51f5-4bc2-b10e-89101bd067c8","Type":"ContainerStarted","Data":"1cd80a4fd7c80704bbbcec666b35ac386355749754dd04c54694ac5219cf7cd6"}
Oct 30 21:36:07.908630 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:07.908577    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mxsln" podStartSLOduration=1.90856036 podStartE2EDuration="1.90856036s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:36:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:36:07.90709499 +0000 UTC m=+480.708907476" watchObservedRunningTime="2026-10-30 21:36:07.90856036 +0000 UTC m=+480.710372826"
Oct 30 21:36:08.317384 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:08.314720    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mxsln"
Oct 30 21:36:08.335583 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:08.335528    5032 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-mxsln container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Oct 30 21:36:08.336068 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:08.335604    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mxsln" podUID="f77afa6c-51f5-4bc2-b10e-89101bd067c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Oct 30 21:36:08.848188 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:08.848151    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-57889585d4-5rjwf"]
Oct 30 21:36:08.852227 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:08.852208    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-57889585d4-5rjwf"
Oct 30 21:36:08.859301 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:08.859275    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"ai-gateway-controller-parameters\""
Oct 30 21:36:08.862328 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:08.862297    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-dockercfg-d4vm2\""
Oct 30 21:36:08.862429 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:08.862333    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-metrics-tls\""
Oct 30 21:36:08.865956 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:08.865930    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-57889585d4-5rjwf"]
Oct 30 21:36:08.968818 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:08.968777    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4dc7efd7-d1c8-48da-b76f-9faae51f01e2-metrics-certs\") pod \"ai-gateway-controller-57889585d4-5rjwf\" (UID: \"4dc7efd7-d1c8-48da-b76f-9faae51f01e2\") " pod="opendatahub/ai-gateway-controller-57889585d4-5rjwf"
Oct 30 21:36:08.969065 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:08.969042    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9hrd\" (UniqueName: \"kubernetes.io/projected/4dc7efd7-d1c8-48da-b76f-9faae51f01e2-kube-api-access-b9hrd\") pod \"ai-gateway-controller-57889585d4-5rjwf\" (UID: \"4dc7efd7-d1c8-48da-b76f-9faae51f01e2\") " pod="opendatahub/ai-gateway-controller-57889585d4-5rjwf"
Oct 30 21:36:09.070574 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:09.070539    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4dc7efd7-d1c8-48da-b76f-9faae51f01e2-metrics-certs\") pod \"ai-gateway-controller-57889585d4-5rjwf\" (UID: \"4dc7efd7-d1c8-48da-b76f-9faae51f01e2\") " pod="opendatahub/ai-gateway-controller-57889585d4-5rjwf"
Oct 30 21:36:09.070760 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:09.070638    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9hrd\" (UniqueName: \"kubernetes.io/projected/4dc7efd7-d1c8-48da-b76f-9faae51f01e2-kube-api-access-b9hrd\") pod \"ai-gateway-controller-57889585d4-5rjwf\" (UID: \"4dc7efd7-d1c8-48da-b76f-9faae51f01e2\") " pod="opendatahub/ai-gateway-controller-57889585d4-5rjwf"
Oct 30 21:36:09.073302 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:09.073275    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4dc7efd7-d1c8-48da-b76f-9faae51f01e2-metrics-certs\") pod \"ai-gateway-controller-57889585d4-5rjwf\" (UID: \"4dc7efd7-d1c8-48da-b76f-9faae51f01e2\") " pod="opendatahub/ai-gateway-controller-57889585d4-5rjwf"
Oct 30 21:36:09.083426 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:09.083394    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9hrd\" (UniqueName: \"kubernetes.io/projected/4dc7efd7-d1c8-48da-b76f-9faae51f01e2-kube-api-access-b9hrd\") pod \"ai-gateway-controller-57889585d4-5rjwf\" (UID: \"4dc7efd7-d1c8-48da-b76f-9faae51f01e2\") " pod="opendatahub/ai-gateway-controller-57889585d4-5rjwf"
Oct 30 21:36:09.166721 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:09.166635    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-57889585d4-5rjwf"
Oct 30 21:36:09.332409 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:09.309379    5032 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-mxsln container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Oct 30 21:36:09.332409 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:09.309442    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mxsln" podUID="f77afa6c-51f5-4bc2-b10e-89101bd067c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Oct 30 21:36:09.472579 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:09.472553    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-57889585d4-5rjwf"]
Oct 30 21:36:09.475090 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:36:09.475060    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4dc7efd7_d1c8_48da_b76f_9faae51f01e2.slice/crio-4b17748f41696bbf40d5f021ee48e404c7b6611a85060b4b2eac9903e7550156 WatchSource:0}: Error finding container 4b17748f41696bbf40d5f021ee48e404c7b6611a85060b4b2eac9903e7550156: Status 404 returned error can't find the container with id 4b17748f41696bbf40d5f021ee48e404c7b6611a85060b4b2eac9903e7550156
Oct 30 21:36:09.896050 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:09.896008    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-57889585d4-5rjwf" event={"ID":"4dc7efd7-d1c8-48da-b76f-9faae51f01e2","Type":"ContainerStarted","Data":"4b17748f41696bbf40d5f021ee48e404c7b6611a85060b4b2eac9903e7550156"}
Oct 30 21:36:10.333386 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:10.332723    5032 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-mxsln container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Oct 30 21:36:10.333386 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:10.332785    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mxsln" podUID="f77afa6c-51f5-4bc2-b10e-89101bd067c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Oct 30 21:36:11.335399 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:11.325808    5032 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-mxsln container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Oct 30 21:36:11.335399 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:11.325910    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mxsln" podUID="f77afa6c-51f5-4bc2-b10e-89101bd067c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Oct 30 21:36:11.930386 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:11.909439    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-57889585d4-5rjwf" event={"ID":"4dc7efd7-d1c8-48da-b76f-9faae51f01e2","Type":"ContainerStarted","Data":"5653badcb4293074504c6214260ec9077e46470d5ac56f903ca2054033791620"}
Oct 30 21:36:11.930386 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:11.910321    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-57889585d4-5rjwf"
Oct 30 21:36:11.949397 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:11.949336    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-57889585d4-5rjwf" podStartSLOduration=2.194878068 podStartE2EDuration="3.949321694s" totalImagesPullingTime="1.754443626s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:36:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:36:11.947654625 +0000 UTC m=+484.749467089" watchObservedRunningTime="2026-10-30 21:36:11.949321694 +0000 UTC m=+484.751134154"
Oct 30 21:36:12.325841 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:12.325801    5032 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-mxsln container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Oct 30 21:36:12.326152 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:12.326125    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mxsln" podUID="f77afa6c-51f5-4bc2-b10e-89101bd067c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Oct 30 21:36:13.318392 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:13.310484    5032 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-mxsln container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Oct 30 21:36:13.318392 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:13.310545    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mxsln" podUID="f77afa6c-51f5-4bc2-b10e-89101bd067c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Oct 30 21:36:14.309648 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:14.309603    5032 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-mxsln container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Oct 30 21:36:14.309835 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:14.309670    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mxsln" podUID="f77afa6c-51f5-4bc2-b10e-89101bd067c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Oct 30 21:36:15.907100 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:15.907064    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mxsln"
Oct 30 21:36:15.908290 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:15.908265    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mxsln"
Oct 30 21:36:15.908736 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:15.908710    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mxsln"
Oct 30 21:36:18.345083 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:36:18.344999    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 21:36:21.911117 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:21.911078    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-c46gq"]
Oct 30 21:36:21.915065 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:21.915043    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-c46gq"
Oct 30 21:36:21.929237 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:21.929211    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-c46gq"]
Oct 30 21:36:21.944551 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:21.944525    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-7s44p"]
Oct 30 21:36:21.948834 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:21.948812    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-7s44p"
Oct 30 21:36:21.966702 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:21.966675    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-7s44p"]
Oct 30 21:36:22.022009 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.021972    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3b353c1-3812-44aa-b77d-d7ddfc3b7057-istio-envoy\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-c46gq\" (UID: \"e3b353c1-3812-44aa-b77d-d7ddfc3b7057\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-c46gq"
Oct 30 21:36:22.022009 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.022008    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3b353c1-3812-44aa-b77d-d7ddfc3b7057-istio-data\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-c46gq\" (UID: \"e3b353c1-3812-44aa-b77d-d7ddfc3b7057\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-c46gq"
Oct 30 21:36:22.022248 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.022028    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3b353c1-3812-44aa-b77d-d7ddfc3b7057-workload-socket\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-c46gq\" (UID: \"e3b353c1-3812-44aa-b77d-d7ddfc3b7057\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-c46gq"
Oct 30 21:36:22.022248 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.022154    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvqdx\" (UniqueName: \"kubernetes.io/projected/e3b353c1-3812-44aa-b77d-d7ddfc3b7057-kube-api-access-qvqdx\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-c46gq\" (UID: \"e3b353c1-3812-44aa-b77d-d7ddfc3b7057\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-c46gq"
Oct 30 21:36:22.022248 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.022190    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3b353c1-3812-44aa-b77d-d7ddfc3b7057-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-c46gq\" (UID: \"e3b353c1-3812-44aa-b77d-d7ddfc3b7057\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-c46gq"
Oct 30 21:36:22.022248 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.022224    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3b353c1-3812-44aa-b77d-d7ddfc3b7057-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-c46gq\" (UID: \"e3b353c1-3812-44aa-b77d-d7ddfc3b7057\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-c46gq"
Oct 30 21:36:22.022492 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.022301    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3b353c1-3812-44aa-b77d-d7ddfc3b7057-credential-socket\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-c46gq\" (UID: \"e3b353c1-3812-44aa-b77d-d7ddfc3b7057\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-c46gq"
Oct 30 21:36:22.022492 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.022408    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3b353c1-3812-44aa-b77d-d7ddfc3b7057-workload-certs\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-c46gq\" (UID: \"e3b353c1-3812-44aa-b77d-d7ddfc3b7057\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-c46gq"
Oct 30 21:36:22.022492 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.022439    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3b353c1-3812-44aa-b77d-d7ddfc3b7057-istio-token\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-c46gq\" (UID: \"e3b353c1-3812-44aa-b77d-d7ddfc3b7057\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-c46gq"
Oct 30 21:36:22.123954 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.123917    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3b353c1-3812-44aa-b77d-d7ddfc3b7057-workload-socket\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-c46gq\" (UID: \"e3b353c1-3812-44aa-b77d-d7ddfc3b7057\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-c46gq"
Oct 30 21:36:22.124140 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.123963    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/457cd47c-9543-4d82-875f-f1e45d6d5b70-istio-token\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-7s44p\" (UID: \"457cd47c-9543-4d82-875f-f1e45d6d5b70\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-7s44p"
Oct 30 21:36:22.124140 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.124004    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvqdx\" (UniqueName: \"kubernetes.io/projected/e3b353c1-3812-44aa-b77d-d7ddfc3b7057-kube-api-access-qvqdx\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-c46gq\" (UID: \"e3b353c1-3812-44aa-b77d-d7ddfc3b7057\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-c46gq"
Oct 30 21:36:22.124140 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.124038    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3b353c1-3812-44aa-b77d-d7ddfc3b7057-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-c46gq\" (UID: \"e3b353c1-3812-44aa-b77d-d7ddfc3b7057\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-c46gq"
Oct 30 21:36:22.124140 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.124068    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/457cd47c-9543-4d82-875f-f1e45d6d5b70-credential-socket\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-7s44p\" (UID: \"457cd47c-9543-4d82-875f-f1e45d6d5b70\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-7s44p"
Oct 30 21:36:22.124291 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.124262    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/457cd47c-9543-4d82-875f-f1e45d6d5b70-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-7s44p\" (UID: \"457cd47c-9543-4d82-875f-f1e45d6d5b70\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-7s44p"
Oct 30 21:36:22.124336 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.124291    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3b353c1-3812-44aa-b77d-d7ddfc3b7057-workload-socket\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-c46gq\" (UID: \"e3b353c1-3812-44aa-b77d-d7ddfc3b7057\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-c46gq"
Oct 30 21:36:22.124336 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.124298    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3b353c1-3812-44aa-b77d-d7ddfc3b7057-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-c46gq\" (UID: \"e3b353c1-3812-44aa-b77d-d7ddfc3b7057\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-c46gq"
Oct 30 21:36:22.124497 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.124479    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3b353c1-3812-44aa-b77d-d7ddfc3b7057-credential-socket\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-c46gq\" (UID: \"e3b353c1-3812-44aa-b77d-d7ddfc3b7057\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-c46gq"
Oct 30 21:36:22.124540 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.124526    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3b353c1-3812-44aa-b77d-d7ddfc3b7057-workload-certs\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-c46gq\" (UID: \"e3b353c1-3812-44aa-b77d-d7ddfc3b7057\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-c46gq"
Oct 30 21:36:22.124584 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.124554    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3b353c1-3812-44aa-b77d-d7ddfc3b7057-istio-token\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-c46gq\" (UID: \"e3b353c1-3812-44aa-b77d-d7ddfc3b7057\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-c46gq"
Oct 30 21:36:22.124619 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.124599    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/457cd47c-9543-4d82-875f-f1e45d6d5b70-istio-envoy\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-7s44p\" (UID: \"457cd47c-9543-4d82-875f-f1e45d6d5b70\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-7s44p"
Oct 30 21:36:22.124664 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.124648    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/457cd47c-9543-4d82-875f-f1e45d6d5b70-istio-data\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-7s44p\" (UID: \"457cd47c-9543-4d82-875f-f1e45d6d5b70\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-7s44p"
Oct 30 21:36:22.124807 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.124770    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/457cd47c-9543-4d82-875f-f1e45d6d5b70-workload-certs\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-7s44p\" (UID: \"457cd47c-9543-4d82-875f-f1e45d6d5b70\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-7s44p"
Oct 30 21:36:22.124946 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.124826    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/457cd47c-9543-4d82-875f-f1e45d6d5b70-workload-socket\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-7s44p\" (UID: \"457cd47c-9543-4d82-875f-f1e45d6d5b70\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-7s44p"
Oct 30 21:36:22.124946 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.124834    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3b353c1-3812-44aa-b77d-d7ddfc3b7057-workload-certs\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-c46gq\" (UID: \"e3b353c1-3812-44aa-b77d-d7ddfc3b7057\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-c46gq"
Oct 30 21:36:22.124946 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.124868    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3b353c1-3812-44aa-b77d-d7ddfc3b7057-credential-socket\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-c46gq\" (UID: \"e3b353c1-3812-44aa-b77d-d7ddfc3b7057\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-c46gq"
Oct 30 21:36:22.124946 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.124902    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/457cd47c-9543-4d82-875f-f1e45d6d5b70-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-7s44p\" (UID: \"457cd47c-9543-4d82-875f-f1e45d6d5b70\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-7s44p"
Oct 30 21:36:22.124946 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.124941    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jztf6\" (UniqueName: \"kubernetes.io/projected/457cd47c-9543-4d82-875f-f1e45d6d5b70-kube-api-access-jztf6\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-7s44p\" (UID: \"457cd47c-9543-4d82-875f-f1e45d6d5b70\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-7s44p"
Oct 30 21:36:22.125128 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.124992    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3b353c1-3812-44aa-b77d-d7ddfc3b7057-istio-envoy\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-c46gq\" (UID: \"e3b353c1-3812-44aa-b77d-d7ddfc3b7057\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-c46gq"
Oct 30 21:36:22.125128 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.125031    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3b353c1-3812-44aa-b77d-d7ddfc3b7057-istio-data\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-c46gq\" (UID: \"e3b353c1-3812-44aa-b77d-d7ddfc3b7057\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-c46gq"
Oct 30 21:36:22.125128 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.125047    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3b353c1-3812-44aa-b77d-d7ddfc3b7057-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-c46gq\" (UID: \"e3b353c1-3812-44aa-b77d-d7ddfc3b7057\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-c46gq"
Oct 30 21:36:22.125292 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.125275    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3b353c1-3812-44aa-b77d-d7ddfc3b7057-istio-data\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-c46gq\" (UID: \"e3b353c1-3812-44aa-b77d-d7ddfc3b7057\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-c46gq"
Oct 30 21:36:22.126831 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.126803    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3b353c1-3812-44aa-b77d-d7ddfc3b7057-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-c46gq\" (UID: \"e3b353c1-3812-44aa-b77d-d7ddfc3b7057\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-c46gq"
Oct 30 21:36:22.126982 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.126967    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3b353c1-3812-44aa-b77d-d7ddfc3b7057-istio-envoy\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-c46gq\" (UID: \"e3b353c1-3812-44aa-b77d-d7ddfc3b7057\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-c46gq"
Oct 30 21:36:22.136986 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.136966    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvqdx\" (UniqueName: \"kubernetes.io/projected/e3b353c1-3812-44aa-b77d-d7ddfc3b7057-kube-api-access-qvqdx\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-c46gq\" (UID: \"e3b353c1-3812-44aa-b77d-d7ddfc3b7057\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-c46gq"
Oct 30 21:36:22.137098 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.137028    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3b353c1-3812-44aa-b77d-d7ddfc3b7057-istio-token\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-c46gq\" (UID: \"e3b353c1-3812-44aa-b77d-d7ddfc3b7057\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-c46gq"
Oct 30 21:36:22.226226 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.226194    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/457cd47c-9543-4d82-875f-f1e45d6d5b70-workload-socket\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-7s44p\" (UID: \"457cd47c-9543-4d82-875f-f1e45d6d5b70\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-7s44p"
Oct 30 21:36:22.226226 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.226227    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/457cd47c-9543-4d82-875f-f1e45d6d5b70-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-7s44p\" (UID: \"457cd47c-9543-4d82-875f-f1e45d6d5b70\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-7s44p"
Oct 30 21:36:22.226480 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.226245    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jztf6\" (UniqueName: \"kubernetes.io/projected/457cd47c-9543-4d82-875f-f1e45d6d5b70-kube-api-access-jztf6\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-7s44p\" (UID: \"457cd47c-9543-4d82-875f-f1e45d6d5b70\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-7s44p"
Oct 30 21:36:22.226480 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.226278    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/457cd47c-9543-4d82-875f-f1e45d6d5b70-istio-token\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-7s44p\" (UID: \"457cd47c-9543-4d82-875f-f1e45d6d5b70\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-7s44p"
Oct 30 21:36:22.226480 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.226321    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/457cd47c-9543-4d82-875f-f1e45d6d5b70-credential-socket\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-7s44p\" (UID: \"457cd47c-9543-4d82-875f-f1e45d6d5b70\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-7s44p"
Oct 30 21:36:22.226480 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.226342    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/457cd47c-9543-4d82-875f-f1e45d6d5b70-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-7s44p\" (UID: \"457cd47c-9543-4d82-875f-f1e45d6d5b70\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-7s44p"
Oct 30 21:36:22.226480 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.226423    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/457cd47c-9543-4d82-875f-f1e45d6d5b70-istio-envoy\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-7s44p\" (UID: \"457cd47c-9543-4d82-875f-f1e45d6d5b70\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-7s44p"
Oct 30 21:36:22.226480 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.226475    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/457cd47c-9543-4d82-875f-f1e45d6d5b70-istio-data\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-7s44p\" (UID: \"457cd47c-9543-4d82-875f-f1e45d6d5b70\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-7s44p"
Oct 30 21:36:22.226779 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.226512    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/457cd47c-9543-4d82-875f-f1e45d6d5b70-workload-certs\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-7s44p\" (UID: \"457cd47c-9543-4d82-875f-f1e45d6d5b70\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-7s44p"
Oct 30 21:36:22.226779 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.226722    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/457cd47c-9543-4d82-875f-f1e45d6d5b70-workload-socket\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-7s44p\" (UID: \"457cd47c-9543-4d82-875f-f1e45d6d5b70\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-7s44p"
Oct 30 21:36:22.226779 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.226747    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/457cd47c-9543-4d82-875f-f1e45d6d5b70-credential-socket\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-7s44p\" (UID: \"457cd47c-9543-4d82-875f-f1e45d6d5b70\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-7s44p"
Oct 30 21:36:22.226941 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.226832    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/457cd47c-9543-4d82-875f-f1e45d6d5b70-workload-certs\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-7s44p\" (UID: \"457cd47c-9543-4d82-875f-f1e45d6d5b70\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-7s44p"
Oct 30 21:36:22.227000 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.226940    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/457cd47c-9543-4d82-875f-f1e45d6d5b70-istio-data\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-7s44p\" (UID: \"457cd47c-9543-4d82-875f-f1e45d6d5b70\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-7s44p"
Oct 30 21:36:22.227214 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.227194    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/457cd47c-9543-4d82-875f-f1e45d6d5b70-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-7s44p\" (UID: \"457cd47c-9543-4d82-875f-f1e45d6d5b70\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-7s44p"
Oct 30 21:36:22.228666 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.228642    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-c46gq"
Oct 30 21:36:22.228777 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.228729    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/457cd47c-9543-4d82-875f-f1e45d6d5b70-istio-envoy\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-7s44p\" (UID: \"457cd47c-9543-4d82-875f-f1e45d6d5b70\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-7s44p"
Oct 30 21:36:22.228870 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.228853    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/457cd47c-9543-4d82-875f-f1e45d6d5b70-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-7s44p\" (UID: \"457cd47c-9543-4d82-875f-f1e45d6d5b70\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-7s44p"
Oct 30 21:36:22.239228 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.239206    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jztf6\" (UniqueName: \"kubernetes.io/projected/457cd47c-9543-4d82-875f-f1e45d6d5b70-kube-api-access-jztf6\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-7s44p\" (UID: \"457cd47c-9543-4d82-875f-f1e45d6d5b70\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-7s44p"
Oct 30 21:36:22.239316 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.239211    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/457cd47c-9543-4d82-875f-f1e45d6d5b70-istio-token\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-7s44p\" (UID: \"457cd47c-9543-4d82-875f-f1e45d6d5b70\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-7s44p"
Oct 30 21:36:22.267169 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.267140    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-7s44p"
Oct 30 21:36:22.515666 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.515583    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-c46gq"]
Oct 30 21:36:22.518649 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:36:22.518608    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3b353c1_3812_44aa_b77d_d7ddfc3b7057.slice/crio-c5464eb8fe42558ac04a040fa7aa3b8d64b4e5fa4f25e239b478f784ac304981 WatchSource:0}: Error finding container c5464eb8fe42558ac04a040fa7aa3b8d64b4e5fa4f25e239b478f784ac304981: Status 404 returned error can't find the container with id c5464eb8fe42558ac04a040fa7aa3b8d64b4e5fa4f25e239b478f784ac304981
Oct 30 21:36:22.520920 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.520884    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:36:22.521005 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.520951    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:36:22.521005 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.520983    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:36:22.567267 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.567245    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-7s44p"]
Oct 30 21:36:22.569259 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:36:22.569233    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod457cd47c_9543_4d82_875f_f1e45d6d5b70.slice/crio-63972141d7afb3fd49960aaf50ec4ab14039ff96b796d5431f5f72bbad22b280 WatchSource:0}: Error finding container 63972141d7afb3fd49960aaf50ec4ab14039ff96b796d5431f5f72bbad22b280: Status 404 returned error can't find the container with id 63972141d7afb3fd49960aaf50ec4ab14039ff96b796d5431f5f72bbad22b280
Oct 30 21:36:22.572018 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.571986    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:36:22.572092 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.572054    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:36:22.572133 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.572101    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:36:22.996670 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.996628    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-7s44p" event={"ID":"457cd47c-9543-4d82-875f-f1e45d6d5b70","Type":"ContainerStarted","Data":"527b35e3ebcd49e0dda77f80720c46f83fbea6b5efdc1581e9bf30cea583b07f"}
Oct 30 21:36:22.997130 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.996703    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-7s44p" event={"ID":"457cd47c-9543-4d82-875f-f1e45d6d5b70","Type":"ContainerStarted","Data":"63972141d7afb3fd49960aaf50ec4ab14039ff96b796d5431f5f72bbad22b280"}
Oct 30 21:36:22.998264 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.998238    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-c46gq" event={"ID":"e3b353c1-3812-44aa-b77d-d7ddfc3b7057","Type":"ContainerStarted","Data":"ff5d3f4c84c203d945ce0e069dcab879648c06b1d8d4f2d0a305bfafe899c98d"}
Oct 30 21:36:22.998398 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:22.998279    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-c46gq" event={"ID":"e3b353c1-3812-44aa-b77d-d7ddfc3b7057","Type":"ContainerStarted","Data":"c5464eb8fe42558ac04a040fa7aa3b8d64b4e5fa4f25e239b478f784ac304981"}
Oct 30 21:36:23.028926 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:23.028864    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-7s44p" podStartSLOduration=2.028841912 podStartE2EDuration="2.028841912s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:36:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:36:23.021591373 +0000 UTC m=+495.823403837" watchObservedRunningTime="2026-10-30 21:36:23.028841912 +0000 UTC m=+495.830654378"
Oct 30 21:36:23.054727 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:23.054661    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-c46gq" podStartSLOduration=2.05464139 podStartE2EDuration="2.05464139s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:36:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:36:23.05096731 +0000 UTC m=+495.852779774" watchObservedRunningTime="2026-10-30 21:36:23.05464139 +0000 UTC m=+495.856453860"
Oct 30 21:36:23.230808 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:23.229786    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-c46gq"
Oct 30 21:36:23.251445 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:23.240572    5032 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-c46gq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Oct 30 21:36:23.251445 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:23.240658    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-c46gq" podUID="e3b353c1-3812-44aa-b77d-d7ddfc3b7057" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Oct 30 21:36:23.267599 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:23.267556    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-7s44p"
Oct 30 21:36:23.269200 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:23.269175    5032 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-7s44p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Oct 30 21:36:23.269347 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:23.269240    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-7s44p" podUID="457cd47c-9543-4d82-875f-f1e45d6d5b70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Oct 30 21:36:23.960426 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:23.959710    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-57889585d4-5rjwf"
Oct 30 21:36:24.244446 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:24.243802    5032 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-c46gq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Oct 30 21:36:24.244446 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:24.243865    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-c46gq" podUID="e3b353c1-3812-44aa-b77d-d7ddfc3b7057" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Oct 30 21:36:24.267693 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:24.267650    5032 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-7s44p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Oct 30 21:36:24.267851 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:24.267711    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-7s44p" podUID="457cd47c-9543-4d82-875f-f1e45d6d5b70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Oct 30 21:36:25.235385 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:25.234786    5032 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-c46gq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Oct 30 21:36:25.235385 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:25.234854    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-c46gq" podUID="e3b353c1-3812-44aa-b77d-d7ddfc3b7057" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Oct 30 21:36:25.268104 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:25.268056    5032 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-7s44p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Oct 30 21:36:25.268506 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:25.268118    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-7s44p" podUID="457cd47c-9543-4d82-875f-f1e45d6d5b70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Oct 30 21:36:26.233384 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:26.232875    5032 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-c46gq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Oct 30 21:36:26.233384 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:26.232972    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-c46gq" podUID="e3b353c1-3812-44aa-b77d-d7ddfc3b7057" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Oct 30 21:36:26.268119 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:26.268084    5032 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-7s44p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Oct 30 21:36:26.268563 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:26.268135    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-7s44p" podUID="457cd47c-9543-4d82-875f-f1e45d6d5b70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Oct 30 21:36:27.230217 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:27.229764    5032 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-c46gq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Oct 30 21:36:27.230217 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:27.229832    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-c46gq" podUID="e3b353c1-3812-44aa-b77d-d7ddfc3b7057" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Oct 30 21:36:27.267755 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:27.267717    5032 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-7s44p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Oct 30 21:36:27.267961 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:27.267780    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-7s44p" podUID="457cd47c-9543-4d82-875f-f1e45d6d5b70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Oct 30 21:36:28.234391 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:28.233870    5032 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-c46gq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Oct 30 21:36:28.234391 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:28.233932    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-c46gq" podUID="e3b353c1-3812-44aa-b77d-d7ddfc3b7057" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Oct 30 21:36:28.267838 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:28.267803    5032 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-7s44p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Oct 30 21:36:28.268145 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:28.268099    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-7s44p" podUID="457cd47c-9543-4d82-875f-f1e45d6d5b70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Oct 30 21:36:29.230125 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:29.230083    5032 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-c46gq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Oct 30 21:36:29.230315 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:29.230141    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-c46gq" podUID="e3b353c1-3812-44aa-b77d-d7ddfc3b7057" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Oct 30 21:36:29.268256 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:29.268207    5032 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-7s44p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Oct 30 21:36:29.268786 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:29.268290    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-7s44p" podUID="457cd47c-9543-4d82-875f-f1e45d6d5b70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Oct 30 21:36:30.969666 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:30.969596    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-c46gq"
Oct 30 21:36:30.970729 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:30.970202    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-c46gq"
Oct 30 21:36:30.971435 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:30.971412    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-c46gq"
Oct 30 21:36:31.045055 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:31.045027    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-7s44p"
Oct 30 21:36:31.045371 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:31.045320    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-7s44p"
Oct 30 21:36:31.046148 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:31.046126    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-7s44p"
Oct 30 21:36:48.344062 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:36:48.344020    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 21:36:57.594009 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:57.593974    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-0fa864-76b48d9455-wxt6j"]
Oct 30 21:36:57.600000 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:57.599976    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-0fa864-76b48d9455-wxt6j"
Oct 30 21:36:57.612218 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:57.612148    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-0fa864\""
Oct 30 21:36:57.612414 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:57.612191    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-0fa864-dockercfg-twwgs\""
Oct 30 21:36:57.619469 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:57.619441    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-0fa864-76b48d9455-wxt6j"]
Oct 30 21:36:57.620104 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:57.620074    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzrkf\" (UniqueName: \"kubernetes.io/projected/18d702a7-00ad-4a6b-8de5-806e5900cd7d-kube-api-access-bzrkf\") pod \"payload-pre-processing-e2e-worker-w4-0fa864-76b48d9455-wxt6j\" (UID: \"18d702a7-00ad-4a6b-8de5-806e5900cd7d\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-0fa864-76b48d9455-wxt6j"
Oct 30 21:36:57.620219 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:57.620200    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18d702a7-00ad-4a6b-8de5-806e5900cd7d-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-0fa864-76b48d9455-wxt6j\" (UID: \"18d702a7-00ad-4a6b-8de5-806e5900cd7d\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-0fa864-76b48d9455-wxt6j"
Oct 30 21:36:57.631015 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:57.630980    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-0fa864-5d85948857-tw7k6"]
Oct 30 21:36:57.635774 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:57.635752    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-0fa864-5d85948857-tw7k6"
Oct 30 21:36:57.654782 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:57.654749    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-0fa864-5d85948857-tw7k6"]
Oct 30 21:36:57.744077 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:57.744015    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18d702a7-00ad-4a6b-8de5-806e5900cd7d-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-0fa864-76b48d9455-wxt6j\" (UID: \"18d702a7-00ad-4a6b-8de5-806e5900cd7d\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-0fa864-76b48d9455-wxt6j"
Oct 30 21:36:57.744294 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:57.744181    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzrkf\" (UniqueName: \"kubernetes.io/projected/18d702a7-00ad-4a6b-8de5-806e5900cd7d-kube-api-access-bzrkf\") pod \"payload-pre-processing-e2e-worker-w4-0fa864-76b48d9455-wxt6j\" (UID: \"18d702a7-00ad-4a6b-8de5-806e5900cd7d\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-0fa864-76b48d9455-wxt6j"
Oct 30 21:36:57.745581 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:57.745503    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18d702a7-00ad-4a6b-8de5-806e5900cd7d-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-0fa864-76b48d9455-wxt6j\" (UID: \"18d702a7-00ad-4a6b-8de5-806e5900cd7d\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-0fa864-76b48d9455-wxt6j"
Oct 30 21:36:57.757272 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:57.757227    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzrkf\" (UniqueName: \"kubernetes.io/projected/18d702a7-00ad-4a6b-8de5-806e5900cd7d-kube-api-access-bzrkf\") pod \"payload-pre-processing-e2e-worker-w4-0fa864-76b48d9455-wxt6j\" (UID: \"18d702a7-00ad-4a6b-8de5-806e5900cd7d\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-0fa864-76b48d9455-wxt6j"
Oct 30 21:36:57.845207 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:57.845122    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76de8529-43f7-4285-864b-f8242fca2cae-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-0fa864-5d85948857-tw7k6\" (UID: \"76de8529-43f7-4285-864b-f8242fca2cae\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-0fa864-5d85948857-tw7k6"
Oct 30 21:36:57.845377 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:57.845302    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fc4b\" (UniqueName: \"kubernetes.io/projected/76de8529-43f7-4285-864b-f8242fca2cae-kube-api-access-8fc4b\") pod \"payload-processing-e2e-worker-w4-0fa864-5d85948857-tw7k6\" (UID: \"76de8529-43f7-4285-864b-f8242fca2cae\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-0fa864-5d85948857-tw7k6"
Oct 30 21:36:57.919880 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:57.919843    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-0fa864-76b48d9455-wxt6j"
Oct 30 21:36:57.946299 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:57.946262    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fc4b\" (UniqueName: \"kubernetes.io/projected/76de8529-43f7-4285-864b-f8242fca2cae-kube-api-access-8fc4b\") pod \"payload-processing-e2e-worker-w4-0fa864-5d85948857-tw7k6\" (UID: \"76de8529-43f7-4285-864b-f8242fca2cae\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-0fa864-5d85948857-tw7k6"
Oct 30 21:36:57.946474 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:57.946423    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76de8529-43f7-4285-864b-f8242fca2cae-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-0fa864-5d85948857-tw7k6\" (UID: \"76de8529-43f7-4285-864b-f8242fca2cae\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-0fa864-5d85948857-tw7k6"
Oct 30 21:36:57.947213 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:57.947184    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76de8529-43f7-4285-864b-f8242fca2cae-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-0fa864-5d85948857-tw7k6\" (UID: \"76de8529-43f7-4285-864b-f8242fca2cae\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-0fa864-5d85948857-tw7k6"
Oct 30 21:36:57.967186 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:57.967152    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fc4b\" (UniqueName: \"kubernetes.io/projected/76de8529-43f7-4285-864b-f8242fca2cae-kube-api-access-8fc4b\") pod \"payload-processing-e2e-worker-w4-0fa864-5d85948857-tw7k6\" (UID: \"76de8529-43f7-4285-864b-f8242fca2cae\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-0fa864-5d85948857-tw7k6"
Oct 30 21:36:58.217040 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:58.217014    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-0fa864-76b48d9455-wxt6j"]
Oct 30 21:36:58.219837 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:36:58.219805    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18d702a7_00ad_4a6b_8de5_806e5900cd7d.slice/crio-49245afbb744dc5657c452638f29a817201827be5971d66b781d804f79214375 WatchSource:0}: Error finding container 49245afbb744dc5657c452638f29a817201827be5971d66b781d804f79214375: Status 404 returned error can't find the container with id 49245afbb744dc5657c452638f29a817201827be5971d66b781d804f79214375
Oct 30 21:36:58.235829 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:58.235799    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-0fa864-76b48d9455-wxt6j" event={"ID":"18d702a7-00ad-4a6b-8de5-806e5900cd7d","Type":"ContainerStarted","Data":"49245afbb744dc5657c452638f29a817201827be5971d66b781d804f79214375"}
Oct 30 21:36:58.263194 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:58.263159    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-0fa864-5d85948857-tw7k6"
Oct 30 21:36:58.438486 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:58.438456    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-caa3dc-85fc67b867-gsjl2"]
Oct 30 21:36:58.442548 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:58.442528    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-caa3dc-85fc67b867-gsjl2"
Oct 30 21:36:58.450784 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:58.450761    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28ed1f60-5004-4fb1-bb80-33cd040611f6-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w5-caa3dc-85fc67b867-gsjl2\" (UID: \"28ed1f60-5004-4fb1-bb80-33cd040611f6\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-caa3dc-85fc67b867-gsjl2"
Oct 30 21:36:58.450893 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:58.450801    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x99kb\" (UniqueName: \"kubernetes.io/projected/28ed1f60-5004-4fb1-bb80-33cd040611f6-kube-api-access-x99kb\") pod \"payload-pre-processing-e2e-worker-w5-caa3dc-85fc67b867-gsjl2\" (UID: \"28ed1f60-5004-4fb1-bb80-33cd040611f6\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-caa3dc-85fc67b867-gsjl2"
Oct 30 21:36:58.452877 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:58.452835    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w5-caa3dc\""
Oct 30 21:36:58.452984 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:58.452889    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w5-caa3dc-dockercfg-7h9zc\""
Oct 30 21:36:58.455311 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:58.455286    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-caa3dc-85fc67b867-gsjl2"]
Oct 30 21:36:58.471251 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:58.471161    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-caa3dc-76db8c686c-pxdzh"]
Oct 30 21:36:58.475335 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:58.475309    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w5-caa3dc-76db8c686c-pxdzh"
Oct 30 21:36:58.484336 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:58.484313    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-caa3dc-76db8c686c-pxdzh"]
Oct 30 21:36:58.556925 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:58.551697    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28ed1f60-5004-4fb1-bb80-33cd040611f6-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w5-caa3dc-85fc67b867-gsjl2\" (UID: \"28ed1f60-5004-4fb1-bb80-33cd040611f6\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-caa3dc-85fc67b867-gsjl2"
Oct 30 21:36:58.556925 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:58.551761    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/467142c6-5675-4dd8-87a0-14e1f9fddc57-plugins-config-volume\") pod \"payload-processing-e2e-worker-w5-caa3dc-76db8c686c-pxdzh\" (UID: \"467142c6-5675-4dd8-87a0-14e1f9fddc57\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-caa3dc-76db8c686c-pxdzh"
Oct 30 21:36:58.556925 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:58.551800    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x99kb\" (UniqueName: \"kubernetes.io/projected/28ed1f60-5004-4fb1-bb80-33cd040611f6-kube-api-access-x99kb\") pod \"payload-pre-processing-e2e-worker-w5-caa3dc-85fc67b867-gsjl2\" (UID: \"28ed1f60-5004-4fb1-bb80-33cd040611f6\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-caa3dc-85fc67b867-gsjl2"
Oct 30 21:36:58.556925 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:58.551841    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6cgfr\" (UniqueName: \"kubernetes.io/projected/467142c6-5675-4dd8-87a0-14e1f9fddc57-kube-api-access-6cgfr\") pod \"payload-processing-e2e-worker-w5-caa3dc-76db8c686c-pxdzh\" (UID: \"467142c6-5675-4dd8-87a0-14e1f9fddc57\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-caa3dc-76db8c686c-pxdzh"
Oct 30 21:36:58.556925 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:58.552930    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28ed1f60-5004-4fb1-bb80-33cd040611f6-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w5-caa3dc-85fc67b867-gsjl2\" (UID: \"28ed1f60-5004-4fb1-bb80-33cd040611f6\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-caa3dc-85fc67b867-gsjl2"
Oct 30 21:36:58.556925 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:36:58.553727    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76de8529_43f7_4285_864b_f8242fca2cae.slice/crio-18ca07d04bd0711cb0eb3e9980c52a59a34f0bcadec7249c07646d4155de1916 WatchSource:0}: Error finding container 18ca07d04bd0711cb0eb3e9980c52a59a34f0bcadec7249c07646d4155de1916: Status 404 returned error can't find the container with id 18ca07d04bd0711cb0eb3e9980c52a59a34f0bcadec7249c07646d4155de1916
Oct 30 21:36:58.556925 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:58.554375    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-0fa864-5d85948857-tw7k6"]
Oct 30 21:36:58.566586 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:58.566560    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x99kb\" (UniqueName: \"kubernetes.io/projected/28ed1f60-5004-4fb1-bb80-33cd040611f6-kube-api-access-x99kb\") pod \"payload-pre-processing-e2e-worker-w5-caa3dc-85fc67b867-gsjl2\" (UID: \"28ed1f60-5004-4fb1-bb80-33cd040611f6\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-caa3dc-85fc67b867-gsjl2"
Oct 30 21:36:58.652985 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:58.652954    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6cgfr\" (UniqueName: \"kubernetes.io/projected/467142c6-5675-4dd8-87a0-14e1f9fddc57-kube-api-access-6cgfr\") pod \"payload-processing-e2e-worker-w5-caa3dc-76db8c686c-pxdzh\" (UID: \"467142c6-5675-4dd8-87a0-14e1f9fddc57\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-caa3dc-76db8c686c-pxdzh"
Oct 30 21:36:58.653413 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:58.653057    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/467142c6-5675-4dd8-87a0-14e1f9fddc57-plugins-config-volume\") pod \"payload-processing-e2e-worker-w5-caa3dc-76db8c686c-pxdzh\" (UID: \"467142c6-5675-4dd8-87a0-14e1f9fddc57\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-caa3dc-76db8c686c-pxdzh"
Oct 30 21:36:58.653789 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:58.653765    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/467142c6-5675-4dd8-87a0-14e1f9fddc57-plugins-config-volume\") pod \"payload-processing-e2e-worker-w5-caa3dc-76db8c686c-pxdzh\" (UID: \"467142c6-5675-4dd8-87a0-14e1f9fddc57\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-caa3dc-76db8c686c-pxdzh"
Oct 30 21:36:58.668567 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:58.668542    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6cgfr\" (UniqueName: \"kubernetes.io/projected/467142c6-5675-4dd8-87a0-14e1f9fddc57-kube-api-access-6cgfr\") pod \"payload-processing-e2e-worker-w5-caa3dc-76db8c686c-pxdzh\" (UID: \"467142c6-5675-4dd8-87a0-14e1f9fddc57\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-caa3dc-76db8c686c-pxdzh"
Oct 30 21:36:58.757140 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:58.757055    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-caa3dc-85fc67b867-gsjl2"
Oct 30 21:36:58.791200 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:58.791163    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w5-caa3dc-76db8c686c-pxdzh"
Oct 30 21:36:58.933908 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:58.933866    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-0fa864-6bc77cf54d-2zd6r"]
Oct 30 21:36:58.941033 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:58.941009    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-0fa864-6bc77cf54d-2zd6r"
Oct 30 21:36:58.954305 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:58.954281    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w4-0fa864-serving-cert\""
Oct 30 21:36:58.958593 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:58.957985    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-0fa864-6bc77cf54d-2zd6r"]
Oct 30 21:36:58.958593 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:58.958136    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b19b4f72-13ba-4a3f-b3a8-73bc3c4c1d4b-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-0fa864-6bc77cf54d-2zd6r\" (UID: \"b19b4f72-13ba-4a3f-b3a8-73bc3c4c1d4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-0fa864-6bc77cf54d-2zd6r"
Oct 30 21:36:58.958593 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:58.958178    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8pqq\" (UniqueName: \"kubernetes.io/projected/b19b4f72-13ba-4a3f-b3a8-73bc3c4c1d4b-kube-api-access-g8pqq\") pod \"maas-api-e2e-worker-w4-0fa864-6bc77cf54d-2zd6r\" (UID: \"b19b4f72-13ba-4a3f-b3a8-73bc3c4c1d4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-0fa864-6bc77cf54d-2zd6r"
Oct 30 21:36:58.958593 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:58.958197    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b19b4f72-13ba-4a3f-b3a8-73bc3c4c1d4b-metrics-certs\") pod \"maas-api-e2e-worker-w4-0fa864-6bc77cf54d-2zd6r\" (UID: \"b19b4f72-13ba-4a3f-b3a8-73bc3c4c1d4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-0fa864-6bc77cf54d-2zd6r"
Oct 30 21:36:58.961714 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:58.959954    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b19b4f72-13ba-4a3f-b3a8-73bc3c4c1d4b-maas-api-tls\") pod \"maas-api-e2e-worker-w4-0fa864-6bc77cf54d-2zd6r\" (UID: \"b19b4f72-13ba-4a3f-b3a8-73bc3c4c1d4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-0fa864-6bc77cf54d-2zd6r"
Oct 30 21:36:59.064979 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:59.063286    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b19b4f72-13ba-4a3f-b3a8-73bc3c4c1d4b-maas-api-tls\") pod \"maas-api-e2e-worker-w4-0fa864-6bc77cf54d-2zd6r\" (UID: \"b19b4f72-13ba-4a3f-b3a8-73bc3c4c1d4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-0fa864-6bc77cf54d-2zd6r"
Oct 30 21:36:59.064979 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:59.063410    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b19b4f72-13ba-4a3f-b3a8-73bc3c4c1d4b-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-0fa864-6bc77cf54d-2zd6r\" (UID: \"b19b4f72-13ba-4a3f-b3a8-73bc3c4c1d4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-0fa864-6bc77cf54d-2zd6r"
Oct 30 21:36:59.064979 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:59.063485    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8pqq\" (UniqueName: \"kubernetes.io/projected/b19b4f72-13ba-4a3f-b3a8-73bc3c4c1d4b-kube-api-access-g8pqq\") pod \"maas-api-e2e-worker-w4-0fa864-6bc77cf54d-2zd6r\" (UID: \"b19b4f72-13ba-4a3f-b3a8-73bc3c4c1d4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-0fa864-6bc77cf54d-2zd6r"
Oct 30 21:36:59.064979 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:59.063518    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b19b4f72-13ba-4a3f-b3a8-73bc3c4c1d4b-metrics-certs\") pod \"maas-api-e2e-worker-w4-0fa864-6bc77cf54d-2zd6r\" (UID: \"b19b4f72-13ba-4a3f-b3a8-73bc3c4c1d4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-0fa864-6bc77cf54d-2zd6r"
Oct 30 21:36:59.064979 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:59.064912    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b19b4f72-13ba-4a3f-b3a8-73bc3c4c1d4b-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-0fa864-6bc77cf54d-2zd6r\" (UID: \"b19b4f72-13ba-4a3f-b3a8-73bc3c4c1d4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-0fa864-6bc77cf54d-2zd6r"
Oct 30 21:36:59.073689 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:59.073655    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b19b4f72-13ba-4a3f-b3a8-73bc3c4c1d4b-metrics-certs\") pod \"maas-api-e2e-worker-w4-0fa864-6bc77cf54d-2zd6r\" (UID: \"b19b4f72-13ba-4a3f-b3a8-73bc3c4c1d4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-0fa864-6bc77cf54d-2zd6r"
Oct 30 21:36:59.090496 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:59.090462    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-caa3dc-85fc67b867-gsjl2"]
Oct 30 21:36:59.097560 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:59.097506    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b19b4f72-13ba-4a3f-b3a8-73bc3c4c1d4b-maas-api-tls\") pod \"maas-api-e2e-worker-w4-0fa864-6bc77cf54d-2zd6r\" (UID: \"b19b4f72-13ba-4a3f-b3a8-73bc3c4c1d4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-0fa864-6bc77cf54d-2zd6r"
Oct 30 21:36:59.106040 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:36:59.106001    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28ed1f60_5004_4fb1_bb80_33cd040611f6.slice/crio-e9cd86444d42cda2a7f2113aaf4b869272b0f295d04de2799d245b1e993afdff WatchSource:0}: Error finding container e9cd86444d42cda2a7f2113aaf4b869272b0f295d04de2799d245b1e993afdff: Status 404 returned error can't find the container with id e9cd86444d42cda2a7f2113aaf4b869272b0f295d04de2799d245b1e993afdff
Oct 30 21:36:59.108526 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:59.108499    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8pqq\" (UniqueName: \"kubernetes.io/projected/b19b4f72-13ba-4a3f-b3a8-73bc3c4c1d4b-kube-api-access-g8pqq\") pod \"maas-api-e2e-worker-w4-0fa864-6bc77cf54d-2zd6r\" (UID: \"b19b4f72-13ba-4a3f-b3a8-73bc3c4c1d4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-0fa864-6bc77cf54d-2zd6r"
Oct 30 21:36:59.130546 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:59.130514    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-caa3dc-76db8c686c-pxdzh"]
Oct 30 21:36:59.134480 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:36:59.134443    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod467142c6_5675_4dd8_87a0_14e1f9fddc57.slice/crio-1e90032cb941b3d3473723c4c817b8d4018ea09786354f5a11861794cc9f14aa WatchSource:0}: Error finding container 1e90032cb941b3d3473723c4c817b8d4018ea09786354f5a11861794cc9f14aa: Status 404 returned error can't find the container with id 1e90032cb941b3d3473723c4c817b8d4018ea09786354f5a11861794cc9f14aa
Oct 30 21:36:59.248561 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:59.248523    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w5-caa3dc-76db8c686c-pxdzh" event={"ID":"467142c6-5675-4dd8-87a0-14e1f9fddc57","Type":"ContainerStarted","Data":"1e90032cb941b3d3473723c4c817b8d4018ea09786354f5a11861794cc9f14aa"}
Oct 30 21:36:59.250453 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:59.250423    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-0fa864-5d85948857-tw7k6" event={"ID":"76de8529-43f7-4285-864b-f8242fca2cae","Type":"ContainerStarted","Data":"18ca07d04bd0711cb0eb3e9980c52a59a34f0bcadec7249c07646d4155de1916"}
Oct 30 21:36:59.251974 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:59.251942    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-caa3dc-85fc67b867-gsjl2" event={"ID":"28ed1f60-5004-4fb1-bb80-33cd040611f6","Type":"ContainerStarted","Data":"e9cd86444d42cda2a7f2113aaf4b869272b0f295d04de2799d245b1e993afdff"}
Oct 30 21:36:59.268689 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:59.268658    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-0fa864-6bc77cf54d-2zd6r"
Oct 30 21:36:59.588144 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:59.588104    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-0fa864-6bc77cf54d-2zd6r"]
Oct 30 21:36:59.594880 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:36:59.594756    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb19b4f72_13ba_4a3f_b3a8_73bc3c4c1d4b.slice/crio-d1b85680cc7da8155f8374da38786c2084dd27dfc30aa088f0a372ef545ea31f WatchSource:0}: Error finding container d1b85680cc7da8155f8374da38786c2084dd27dfc30aa088f0a372ef545ea31f: Status 404 returned error can't find the container with id d1b85680cc7da8155f8374da38786c2084dd27dfc30aa088f0a372ef545ea31f
Oct 30 21:36:59.652119 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:59.652068    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-1d8034-5b96976fc5-hsr78"]
Oct 30 21:36:59.658072 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:59.658016    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-1d8034-5b96976fc5-hsr78"
Oct 30 21:36:59.671580 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:59.671550    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w3-1d8034-dockercfg-jx74d\""
Oct 30 21:36:59.672119 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:59.672095    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/27aef427-6b1e-4845-b8d9-a83cba81b0e0-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-1d8034-5b96976fc5-hsr78\" (UID: \"27aef427-6b1e-4845-b8d9-a83cba81b0e0\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-1d8034-5b96976fc5-hsr78"
Oct 30 21:36:59.672247 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:59.672216    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nggn9\" (UniqueName: \"kubernetes.io/projected/27aef427-6b1e-4845-b8d9-a83cba81b0e0-kube-api-access-nggn9\") pod \"payload-pre-processing-e2e-worker-w3-1d8034-5b96976fc5-hsr78\" (UID: \"27aef427-6b1e-4845-b8d9-a83cba81b0e0\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-1d8034-5b96976fc5-hsr78"
Oct 30 21:36:59.687415 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:59.674096    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-1d8034-5b96976fc5-hsr78"]
Oct 30 21:36:59.729888 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:59.729850    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-1d8034-6bbd97dcf7-l2gd8"]
Oct 30 21:36:59.736685 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:59.736659    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-1d8034-6bbd97dcf7-l2gd8"
Oct 30 21:36:59.737516 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:59.737488    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w3-1d8034\""
Oct 30 21:36:59.759660 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:59.759629    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-1d8034-6bbd97dcf7-l2gd8"]
Oct 30 21:36:59.773070 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:59.773036    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/27aef427-6b1e-4845-b8d9-a83cba81b0e0-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-1d8034-5b96976fc5-hsr78\" (UID: \"27aef427-6b1e-4845-b8d9-a83cba81b0e0\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-1d8034-5b96976fc5-hsr78"
Oct 30 21:36:59.773245 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:59.773153    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6d8p\" (UniqueName: \"kubernetes.io/projected/e25d6f68-6e47-449f-b02f-03fff2b2bb30-kube-api-access-j6d8p\") pod \"payload-processing-e2e-worker-w3-1d8034-6bbd97dcf7-l2gd8\" (UID: \"e25d6f68-6e47-449f-b02f-03fff2b2bb30\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-1d8034-6bbd97dcf7-l2gd8"
Oct 30 21:36:59.773245 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:59.773203    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nggn9\" (UniqueName: \"kubernetes.io/projected/27aef427-6b1e-4845-b8d9-a83cba81b0e0-kube-api-access-nggn9\") pod \"payload-pre-processing-e2e-worker-w3-1d8034-5b96976fc5-hsr78\" (UID: \"27aef427-6b1e-4845-b8d9-a83cba81b0e0\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-1d8034-5b96976fc5-hsr78"
Oct 30 21:36:59.773411 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:59.773313    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e25d6f68-6e47-449f-b02f-03fff2b2bb30-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-1d8034-6bbd97dcf7-l2gd8\" (UID: \"e25d6f68-6e47-449f-b02f-03fff2b2bb30\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-1d8034-6bbd97dcf7-l2gd8"
Oct 30 21:36:59.774251 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:59.774224    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/27aef427-6b1e-4845-b8d9-a83cba81b0e0-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-1d8034-5b96976fc5-hsr78\" (UID: \"27aef427-6b1e-4845-b8d9-a83cba81b0e0\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-1d8034-5b96976fc5-hsr78"
Oct 30 21:36:59.824635 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:59.824561    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nggn9\" (UniqueName: \"kubernetes.io/projected/27aef427-6b1e-4845-b8d9-a83cba81b0e0-kube-api-access-nggn9\") pod \"payload-pre-processing-e2e-worker-w3-1d8034-5b96976fc5-hsr78\" (UID: \"27aef427-6b1e-4845-b8d9-a83cba81b0e0\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-1d8034-5b96976fc5-hsr78"
Oct 30 21:36:59.875281 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:59.874020    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6d8p\" (UniqueName: \"kubernetes.io/projected/e25d6f68-6e47-449f-b02f-03fff2b2bb30-kube-api-access-j6d8p\") pod \"payload-processing-e2e-worker-w3-1d8034-6bbd97dcf7-l2gd8\" (UID: \"e25d6f68-6e47-449f-b02f-03fff2b2bb30\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-1d8034-6bbd97dcf7-l2gd8"
Oct 30 21:36:59.875281 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:59.874178    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e25d6f68-6e47-449f-b02f-03fff2b2bb30-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-1d8034-6bbd97dcf7-l2gd8\" (UID: \"e25d6f68-6e47-449f-b02f-03fff2b2bb30\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-1d8034-6bbd97dcf7-l2gd8"
Oct 30 21:36:59.875281 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:59.875180    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e25d6f68-6e47-449f-b02f-03fff2b2bb30-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-1d8034-6bbd97dcf7-l2gd8\" (UID: \"e25d6f68-6e47-449f-b02f-03fff2b2bb30\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-1d8034-6bbd97dcf7-l2gd8"
Oct 30 21:36:59.890339 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:59.890303    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6d8p\" (UniqueName: \"kubernetes.io/projected/e25d6f68-6e47-449f-b02f-03fff2b2bb30-kube-api-access-j6d8p\") pod \"payload-processing-e2e-worker-w3-1d8034-6bbd97dcf7-l2gd8\" (UID: \"e25d6f68-6e47-449f-b02f-03fff2b2bb30\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-1d8034-6bbd97dcf7-l2gd8"
Oct 30 21:36:59.978557 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:36:59.978520    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-1d8034-5b96976fc5-hsr78"
Oct 30 21:37:00.131945 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:00.130143    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-1d8034-6bbd97dcf7-l2gd8"
Oct 30 21:37:00.272162 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:00.271273    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-0fa864-6bc77cf54d-2zd6r" event={"ID":"b19b4f72-13ba-4a3f-b3a8-73bc3c4c1d4b","Type":"ContainerStarted","Data":"dee681b536a7a792f6d1ed3d190ec48d01d610ee18b3f6b64155a9fbf6e78368"}
Oct 30 21:37:00.272162 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:00.271372    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-0fa864-6bc77cf54d-2zd6r" event={"ID":"b19b4f72-13ba-4a3f-b3a8-73bc3c4c1d4b","Type":"ContainerStarted","Data":"d1b85680cc7da8155f8374da38786c2084dd27dfc30aa088f0a372ef545ea31f"}
Oct 30 21:37:00.272162 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:00.272112    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-0fa864-6bc77cf54d-2zd6r"
Oct 30 21:37:00.305519 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:00.305448    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-0fa864-6bc77cf54d-2zd6r" podStartSLOduration=1.845848201 podStartE2EDuration="2.3054277s" totalImagesPullingTime="459.579499ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:36:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:37:00.301636517 +0000 UTC m=+533.103448978" watchObservedRunningTime="2026-10-30 21:37:00.3054277 +0000 UTC m=+533.107240181"
Oct 30 21:37:00.310796 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:00.310491    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-1d8034-5b96976fc5-hsr78"]
Oct 30 21:37:00.315320 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:37:00.315286    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27aef427_6b1e_4845_b8d9_a83cba81b0e0.slice/crio-5d69c766bffc128b32ea44997ca5922eff6540283897651764612cec07cb9a85 WatchSource:0}: Error finding container 5d69c766bffc128b32ea44997ca5922eff6540283897651764612cec07cb9a85: Status 404 returned error can't find the container with id 5d69c766bffc128b32ea44997ca5922eff6540283897651764612cec07cb9a85
Oct 30 21:37:00.388449 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:00.387403    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-000f64-765dc8c9bf-lxnrx"]
Oct 30 21:37:00.397882 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:00.396968    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-000f64-765dc8c9bf-lxnrx"
Oct 30 21:37:00.414011 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:00.413978    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-000f64-765dc8c9bf-lxnrx"]
Oct 30 21:37:00.414173 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:00.414125    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-000f64-serving-cert\""
Oct 30 21:37:00.479057 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:00.479024    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-1d8034-6bbd97dcf7-l2gd8"]
Oct 30 21:37:00.491296 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:37:00.491214    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode25d6f68_6e47_449f_b02f_03fff2b2bb30.slice/crio-c18a102100a9a20e66a0e5cb794b547fb308896b667e989e0c6c31da400f58fb WatchSource:0}: Error finding container c18a102100a9a20e66a0e5cb794b547fb308896b667e989e0c6c31da400f58fb: Status 404 returned error can't find the container with id c18a102100a9a20e66a0e5cb794b547fb308896b667e989e0c6c31da400f58fb
Oct 30 21:37:00.592290 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:00.592249    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvrpl\" (UniqueName: \"kubernetes.io/projected/96a36ed7-d92e-4748-96b8-42367bcb1b48-kube-api-access-zvrpl\") pod \"maas-api-e2e-worker-w0-000f64-765dc8c9bf-lxnrx\" (UID: \"96a36ed7-d92e-4748-96b8-42367bcb1b48\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-000f64-765dc8c9bf-lxnrx"
Oct 30 21:37:00.592503 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:00.592391    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96a36ed7-d92e-4748-96b8-42367bcb1b48-maas-api-tls\") pod \"maas-api-e2e-worker-w0-000f64-765dc8c9bf-lxnrx\" (UID: \"96a36ed7-d92e-4748-96b8-42367bcb1b48\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-000f64-765dc8c9bf-lxnrx"
Oct 30 21:37:00.592581 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:00.592512    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/96a36ed7-d92e-4748-96b8-42367bcb1b48-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-000f64-765dc8c9bf-lxnrx\" (UID: \"96a36ed7-d92e-4748-96b8-42367bcb1b48\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-000f64-765dc8c9bf-lxnrx"
Oct 30 21:37:00.592581 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:00.592559    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96a36ed7-d92e-4748-96b8-42367bcb1b48-metrics-certs\") pod \"maas-api-e2e-worker-w0-000f64-765dc8c9bf-lxnrx\" (UID: \"96a36ed7-d92e-4748-96b8-42367bcb1b48\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-000f64-765dc8c9bf-lxnrx"
Oct 30 21:37:00.694211 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:00.694109    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/96a36ed7-d92e-4748-96b8-42367bcb1b48-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-000f64-765dc8c9bf-lxnrx\" (UID: \"96a36ed7-d92e-4748-96b8-42367bcb1b48\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-000f64-765dc8c9bf-lxnrx"
Oct 30 21:37:00.694211 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:00.694169    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96a36ed7-d92e-4748-96b8-42367bcb1b48-metrics-certs\") pod \"maas-api-e2e-worker-w0-000f64-765dc8c9bf-lxnrx\" (UID: \"96a36ed7-d92e-4748-96b8-42367bcb1b48\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-000f64-765dc8c9bf-lxnrx"
Oct 30 21:37:00.694863 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:00.694251    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvrpl\" (UniqueName: \"kubernetes.io/projected/96a36ed7-d92e-4748-96b8-42367bcb1b48-kube-api-access-zvrpl\") pod \"maas-api-e2e-worker-w0-000f64-765dc8c9bf-lxnrx\" (UID: \"96a36ed7-d92e-4748-96b8-42367bcb1b48\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-000f64-765dc8c9bf-lxnrx"
Oct 30 21:37:00.694863 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:00.694303    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96a36ed7-d92e-4748-96b8-42367bcb1b48-maas-api-tls\") pod \"maas-api-e2e-worker-w0-000f64-765dc8c9bf-lxnrx\" (UID: \"96a36ed7-d92e-4748-96b8-42367bcb1b48\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-000f64-765dc8c9bf-lxnrx"
Oct 30 21:37:00.695008 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:00.694983    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/96a36ed7-d92e-4748-96b8-42367bcb1b48-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-000f64-765dc8c9bf-lxnrx\" (UID: \"96a36ed7-d92e-4748-96b8-42367bcb1b48\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-000f64-765dc8c9bf-lxnrx"
Oct 30 21:37:00.698353 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:00.698323    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96a36ed7-d92e-4748-96b8-42367bcb1b48-metrics-certs\") pod \"maas-api-e2e-worker-w0-000f64-765dc8c9bf-lxnrx\" (UID: \"96a36ed7-d92e-4748-96b8-42367bcb1b48\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-000f64-765dc8c9bf-lxnrx"
Oct 30 21:37:00.708407 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:00.708375    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96a36ed7-d92e-4748-96b8-42367bcb1b48-maas-api-tls\") pod \"maas-api-e2e-worker-w0-000f64-765dc8c9bf-lxnrx\" (UID: \"96a36ed7-d92e-4748-96b8-42367bcb1b48\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-000f64-765dc8c9bf-lxnrx"
Oct 30 21:37:00.711614 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:00.711585    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvrpl\" (UniqueName: \"kubernetes.io/projected/96a36ed7-d92e-4748-96b8-42367bcb1b48-kube-api-access-zvrpl\") pod \"maas-api-e2e-worker-w0-000f64-765dc8c9bf-lxnrx\" (UID: \"96a36ed7-d92e-4748-96b8-42367bcb1b48\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-000f64-765dc8c9bf-lxnrx"
Oct 30 21:37:00.718320 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:00.718285    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-000f64-765dc8c9bf-lxnrx"
Oct 30 21:37:00.878401 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:00.878351    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-000f64-85b9f75579-vc4bc"]
Oct 30 21:37:00.882998 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:00.882971    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-000f64-85b9f75579-vc4bc"
Oct 30 21:37:00.895968 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:00.895883    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-000f64\""
Oct 30 21:37:00.901865 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:00.901802    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d3bbb2d6-2cff-40ad-90fa-7e232d26a860-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-000f64-85b9f75579-vc4bc\" (UID: \"d3bbb2d6-2cff-40ad-90fa-7e232d26a860\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-000f64-85b9f75579-vc4bc"
Oct 30 21:37:00.902093 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:00.901911    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45np9\" (UniqueName: \"kubernetes.io/projected/d3bbb2d6-2cff-40ad-90fa-7e232d26a860-kube-api-access-45np9\") pod \"payload-pre-processing-e2e-worker-w0-000f64-85b9f75579-vc4bc\" (UID: \"d3bbb2d6-2cff-40ad-90fa-7e232d26a860\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-000f64-85b9f75579-vc4bc"
Oct 30 21:37:00.902218 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:00.902100    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-000f64-85b9f75579-vc4bc"]
Oct 30 21:37:00.948879 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:00.948804    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-000f64-6576d48f75-5rdtz"]
Oct 30 21:37:00.955479 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:00.955450    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-000f64-6576d48f75-5rdtz"
Oct 30 21:37:00.962744 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:00.962715    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-000f64-6576d48f75-5rdtz"]
Oct 30 21:37:00.969246 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:00.968991    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-000f64-dockercfg-mgks8\""
Oct 30 21:37:01.004136 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:01.004096    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45np9\" (UniqueName: \"kubernetes.io/projected/d3bbb2d6-2cff-40ad-90fa-7e232d26a860-kube-api-access-45np9\") pod \"payload-pre-processing-e2e-worker-w0-000f64-85b9f75579-vc4bc\" (UID: \"d3bbb2d6-2cff-40ad-90fa-7e232d26a860\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-000f64-85b9f75579-vc4bc"
Oct 30 21:37:01.004323 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:01.004249    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d3bbb2d6-2cff-40ad-90fa-7e232d26a860-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-000f64-85b9f75579-vc4bc\" (UID: \"d3bbb2d6-2cff-40ad-90fa-7e232d26a860\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-000f64-85b9f75579-vc4bc"
Oct 30 21:37:01.008292 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:01.008223    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d3bbb2d6-2cff-40ad-90fa-7e232d26a860-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-000f64-85b9f75579-vc4bc\" (UID: \"d3bbb2d6-2cff-40ad-90fa-7e232d26a860\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-000f64-85b9f75579-vc4bc"
Oct 30 21:37:01.068596 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:01.068564    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-45np9\" (UniqueName: \"kubernetes.io/projected/d3bbb2d6-2cff-40ad-90fa-7e232d26a860-kube-api-access-45np9\") pod \"payload-pre-processing-e2e-worker-w0-000f64-85b9f75579-vc4bc\" (UID: \"d3bbb2d6-2cff-40ad-90fa-7e232d26a860\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-000f64-85b9f75579-vc4bc"
Oct 30 21:37:01.086400 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:01.086354    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-000f64-765dc8c9bf-lxnrx"]
Oct 30 21:37:01.107353 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:01.106713    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c9e3ad2b-93c6-46fd-8b0c-87e24852081e-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-000f64-6576d48f75-5rdtz\" (UID: \"c9e3ad2b-93c6-46fd-8b0c-87e24852081e\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-000f64-6576d48f75-5rdtz"
Oct 30 21:37:01.107353 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:01.106779    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szmvv\" (UniqueName: \"kubernetes.io/projected/c9e3ad2b-93c6-46fd-8b0c-87e24852081e-kube-api-access-szmvv\") pod \"payload-processing-e2e-worker-w0-000f64-6576d48f75-5rdtz\" (UID: \"c9e3ad2b-93c6-46fd-8b0c-87e24852081e\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-000f64-6576d48f75-5rdtz"
Oct 30 21:37:01.206910 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:01.206605    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-000f64-85b9f75579-vc4bc"
Oct 30 21:37:01.209124 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:01.207673    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c9e3ad2b-93c6-46fd-8b0c-87e24852081e-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-000f64-6576d48f75-5rdtz\" (UID: \"c9e3ad2b-93c6-46fd-8b0c-87e24852081e\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-000f64-6576d48f75-5rdtz"
Oct 30 21:37:01.209124 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:01.207745    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szmvv\" (UniqueName: \"kubernetes.io/projected/c9e3ad2b-93c6-46fd-8b0c-87e24852081e-kube-api-access-szmvv\") pod \"payload-processing-e2e-worker-w0-000f64-6576d48f75-5rdtz\" (UID: \"c9e3ad2b-93c6-46fd-8b0c-87e24852081e\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-000f64-6576d48f75-5rdtz"
Oct 30 21:37:01.209124 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:01.209047    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c9e3ad2b-93c6-46fd-8b0c-87e24852081e-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-000f64-6576d48f75-5rdtz\" (UID: \"c9e3ad2b-93c6-46fd-8b0c-87e24852081e\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-000f64-6576d48f75-5rdtz"
Oct 30 21:37:01.245901 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:01.245867    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-szmvv\" (UniqueName: \"kubernetes.io/projected/c9e3ad2b-93c6-46fd-8b0c-87e24852081e-kube-api-access-szmvv\") pod \"payload-processing-e2e-worker-w0-000f64-6576d48f75-5rdtz\" (UID: \"c9e3ad2b-93c6-46fd-8b0c-87e24852081e\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-000f64-6576d48f75-5rdtz"
Oct 30 21:37:01.278599 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:01.278564    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-000f64-6576d48f75-5rdtz"
Oct 30 21:37:01.289188 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:01.289113    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-1d8034-6bbd97dcf7-l2gd8" event={"ID":"e25d6f68-6e47-449f-b02f-03fff2b2bb30","Type":"ContainerStarted","Data":"c18a102100a9a20e66a0e5cb794b547fb308896b667e989e0c6c31da400f58fb"}
Oct 30 21:37:01.295209 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:01.295120    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-1d8034-5b96976fc5-hsr78" event={"ID":"27aef427-6b1e-4845-b8d9-a83cba81b0e0","Type":"ContainerStarted","Data":"5d69c766bffc128b32ea44997ca5922eff6540283897651764612cec07cb9a85"}
Oct 30 21:37:01.298341 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:01.298287    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-000f64-765dc8c9bf-lxnrx" event={"ID":"96a36ed7-d92e-4748-96b8-42367bcb1b48","Type":"ContainerStarted","Data":"3ece370df703018acd4c8f42e46c94c7122e197adece2b2d5043a1f02d8646d6"}
Oct 30 21:37:01.334698 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:01.334663    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-7dfff0-579b4f5ddc-kxqlf"]
Oct 30 21:37:01.342992 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:01.341890    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-7dfff0-579b4f5ddc-kxqlf"
Oct 30 21:37:01.382311 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:01.381384    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w6-7dfff0\""
Oct 30 21:37:01.391669 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:01.391395    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-7dfff0-579b4f5ddc-kxqlf"]
Oct 30 21:37:01.429517 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:01.429487    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w6-7dfff0-dockercfg-fxbnc\""
Oct 30 21:37:01.512271 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:01.511963    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b53e75a7-c215-4fc9-a64b-8720382c6acd-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w6-7dfff0-579b4f5ddc-kxqlf\" (UID: \"b53e75a7-c215-4fc9-a64b-8720382c6acd\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-7dfff0-579b4f5ddc-kxqlf"
Oct 30 21:37:01.512271 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:01.512052    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jn22k\" (UniqueName: \"kubernetes.io/projected/b53e75a7-c215-4fc9-a64b-8720382c6acd-kube-api-access-jn22k\") pod \"payload-pre-processing-e2e-worker-w6-7dfff0-579b4f5ddc-kxqlf\" (UID: \"b53e75a7-c215-4fc9-a64b-8720382c6acd\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-7dfff0-579b4f5ddc-kxqlf"
Oct 30 21:37:01.589223 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:01.588290    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-000f64-85b9f75579-vc4bc"]
Oct 30 21:37:01.614247 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:01.614212    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b53e75a7-c215-4fc9-a64b-8720382c6acd-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w6-7dfff0-579b4f5ddc-kxqlf\" (UID: \"b53e75a7-c215-4fc9-a64b-8720382c6acd\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-7dfff0-579b4f5ddc-kxqlf"
Oct 30 21:37:01.614427 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:01.614383    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jn22k\" (UniqueName: \"kubernetes.io/projected/b53e75a7-c215-4fc9-a64b-8720382c6acd-kube-api-access-jn22k\") pod \"payload-pre-processing-e2e-worker-w6-7dfff0-579b4f5ddc-kxqlf\" (UID: \"b53e75a7-c215-4fc9-a64b-8720382c6acd\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-7dfff0-579b4f5ddc-kxqlf"
Oct 30 21:37:01.617759 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:01.617696    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b53e75a7-c215-4fc9-a64b-8720382c6acd-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w6-7dfff0-579b4f5ddc-kxqlf\" (UID: \"b53e75a7-c215-4fc9-a64b-8720382c6acd\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-7dfff0-579b4f5ddc-kxqlf"
Oct 30 21:37:01.630204 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:01.630118    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-000f64-6576d48f75-5rdtz"]
Oct 30 21:37:01.632657 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:37:01.632307    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9e3ad2b_93c6_46fd_8b0c_87e24852081e.slice/crio-4747ecc63ff2222acfb576b0c4a2972692df92bac39f96997695717b015256b1 WatchSource:0}: Error finding container 4747ecc63ff2222acfb576b0c4a2972692df92bac39f96997695717b015256b1: Status 404 returned error can't find the container with id 4747ecc63ff2222acfb576b0c4a2972692df92bac39f96997695717b015256b1
Oct 30 21:37:01.634160 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:01.634031    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jn22k\" (UniqueName: \"kubernetes.io/projected/b53e75a7-c215-4fc9-a64b-8720382c6acd-kube-api-access-jn22k\") pod \"payload-pre-processing-e2e-worker-w6-7dfff0-579b4f5ddc-kxqlf\" (UID: \"b53e75a7-c215-4fc9-a64b-8720382c6acd\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-7dfff0-579b4f5ddc-kxqlf"
Oct 30 21:37:01.734800 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:01.734559    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-7dfff0-579b4f5ddc-kxqlf"
Oct 30 21:37:01.962117 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:01.962082    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 21:37:02.096235 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:02.096149    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-7dfff0-579b4f5ddc-kxqlf"]
Oct 30 21:37:02.316512 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:02.315941    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-000f64-765dc8c9bf-lxnrx" event={"ID":"96a36ed7-d92e-4748-96b8-42367bcb1b48","Type":"ContainerStarted","Data":"a8ba673e641b1a3a5d0e747639761d8b75fe535f9dd360aedb2bd10a6fe05297"}
Oct 30 21:37:02.316512 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:02.316446    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-000f64-765dc8c9bf-lxnrx"
Oct 30 21:37:02.320284 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:02.320210    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-000f64-6576d48f75-5rdtz" event={"ID":"c9e3ad2b-93c6-46fd-8b0c-87e24852081e","Type":"ContainerStarted","Data":"4747ecc63ff2222acfb576b0c4a2972692df92bac39f96997695717b015256b1"}
Oct 30 21:37:02.323800 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:02.323721    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-7dfff0-579b4f5ddc-kxqlf" event={"ID":"b53e75a7-c215-4fc9-a64b-8720382c6acd","Type":"ContainerStarted","Data":"f88e9c0ec5587071b365e2604ad975131a1bf31b384497d2cad3d97840a09520"}
Oct 30 21:37:02.326873 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:02.326805    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-000f64-85b9f75579-vc4bc" event={"ID":"d3bbb2d6-2cff-40ad-90fa-7e232d26a860","Type":"ContainerStarted","Data":"88c9f14c35b9515a951c8525c3c0c4f05ec7e0a549c6b67c1cc62f85853c463f"}
Oct 30 21:37:02.353763 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:02.352269    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-000f64-765dc8c9bf-lxnrx" podStartSLOduration=1.863692296 podStartE2EDuration="2.352248415s" totalImagesPullingTime="488.556119ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:37:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:37:02.348645083 +0000 UTC m=+535.150457551" watchObservedRunningTime="2026-10-30 21:37:02.352248415 +0000 UTC m=+535.154060860"
Oct 30 21:37:03.014095 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:03.014055    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-db0b54-758d689765-f75zr"]
Oct 30 21:37:03.023703 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:03.023627    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-db0b54-758d689765-f75zr"
Oct 30 21:37:03.034277 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:03.034243    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-db0b54-758d689765-f75zr"]
Oct 30 21:37:03.037236 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:03.037211    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w2-db0b54-dockercfg-p4k8x\""
Oct 30 21:37:03.045035 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:03.044571    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w2-db0b54\""
Oct 30 21:37:03.091138 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:03.091044    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00d3249f-1824-4ba1-8c18-6f6ab1f7ff92-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-db0b54-758d689765-f75zr\" (UID: \"00d3249f-1824-4ba1-8c18-6f6ab1f7ff92\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-db0b54-758d689765-f75zr"
Oct 30 21:37:03.091138 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:03.091115    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzfrv\" (UniqueName: \"kubernetes.io/projected/00d3249f-1824-4ba1-8c18-6f6ab1f7ff92-kube-api-access-jzfrv\") pod \"payload-processing-e2e-worker-w2-db0b54-758d689765-f75zr\" (UID: \"00d3249f-1824-4ba1-8c18-6f6ab1f7ff92\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-db0b54-758d689765-f75zr"
Oct 30 21:37:03.193717 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:03.192963    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00d3249f-1824-4ba1-8c18-6f6ab1f7ff92-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-db0b54-758d689765-f75zr\" (UID: \"00d3249f-1824-4ba1-8c18-6f6ab1f7ff92\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-db0b54-758d689765-f75zr"
Oct 30 21:37:03.193717 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:03.193068    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzfrv\" (UniqueName: \"kubernetes.io/projected/00d3249f-1824-4ba1-8c18-6f6ab1f7ff92-kube-api-access-jzfrv\") pod \"payload-processing-e2e-worker-w2-db0b54-758d689765-f75zr\" (UID: \"00d3249f-1824-4ba1-8c18-6f6ab1f7ff92\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-db0b54-758d689765-f75zr"
Oct 30 21:37:03.198881 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:03.194254    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00d3249f-1824-4ba1-8c18-6f6ab1f7ff92-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-db0b54-758d689765-f75zr\" (UID: \"00d3249f-1824-4ba1-8c18-6f6ab1f7ff92\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-db0b54-758d689765-f75zr"
Oct 30 21:37:03.212087 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:03.212030    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzfrv\" (UniqueName: \"kubernetes.io/projected/00d3249f-1824-4ba1-8c18-6f6ab1f7ff92-kube-api-access-jzfrv\") pod \"payload-processing-e2e-worker-w2-db0b54-758d689765-f75zr\" (UID: \"00d3249f-1824-4ba1-8c18-6f6ab1f7ff92\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-db0b54-758d689765-f75zr"
Oct 30 21:37:03.355855 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:03.355781    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-db0b54-758d689765-f75zr"
Oct 30 21:37:03.727526 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:03.726922    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-db0b54-758d689765-f75zr"]
Oct 30 21:37:03.749316 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:37:03.749089    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00d3249f_1824_4ba1_8c18_6f6ab1f7ff92.slice/crio-f8c381b8d7f4b62de6bf349613b8db4c67903d9296900cacf420028e2c68ac53 WatchSource:0}: Error finding container f8c381b8d7f4b62de6bf349613b8db4c67903d9296900cacf420028e2c68ac53: Status 404 returned error can't find the container with id f8c381b8d7f4b62de6bf349613b8db4c67903d9296900cacf420028e2c68ac53
Oct 30 21:37:03.818251 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:03.818213    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-7dfff0-6ccb87d846-7ml8d"]
Oct 30 21:37:03.826710 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:03.826683    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-7dfff0-6ccb87d846-7ml8d"
Oct 30 21:37:03.836172 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:03.836063    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w6-7dfff0-serving-cert\""
Oct 30 21:37:03.849031 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:03.848993    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-7dfff0-6ccb87d846-7ml8d"]
Oct 30 21:37:04.013113 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:04.013024    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5f6a7d6-010d-4bb5-a333-6f299a5022b5-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-7dfff0-6ccb87d846-7ml8d\" (UID: \"b5f6a7d6-010d-4bb5-a333-6f299a5022b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-7dfff0-6ccb87d846-7ml8d"
Oct 30 21:37:04.013286 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:04.013148    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ch94b\" (UniqueName: \"kubernetes.io/projected/b5f6a7d6-010d-4bb5-a333-6f299a5022b5-kube-api-access-ch94b\") pod \"maas-api-e2e-worker-w6-7dfff0-6ccb87d846-7ml8d\" (UID: \"b5f6a7d6-010d-4bb5-a333-6f299a5022b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-7dfff0-6ccb87d846-7ml8d"
Oct 30 21:37:04.013396 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:04.013338    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b5f6a7d6-010d-4bb5-a333-6f299a5022b5-metrics-certs\") pod \"maas-api-e2e-worker-w6-7dfff0-6ccb87d846-7ml8d\" (UID: \"b5f6a7d6-010d-4bb5-a333-6f299a5022b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-7dfff0-6ccb87d846-7ml8d"
Oct 30 21:37:04.013596 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:04.013574    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5f6a7d6-010d-4bb5-a333-6f299a5022b5-maas-api-tls\") pod \"maas-api-e2e-worker-w6-7dfff0-6ccb87d846-7ml8d\" (UID: \"b5f6a7d6-010d-4bb5-a333-6f299a5022b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-7dfff0-6ccb87d846-7ml8d"
Oct 30 21:37:04.116421 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:04.115652    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b5f6a7d6-010d-4bb5-a333-6f299a5022b5-metrics-certs\") pod \"maas-api-e2e-worker-w6-7dfff0-6ccb87d846-7ml8d\" (UID: \"b5f6a7d6-010d-4bb5-a333-6f299a5022b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-7dfff0-6ccb87d846-7ml8d"
Oct 30 21:37:04.116421 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:04.115792    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5f6a7d6-010d-4bb5-a333-6f299a5022b5-maas-api-tls\") pod \"maas-api-e2e-worker-w6-7dfff0-6ccb87d846-7ml8d\" (UID: \"b5f6a7d6-010d-4bb5-a333-6f299a5022b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-7dfff0-6ccb87d846-7ml8d"
Oct 30 21:37:04.116421 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:04.115853    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5f6a7d6-010d-4bb5-a333-6f299a5022b5-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-7dfff0-6ccb87d846-7ml8d\" (UID: \"b5f6a7d6-010d-4bb5-a333-6f299a5022b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-7dfff0-6ccb87d846-7ml8d"
Oct 30 21:37:04.116421 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:04.115879    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ch94b\" (UniqueName: \"kubernetes.io/projected/b5f6a7d6-010d-4bb5-a333-6f299a5022b5-kube-api-access-ch94b\") pod \"maas-api-e2e-worker-w6-7dfff0-6ccb87d846-7ml8d\" (UID: \"b5f6a7d6-010d-4bb5-a333-6f299a5022b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-7dfff0-6ccb87d846-7ml8d"
Oct 30 21:37:04.117686 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:04.117551    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5f6a7d6-010d-4bb5-a333-6f299a5022b5-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-7dfff0-6ccb87d846-7ml8d\" (UID: \"b5f6a7d6-010d-4bb5-a333-6f299a5022b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-7dfff0-6ccb87d846-7ml8d"
Oct 30 21:37:04.124761 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:04.124706    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b5f6a7d6-010d-4bb5-a333-6f299a5022b5-metrics-certs\") pod \"maas-api-e2e-worker-w6-7dfff0-6ccb87d846-7ml8d\" (UID: \"b5f6a7d6-010d-4bb5-a333-6f299a5022b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-7dfff0-6ccb87d846-7ml8d"
Oct 30 21:37:04.126460 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:04.126437    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5f6a7d6-010d-4bb5-a333-6f299a5022b5-maas-api-tls\") pod \"maas-api-e2e-worker-w6-7dfff0-6ccb87d846-7ml8d\" (UID: \"b5f6a7d6-010d-4bb5-a333-6f299a5022b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-7dfff0-6ccb87d846-7ml8d"
Oct 30 21:37:04.149090 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:04.149064    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ch94b\" (UniqueName: \"kubernetes.io/projected/b5f6a7d6-010d-4bb5-a333-6f299a5022b5-kube-api-access-ch94b\") pod \"maas-api-e2e-worker-w6-7dfff0-6ccb87d846-7ml8d\" (UID: \"b5f6a7d6-010d-4bb5-a333-6f299a5022b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-7dfff0-6ccb87d846-7ml8d"
Oct 30 21:37:04.156613 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:04.155906    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-7dfff0-6ccb87d846-7ml8d"
Oct 30 21:37:04.358050 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:04.357622    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-db0b54-758d689765-f75zr" event={"ID":"00d3249f-1824-4ba1-8c18-6f6ab1f7ff92","Type":"ContainerStarted","Data":"f8c381b8d7f4b62de6bf349613b8db4c67903d9296900cacf420028e2c68ac53"}
Oct 30 21:37:04.366066 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:04.365987    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-0fa864-76b48d9455-wxt6j" event={"ID":"18d702a7-00ad-4a6b-8de5-806e5900cd7d","Type":"ContainerStarted","Data":"0d899832b76b126f7ee364e0454d922673062cbfb2def07182a739f32bf85d4f"}
Oct 30 21:37:04.367925 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:04.366930    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-0fa864-76b48d9455-wxt6j"
Oct 30 21:37:04.510448 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:04.510396    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-0fa864-76b48d9455-wxt6j" podStartSLOduration=1.669069975 podStartE2EDuration="7.510377885s" totalImagesPullingTime="5.84130791s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:36:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:37:04.391943601 +0000 UTC m=+537.193756065" watchObservedRunningTime="2026-10-30 21:37:04.510377885 +0000 UTC m=+537.312190346"
Oct 30 21:37:04.511385 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:04.511314    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-7dfff0-6ccb87d846-7ml8d"]
Oct 30 21:37:04.512603 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:37:04.512579    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5f6a7d6_010d_4bb5_a333_6f299a5022b5.slice/crio-f636da81d2ddb1a50bd565a6f4f25834e4b174beda11874d8428b1f38204b613 WatchSource:0}: Error finding container f636da81d2ddb1a50bd565a6f4f25834e4b174beda11874d8428b1f38204b613: Status 404 returned error can't find the container with id f636da81d2ddb1a50bd565a6f4f25834e4b174beda11874d8428b1f38204b613
Oct 30 21:37:05.375736 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:05.375698    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-db0b54-758d689765-f75zr" event={"ID":"00d3249f-1824-4ba1-8c18-6f6ab1f7ff92","Type":"ContainerStarted","Data":"ca16fad36ce0915273285b3cb403453d153d91475661a2941daba98cbb240035"}
Oct 30 21:37:05.376196 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:05.375795    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w2-db0b54-758d689765-f75zr"
Oct 30 21:37:05.377886 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:05.377677    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-0fa864-5d85948857-tw7k6" event={"ID":"76de8529-43f7-4285-864b-f8242fca2cae","Type":"ContainerStarted","Data":"8a53fc244ecaf6746c1a1d24f9cec8a7e839613c0bb15552cb0610bb93f1b446"}
Oct 30 21:37:05.377886 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:05.377742    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-0fa864-5d85948857-tw7k6"
Oct 30 21:37:05.379548 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:05.379516    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-7dfff0-6ccb87d846-7ml8d" event={"ID":"b5f6a7d6-010d-4bb5-a333-6f299a5022b5","Type":"ContainerStarted","Data":"aa9393190b852941e794ff7c91f89d633c495233afd09ade66eb9f4afe9393b0"}
Oct 30 21:37:05.379652 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:05.379564    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-7dfff0-6ccb87d846-7ml8d" event={"ID":"b5f6a7d6-010d-4bb5-a333-6f299a5022b5","Type":"ContainerStarted","Data":"f636da81d2ddb1a50bd565a6f4f25834e4b174beda11874d8428b1f38204b613"}
Oct 30 21:37:05.379652 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:05.379597    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-7dfff0-6ccb87d846-7ml8d"
Oct 30 21:37:05.381576 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:05.381520    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-000f64-85b9f75579-vc4bc" event={"ID":"d3bbb2d6-2cff-40ad-90fa-7e232d26a860","Type":"ContainerStarted","Data":"b31f5ee41ac60d32dab66e4d10fbb926e533ab4ff94a8d34169a9921248b9d15"}
Oct 30 21:37:05.381673 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:05.381596    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-000f64-85b9f75579-vc4bc"
Oct 30 21:37:05.384047 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:05.384022    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-000f64-6576d48f75-5rdtz" event={"ID":"c9e3ad2b-93c6-46fd-8b0c-87e24852081e","Type":"ContainerStarted","Data":"e23310c51897580dcafb822b29983854a16210e0688ddc6da4a082275bc747df"}
Oct 30 21:37:05.384160 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:05.384077    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-000f64-6576d48f75-5rdtz"
Oct 30 21:37:05.386213 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:05.386189    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-1d8034-6bbd97dcf7-l2gd8" event={"ID":"e25d6f68-6e47-449f-b02f-03fff2b2bb30","Type":"ContainerStarted","Data":"5e56a198bf2559217d743ea619f914b5b821cf25512ea712631b76860cd9dd7e"}
Oct 30 21:37:05.386685 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:05.386669    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w3-1d8034-6bbd97dcf7-l2gd8"
Oct 30 21:37:05.388947 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:05.388528    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-1d8034-5b96976fc5-hsr78" event={"ID":"27aef427-6b1e-4845-b8d9-a83cba81b0e0","Type":"ContainerStarted","Data":"4a6b5c624625d96186bc32eb917cd7d6554b6357f49b5e34177e00d4855580ff"}
Oct 30 21:37:05.388947 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:05.388628    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-1d8034-5b96976fc5-hsr78"
Oct 30 21:37:05.401881 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:05.401838    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w2-db0b54-758d689765-f75zr" podStartSLOduration=2.904139229 podStartE2EDuration="3.401826444s" totalImagesPullingTime="497.687215ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:37:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:37:05.395832383 +0000 UTC m=+538.197644849" watchObservedRunningTime="2026-10-30 21:37:05.401826444 +0000 UTC m=+538.203638937"
Oct 30 21:37:05.428077 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:05.428034    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-7dfff0-6ccb87d846-7ml8d" podStartSLOduration=1.9856768790000001 podStartE2EDuration="2.428020125s" totalImagesPullingTime="442.343246ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:37:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:37:05.427394252 +0000 UTC m=+538.229206717" watchObservedRunningTime="2026-10-30 21:37:05.428020125 +0000 UTC m=+538.229832627"
Oct 30 21:37:05.450765 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:05.450719    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w3-1d8034-6bbd97dcf7-l2gd8" podStartSLOduration=2.407739501 podStartE2EDuration="6.45070491s" totalImagesPullingTime="4.042965409s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:36:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:37:05.449718106 +0000 UTC m=+538.251530571" watchObservedRunningTime="2026-10-30 21:37:05.45070491 +0000 UTC m=+538.252517373"
Oct 30 21:37:05.472002 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:05.471955    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-1d8034-5b96976fc5-hsr78" podStartSLOduration=2.388345603 podStartE2EDuration="6.471942116s" totalImagesPullingTime="4.083596513s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:36:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:37:05.471114535 +0000 UTC m=+538.272927023" watchObservedRunningTime="2026-10-30 21:37:05.471942116 +0000 UTC m=+538.273754579"
Oct 30 21:37:05.491423 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:05.491343    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-000f64-85b9f75579-vc4bc" podStartSLOduration=1.928261271 podStartE2EDuration="5.491331667s" totalImagesPullingTime="3.563070396s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:37:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:37:05.490090021 +0000 UTC m=+538.291902488" watchObservedRunningTime="2026-10-30 21:37:05.491331667 +0000 UTC m=+538.293144130"
Oct 30 21:37:05.513260 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:05.513213    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w4-0fa864-5d85948857-tw7k6" podStartSLOduration=2.167098651 podStartE2EDuration="8.513200484s" totalImagesPullingTime="6.346101833s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:36:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:37:05.512473996 +0000 UTC m=+538.314286461" watchObservedRunningTime="2026-10-30 21:37:05.513200484 +0000 UTC m=+538.315012948"
Oct 30 21:37:05.539246 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:05.539168    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w0-000f64-6576d48f75-5rdtz" podStartSLOduration=1.9397940070000002 podStartE2EDuration="5.539149756s" totalImagesPullingTime="3.599355749s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:37:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:37:05.536133529 +0000 UTC m=+538.337945996" watchObservedRunningTime="2026-10-30 21:37:05.539149756 +0000 UTC m=+538.340962224"
Oct 30 21:37:06.116886 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:06.116853    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 21:37:06.396284 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:06.396179    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w5-caa3dc-76db8c686c-pxdzh" event={"ID":"467142c6-5675-4dd8-87a0-14e1f9fddc57","Type":"ContainerStarted","Data":"b8c232edce4bac248fb110534e824aea7161314ba5b558c05d1e1a78d986e4de"}
Oct 30 21:37:06.396759 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:06.396294    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w5-caa3dc-76db8c686c-pxdzh"
Oct 30 21:37:06.397937 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:06.397914    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-7dfff0-579b4f5ddc-kxqlf" event={"ID":"b53e75a7-c215-4fc9-a64b-8720382c6acd","Type":"ContainerStarted","Data":"65496a6fcedbf59644f1d4de477af46439ec1b1bc1e7a1c567c89211aa24d2fe"}
Oct 30 21:37:06.398064 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:06.397984    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-7dfff0-579b4f5ddc-kxqlf"
Oct 30 21:37:06.399610 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:06.399583    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-caa3dc-85fc67b867-gsjl2" event={"ID":"28ed1f60-5004-4fb1-bb80-33cd040611f6","Type":"ContainerStarted","Data":"525daf5fc5fd23a19e3d8a883a2bc17bc3d4a939d45c2f36b5490e6fb136c888"}
Oct 30 21:37:06.400831 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:06.400810    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-caa3dc-85fc67b867-gsjl2"
Oct 30 21:37:06.418787 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:06.418739    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w5-caa3dc-76db8c686c-pxdzh" podStartSLOduration=2.200429923 podStartE2EDuration="8.418724531s" totalImagesPullingTime="6.218294608s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:36:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:37:06.415967895 +0000 UTC m=+539.217780370" watchObservedRunningTime="2026-10-30 21:37:06.418724531 +0000 UTC m=+539.220536994"
Oct 30 21:37:06.438756 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:06.438685    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-7dfff0-579b4f5ddc-kxqlf" podStartSLOduration=2.047282602 podStartE2EDuration="5.438662696s" totalImagesPullingTime="3.391380094s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:37:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:37:06.436465209 +0000 UTC m=+539.238277674" watchObservedRunningTime="2026-10-30 21:37:06.438662696 +0000 UTC m=+539.240475161"
Oct 30 21:37:06.462296 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:06.460118    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-caa3dc-85fc67b867-gsjl2" podStartSLOduration=2.036184533 podStartE2EDuration="8.460097313s" totalImagesPullingTime="6.42391278s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:36:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:37:06.458000318 +0000 UTC m=+539.259812802" watchObservedRunningTime="2026-10-30 21:37:06.460097313 +0000 UTC m=+539.261909800"
Oct 30 21:37:06.701685 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:06.701648    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-1d8034-d5d875bdd-ncxns"]
Oct 30 21:37:06.708336 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:06.708310    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-1d8034-d5d875bdd-ncxns"
Oct 30 21:37:06.717866 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:06.717822    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-1d8034-d5d875bdd-ncxns"]
Oct 30 21:37:06.719784 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:06.719679    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w3-1d8034-serving-cert\""
Oct 30 21:37:06.754253 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:06.754217    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d5e4361-e970-43dd-b1e7-66ca197b10ce-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-1d8034-d5d875bdd-ncxns\" (UID: \"5d5e4361-e970-43dd-b1e7-66ca197b10ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-1d8034-d5d875bdd-ncxns"
Oct 30 21:37:06.754476 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:06.754444    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m664x\" (UniqueName: \"kubernetes.io/projected/5d5e4361-e970-43dd-b1e7-66ca197b10ce-kube-api-access-m664x\") pod \"maas-api-e2e-worker-w3-1d8034-d5d875bdd-ncxns\" (UID: \"5d5e4361-e970-43dd-b1e7-66ca197b10ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-1d8034-d5d875bdd-ncxns"
Oct 30 21:37:06.754547 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:06.754506    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5d5e4361-e970-43dd-b1e7-66ca197b10ce-metrics-certs\") pod \"maas-api-e2e-worker-w3-1d8034-d5d875bdd-ncxns\" (UID: \"5d5e4361-e970-43dd-b1e7-66ca197b10ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-1d8034-d5d875bdd-ncxns"
Oct 30 21:37:06.754588 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:06.754558    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d5e4361-e970-43dd-b1e7-66ca197b10ce-maas-api-tls\") pod \"maas-api-e2e-worker-w3-1d8034-d5d875bdd-ncxns\" (UID: \"5d5e4361-e970-43dd-b1e7-66ca197b10ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-1d8034-d5d875bdd-ncxns"
Oct 30 21:37:06.856005 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:06.855961    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d5e4361-e970-43dd-b1e7-66ca197b10ce-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-1d8034-d5d875bdd-ncxns\" (UID: \"5d5e4361-e970-43dd-b1e7-66ca197b10ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-1d8034-d5d875bdd-ncxns"
Oct 30 21:37:06.856166 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:06.856132    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m664x\" (UniqueName: \"kubernetes.io/projected/5d5e4361-e970-43dd-b1e7-66ca197b10ce-kube-api-access-m664x\") pod \"maas-api-e2e-worker-w3-1d8034-d5d875bdd-ncxns\" (UID: \"5d5e4361-e970-43dd-b1e7-66ca197b10ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-1d8034-d5d875bdd-ncxns"
Oct 30 21:37:06.856236 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:06.856183    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5d5e4361-e970-43dd-b1e7-66ca197b10ce-metrics-certs\") pod \"maas-api-e2e-worker-w3-1d8034-d5d875bdd-ncxns\" (UID: \"5d5e4361-e970-43dd-b1e7-66ca197b10ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-1d8034-d5d875bdd-ncxns"
Oct 30 21:37:06.856236 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:06.856214    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d5e4361-e970-43dd-b1e7-66ca197b10ce-maas-api-tls\") pod \"maas-api-e2e-worker-w3-1d8034-d5d875bdd-ncxns\" (UID: \"5d5e4361-e970-43dd-b1e7-66ca197b10ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-1d8034-d5d875bdd-ncxns"
Oct 30 21:37:06.857709 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:06.857680    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d5e4361-e970-43dd-b1e7-66ca197b10ce-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-1d8034-d5d875bdd-ncxns\" (UID: \"5d5e4361-e970-43dd-b1e7-66ca197b10ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-1d8034-d5d875bdd-ncxns"
Oct 30 21:37:06.859442 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:06.859420    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5d5e4361-e970-43dd-b1e7-66ca197b10ce-metrics-certs\") pod \"maas-api-e2e-worker-w3-1d8034-d5d875bdd-ncxns\" (UID: \"5d5e4361-e970-43dd-b1e7-66ca197b10ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-1d8034-d5d875bdd-ncxns"
Oct 30 21:37:06.859592 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:06.859574    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d5e4361-e970-43dd-b1e7-66ca197b10ce-maas-api-tls\") pod \"maas-api-e2e-worker-w3-1d8034-d5d875bdd-ncxns\" (UID: \"5d5e4361-e970-43dd-b1e7-66ca197b10ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-1d8034-d5d875bdd-ncxns"
Oct 30 21:37:06.869730 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:06.869706    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m664x\" (UniqueName: \"kubernetes.io/projected/5d5e4361-e970-43dd-b1e7-66ca197b10ce-kube-api-access-m664x\") pod \"maas-api-e2e-worker-w3-1d8034-d5d875bdd-ncxns\" (UID: \"5d5e4361-e970-43dd-b1e7-66ca197b10ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-1d8034-d5d875bdd-ncxns"
Oct 30 21:37:07.029776 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:07.029691    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-1d8034-d5d875bdd-ncxns"
Oct 30 21:37:07.317458 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:07.317430    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-1d8034-d5d875bdd-ncxns"]
Oct 30 21:37:07.318698 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:37:07.318658    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d5e4361_e970_43dd_b1e7_66ca197b10ce.slice/crio-6b32903f6d61ef1104600062adf3f8a337f17ca52d705df8aafe4c2e2f339dfa WatchSource:0}: Error finding container 6b32903f6d61ef1104600062adf3f8a337f17ca52d705df8aafe4c2e2f339dfa: Status 404 returned error can't find the container with id 6b32903f6d61ef1104600062adf3f8a337f17ca52d705df8aafe4c2e2f339dfa
Oct 30 21:37:07.332665 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:07.332642    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-0fa864-6bc77cf54d-2zd6r"
Oct 30 21:37:07.410475 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:07.410436    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-1d8034-d5d875bdd-ncxns" event={"ID":"5d5e4361-e970-43dd-b1e7-66ca197b10ce","Type":"ContainerStarted","Data":"6b32903f6d61ef1104600062adf3f8a337f17ca52d705df8aafe4c2e2f339dfa"}
Oct 30 21:37:08.343138 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:08.343113    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-000f64-765dc8c9bf-lxnrx"
Oct 30 21:37:08.420431 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:08.420387    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-1d8034-d5d875bdd-ncxns" event={"ID":"5d5e4361-e970-43dd-b1e7-66ca197b10ce","Type":"ContainerStarted","Data":"ad554269e6bfe36fd0e874a69813e144c8123ac4534759e48a2bcd6f404c077c"}
Oct 30 21:37:08.420826 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:08.420528    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-1d8034-d5d875bdd-ncxns"
Oct 30 21:37:08.448023 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:08.447971    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-1d8034-d5d875bdd-ncxns" podStartSLOduration=2.048319163 podStartE2EDuration="2.44795832s" totalImagesPullingTime="399.639157ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:37:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:37:08.445387387 +0000 UTC m=+541.247199852" watchObservedRunningTime="2026-10-30 21:37:08.44795832 +0000 UTC m=+541.249770783"
Oct 30 21:37:11.406051 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:11.406024    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-7dfff0-6ccb87d846-7ml8d"
Oct 30 21:37:14.430670 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:14.430642    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-1d8034-d5d875bdd-ncxns"
Oct 30 21:37:16.403678 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:16.403645    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-1d8034-5b96976fc5-hsr78"
Oct 30 21:37:16.404169 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:16.403966    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-0fa864-5d85948857-tw7k6"
Oct 30 21:37:16.404169 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:16.404103    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w2-db0b54-758d689765-f75zr"
Oct 30 21:37:16.404713 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:16.404691    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-000f64-6576d48f75-5rdtz"
Oct 30 21:37:16.405183 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:16.405160    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-000f64-85b9f75579-vc4bc"
Oct 30 21:37:16.405299 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:16.405214    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-0fa864-76b48d9455-wxt6j"
Oct 30 21:37:17.301272 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:17.301232    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 21:37:17.414112 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:17.414076    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w5-caa3dc-76db8c686c-pxdzh"
Oct 30 21:37:17.414628 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:17.414176    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w3-1d8034-6bbd97dcf7-l2gd8"
Oct 30 21:37:17.414628 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:17.414589    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-7dfff0-579b4f5ddc-kxqlf"
Oct 30 21:37:18.346457 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:37:18.346416    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 21:37:18.423557 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:18.423523    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-caa3dc-85fc67b867-gsjl2"
Oct 30 21:37:32.061609 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:32.061575    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 21:37:47.760986 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:37:47.760904    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 21:37:48.345538 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:37:48.345506    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 21:38:03.247755 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:38:03.247718    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 21:38:07.424928 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:38:07.424902    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-sklxd_67292bdc-7fe1-449d-af75-2baa90c16ea9/console-operator/1.log"
Oct 30 21:38:07.425593 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:38:07.425576    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-sklxd_67292bdc-7fe1-449d-af75-2baa90c16ea9/console-operator/1.log"
Oct 30 21:38:08.192895 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:38:08.192858    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx"]
Oct 30 21:38:08.198786 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:38:08.198759    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx"
Oct 30 21:38:08.205813 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:38:08.205763    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-db0b54\"/\"default-dockercfg-ldhl4\""
Oct 30 21:38:08.205965 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:38:08.205848    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-db0b54\"/\"kube-root-ca.crt\""
Oct 30 21:38:08.206091 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:38:08.206068    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-db0b54\"/\"openshift-service-ca.crt\""
Oct 30 21:38:08.206212 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:38:08.206133    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-db0b54\"/\"premium-w2-db0b54-kserve-self-signed-certs\""
Oct 30 21:38:08.208248 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:38:08.208228    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx"]
Oct 30 21:38:08.344912 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:38:08.344879    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtm86\" (UniqueName: \"kubernetes.io/projected/34fdf57d-86f6-474f-a4f6-811e5063c405-kube-api-access-qtm86\") pod \"premium-w2-db0b54-kserve-84695b566c-4p6rx\" (UID: \"34fdf57d-86f6-474f-a4f6-811e5063c405\") " pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx"
Oct 30 21:38:08.345075 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:38:08.344926    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/34fdf57d-86f6-474f-a4f6-811e5063c405-home\") pod \"premium-w2-db0b54-kserve-84695b566c-4p6rx\" (UID: \"34fdf57d-86f6-474f-a4f6-811e5063c405\") " pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx"
Oct 30 21:38:08.345075 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:38:08.345041    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34fdf57d-86f6-474f-a4f6-811e5063c405-tmp-dir\") pod \"premium-w2-db0b54-kserve-84695b566c-4p6rx\" (UID: \"34fdf57d-86f6-474f-a4f6-811e5063c405\") " pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx"
Oct 30 21:38:08.345147 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:38:08.345078    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/34fdf57d-86f6-474f-a4f6-811e5063c405-model-cache\") pod \"premium-w2-db0b54-kserve-84695b566c-4p6rx\" (UID: \"34fdf57d-86f6-474f-a4f6-811e5063c405\") " pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx"
Oct 30 21:38:08.345147 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:38:08.345101    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/34fdf57d-86f6-474f-a4f6-811e5063c405-tls-certs\") pod \"premium-w2-db0b54-kserve-84695b566c-4p6rx\" (UID: \"34fdf57d-86f6-474f-a4f6-811e5063c405\") " pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx"
Oct 30 21:38:08.345258 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:38:08.345236    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/34fdf57d-86f6-474f-a4f6-811e5063c405-dshm\") pod \"premium-w2-db0b54-kserve-84695b566c-4p6rx\" (UID: \"34fdf57d-86f6-474f-a4f6-811e5063c405\") " pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx"
Oct 30 21:38:08.446191 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:38:08.446104    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtm86\" (UniqueName: \"kubernetes.io/projected/34fdf57d-86f6-474f-a4f6-811e5063c405-kube-api-access-qtm86\") pod \"premium-w2-db0b54-kserve-84695b566c-4p6rx\" (UID: \"34fdf57d-86f6-474f-a4f6-811e5063c405\") " pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx"
Oct 30 21:38:08.446191 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:38:08.446149    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/34fdf57d-86f6-474f-a4f6-811e5063c405-home\") pod \"premium-w2-db0b54-kserve-84695b566c-4p6rx\" (UID: \"34fdf57d-86f6-474f-a4f6-811e5063c405\") " pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx"
Oct 30 21:38:08.446714 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:38:08.446196    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34fdf57d-86f6-474f-a4f6-811e5063c405-tmp-dir\") pod \"premium-w2-db0b54-kserve-84695b566c-4p6rx\" (UID: \"34fdf57d-86f6-474f-a4f6-811e5063c405\") " pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx"
Oct 30 21:38:08.446714 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:38:08.446234    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/34fdf57d-86f6-474f-a4f6-811e5063c405-model-cache\") pod \"premium-w2-db0b54-kserve-84695b566c-4p6rx\" (UID: \"34fdf57d-86f6-474f-a4f6-811e5063c405\") " pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx"
Oct 30 21:38:08.446714 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:38:08.446250    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/34fdf57d-86f6-474f-a4f6-811e5063c405-tls-certs\") pod \"premium-w2-db0b54-kserve-84695b566c-4p6rx\" (UID: \"34fdf57d-86f6-474f-a4f6-811e5063c405\") " pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx"
Oct 30 21:38:08.446714 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:38:08.446350    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/34fdf57d-86f6-474f-a4f6-811e5063c405-dshm\") pod \"premium-w2-db0b54-kserve-84695b566c-4p6rx\" (UID: \"34fdf57d-86f6-474f-a4f6-811e5063c405\") " pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx"
Oct 30 21:38:08.446714 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:38:08.446681    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/34fdf57d-86f6-474f-a4f6-811e5063c405-home\") pod \"premium-w2-db0b54-kserve-84695b566c-4p6rx\" (UID: \"34fdf57d-86f6-474f-a4f6-811e5063c405\") " pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx"
Oct 30 21:38:08.446938 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:38:08.446718    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34fdf57d-86f6-474f-a4f6-811e5063c405-tmp-dir\") pod \"premium-w2-db0b54-kserve-84695b566c-4p6rx\" (UID: \"34fdf57d-86f6-474f-a4f6-811e5063c405\") " pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx"
Oct 30 21:38:08.446938 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:38:08.446923    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/34fdf57d-86f6-474f-a4f6-811e5063c405-model-cache\") pod \"premium-w2-db0b54-kserve-84695b566c-4p6rx\" (UID: \"34fdf57d-86f6-474f-a4f6-811e5063c405\") " pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx"
Oct 30 21:38:08.448712 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:38:08.448694    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/34fdf57d-86f6-474f-a4f6-811e5063c405-dshm\") pod \"premium-w2-db0b54-kserve-84695b566c-4p6rx\" (UID: \"34fdf57d-86f6-474f-a4f6-811e5063c405\") " pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx"
Oct 30 21:38:08.448900 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:38:08.448885    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/34fdf57d-86f6-474f-a4f6-811e5063c405-tls-certs\") pod \"premium-w2-db0b54-kserve-84695b566c-4p6rx\" (UID: \"34fdf57d-86f6-474f-a4f6-811e5063c405\") " pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx"
Oct 30 21:38:08.459072 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:38:08.459040    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtm86\" (UniqueName: \"kubernetes.io/projected/34fdf57d-86f6-474f-a4f6-811e5063c405-kube-api-access-qtm86\") pod \"premium-w2-db0b54-kserve-84695b566c-4p6rx\" (UID: \"34fdf57d-86f6-474f-a4f6-811e5063c405\") " pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx"
Oct 30 21:38:08.512988 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:38:08.512736    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx"
Oct 30 21:38:08.799980 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:38:08.799948    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx"]
Oct 30 21:38:08.802313 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:38:08.802284    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod34fdf57d_86f6_474f_a4f6_811e5063c405.slice/crio-ce49f6e2fa90182255a031091f80456f47f97505c675d5476e8d9a23a5a81892 WatchSource:0}: Error finding container ce49f6e2fa90182255a031091f80456f47f97505c675d5476e8d9a23a5a81892: Status 404 returned error can't find the container with id ce49f6e2fa90182255a031091f80456f47f97505c675d5476e8d9a23a5a81892
Oct 30 21:38:08.812087 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:38:08.812051    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx" event={"ID":"34fdf57d-86f6-474f-a4f6-811e5063c405","Type":"ContainerStarted","Data":"ce49f6e2fa90182255a031091f80456f47f97505c675d5476e8d9a23a5a81892"}
Oct 30 21:38:09.820523 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:38:09.820478    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx" event={"ID":"34fdf57d-86f6-474f-a4f6-811e5063c405","Type":"ContainerStarted","Data":"5f265d992ebd991fccd84b93efd73f1d4b3191b2687f726ef34555318f98777b"}
Oct 30 21:38:09.840261 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:38:09.840177    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx" podStartSLOduration=1.840159151 podStartE2EDuration="1.840159151s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:38:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:38:09.839850455 +0000 UTC m=+602.641662926" watchObservedRunningTime="2026-10-30 21:38:09.840159151 +0000 UTC m=+602.641971616"
Oct 30 21:38:10.512909 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:38:10.512870    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx"
Oct 30 21:38:18.346787 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:38:18.346749    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 21:38:18.513767 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:38:18.513715    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx"
Oct 30 21:38:18.527494 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:38:18.527465    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx"
Oct 30 21:38:18.888786 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:38:18.888758    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx"
Oct 30 21:38:20.878419 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:38:20.878390    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 21:38:47.297275 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:38:47.297238    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 21:38:48.345400 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:38:48.345342    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 21:39:08.608882 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:08.608849    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7"]
Oct 30 21:39:08.613278 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:08.613241    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7"
Oct 30 21:39:08.622840 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:08.622773    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-1a81ae-openshift-default-dockercfg-mbfh7\""
Oct 30 21:39:08.632222 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:08.632155    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7"]
Oct 30 21:39:08.729280 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:08.729218    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d5b3ad5-309e-4c34-89a3-034e4966e652-istio-envoy\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7\" (UID: \"0d5b3ad5-309e-4c34-89a3-034e4966e652\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7"
Oct 30 21:39:08.729643 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:08.729613    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d5b3ad5-309e-4c34-89a3-034e4966e652-istio-podinfo\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7\" (UID: \"0d5b3ad5-309e-4c34-89a3-034e4966e652\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7"
Oct 30 21:39:08.729764 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:08.729651    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d5b3ad5-309e-4c34-89a3-034e4966e652-istio-token\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7\" (UID: \"0d5b3ad5-309e-4c34-89a3-034e4966e652\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7"
Oct 30 21:39:08.729827 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:08.729750    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d5b3ad5-309e-4c34-89a3-034e4966e652-credential-socket\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7\" (UID: \"0d5b3ad5-309e-4c34-89a3-034e4966e652\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7"
Oct 30 21:39:08.729910 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:08.729884    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d5b3ad5-309e-4c34-89a3-034e4966e652-istiod-ca-cert\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7\" (UID: \"0d5b3ad5-309e-4c34-89a3-034e4966e652\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7"
Oct 30 21:39:08.730017 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:08.730002    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d5b3ad5-309e-4c34-89a3-034e4966e652-istio-data\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7\" (UID: \"0d5b3ad5-309e-4c34-89a3-034e4966e652\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7"
Oct 30 21:39:08.730091 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:08.730073    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7vwp\" (UniqueName: \"kubernetes.io/projected/0d5b3ad5-309e-4c34-89a3-034e4966e652-kube-api-access-q7vwp\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7\" (UID: \"0d5b3ad5-309e-4c34-89a3-034e4966e652\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7"
Oct 30 21:39:08.730262 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:08.730238    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d5b3ad5-309e-4c34-89a3-034e4966e652-workload-socket\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7\" (UID: \"0d5b3ad5-309e-4c34-89a3-034e4966e652\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7"
Oct 30 21:39:08.730400 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:08.730324    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d5b3ad5-309e-4c34-89a3-034e4966e652-workload-certs\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7\" (UID: \"0d5b3ad5-309e-4c34-89a3-034e4966e652\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7"
Oct 30 21:39:08.831907 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:08.831869    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d5b3ad5-309e-4c34-89a3-034e4966e652-workload-socket\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7\" (UID: \"0d5b3ad5-309e-4c34-89a3-034e4966e652\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7"
Oct 30 21:39:08.832076 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:08.831924    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d5b3ad5-309e-4c34-89a3-034e4966e652-workload-certs\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7\" (UID: \"0d5b3ad5-309e-4c34-89a3-034e4966e652\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7"
Oct 30 21:39:08.832076 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:08.831963    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d5b3ad5-309e-4c34-89a3-034e4966e652-istio-envoy\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7\" (UID: \"0d5b3ad5-309e-4c34-89a3-034e4966e652\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7"
Oct 30 21:39:08.832076 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:08.831980    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d5b3ad5-309e-4c34-89a3-034e4966e652-istio-podinfo\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7\" (UID: \"0d5b3ad5-309e-4c34-89a3-034e4966e652\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7"
Oct 30 21:39:08.832076 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:08.831996    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d5b3ad5-309e-4c34-89a3-034e4966e652-istio-token\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7\" (UID: \"0d5b3ad5-309e-4c34-89a3-034e4966e652\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7"
Oct 30 21:39:08.832076 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:08.832019    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d5b3ad5-309e-4c34-89a3-034e4966e652-credential-socket\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7\" (UID: \"0d5b3ad5-309e-4c34-89a3-034e4966e652\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7"
Oct 30 21:39:08.832076 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:08.832041    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d5b3ad5-309e-4c34-89a3-034e4966e652-istiod-ca-cert\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7\" (UID: \"0d5b3ad5-309e-4c34-89a3-034e4966e652\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7"
Oct 30 21:39:08.832076 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:08.832066    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d5b3ad5-309e-4c34-89a3-034e4966e652-istio-data\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7\" (UID: \"0d5b3ad5-309e-4c34-89a3-034e4966e652\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7"
Oct 30 21:39:08.832431 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:08.832127    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7vwp\" (UniqueName: \"kubernetes.io/projected/0d5b3ad5-309e-4c34-89a3-034e4966e652-kube-api-access-q7vwp\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7\" (UID: \"0d5b3ad5-309e-4c34-89a3-034e4966e652\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7"
Oct 30 21:39:08.832431 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:08.832356    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d5b3ad5-309e-4c34-89a3-034e4966e652-workload-socket\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7\" (UID: \"0d5b3ad5-309e-4c34-89a3-034e4966e652\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7"
Oct 30 21:39:08.832613 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:08.832583    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d5b3ad5-309e-4c34-89a3-034e4966e652-credential-socket\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7\" (UID: \"0d5b3ad5-309e-4c34-89a3-034e4966e652\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7"
Oct 30 21:39:08.832679 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:08.832649    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d5b3ad5-309e-4c34-89a3-034e4966e652-istio-data\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7\" (UID: \"0d5b3ad5-309e-4c34-89a3-034e4966e652\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7"
Oct 30 21:39:08.832758 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:08.832730    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d5b3ad5-309e-4c34-89a3-034e4966e652-workload-certs\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7\" (UID: \"0d5b3ad5-309e-4c34-89a3-034e4966e652\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7"
Oct 30 21:39:08.832879 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:08.832854    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d5b3ad5-309e-4c34-89a3-034e4966e652-istiod-ca-cert\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7\" (UID: \"0d5b3ad5-309e-4c34-89a3-034e4966e652\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7"
Oct 30 21:39:08.834774 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:08.834752    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d5b3ad5-309e-4c34-89a3-034e4966e652-istio-podinfo\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7\" (UID: \"0d5b3ad5-309e-4c34-89a3-034e4966e652\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7"
Oct 30 21:39:08.834881 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:08.834818    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d5b3ad5-309e-4c34-89a3-034e4966e652-istio-envoy\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7\" (UID: \"0d5b3ad5-309e-4c34-89a3-034e4966e652\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7"
Oct 30 21:39:08.844878 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:08.844849    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7vwp\" (UniqueName: \"kubernetes.io/projected/0d5b3ad5-309e-4c34-89a3-034e4966e652-kube-api-access-q7vwp\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7\" (UID: \"0d5b3ad5-309e-4c34-89a3-034e4966e652\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7"
Oct 30 21:39:08.844878 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:08.844868    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d5b3ad5-309e-4c34-89a3-034e4966e652-istio-token\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7\" (UID: \"0d5b3ad5-309e-4c34-89a3-034e4966e652\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7"
Oct 30 21:39:08.948613 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:08.948535    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7"
Oct 30 21:39:09.241114 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:09.241080    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7"]
Oct 30 21:39:09.242909 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:39:09.242877    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d5b3ad5_309e_4c34_89a3_034e4966e652.slice/crio-d4bfa675eb96d503b527fe91ff678783808a554a1094a2c9cb1306cbfd6dcecc WatchSource:0}: Error finding container d4bfa675eb96d503b527fe91ff678783808a554a1094a2c9cb1306cbfd6dcecc: Status 404 returned error can't find the container with id d4bfa675eb96d503b527fe91ff678783808a554a1094a2c9cb1306cbfd6dcecc
Oct 30 21:39:09.247859 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:09.247808    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:39:09.250467 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:09.247913    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:39:09.250467 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:09.247960    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:39:10.199700 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:10.199612    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7" event={"ID":"0d5b3ad5-309e-4c34-89a3-034e4966e652","Type":"ContainerStarted","Data":"105e63cdb204bccc898f9b89b178f9a801281db6bf9fbbad88078ba3a60be6cb"}
Oct 30 21:39:10.199700 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:10.199669    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7" event={"ID":"0d5b3ad5-309e-4c34-89a3-034e4966e652","Type":"ContainerStarted","Data":"d4bfa675eb96d503b527fe91ff678783808a554a1094a2c9cb1306cbfd6dcecc"}
Oct 30 21:39:10.231383 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:10.230584    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7" podStartSLOduration=2.230561406 podStartE2EDuration="2.230561406s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:39:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:39:10.229707711 +0000 UTC m=+663.031520177" watchObservedRunningTime="2026-10-30 21:39:10.230561406 +0000 UTC m=+663.032373872"
Oct 30 21:39:10.949060 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:10.949025    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7"
Oct 30 21:39:10.950458 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:10.950428    5032 patch_prober.go:29] interesting pod/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Oct 30 21:39:10.950582 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:10.950498    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7" podUID="0d5b3ad5-309e-4c34-89a3-034e4966e652" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Oct 30 21:39:11.950397 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:11.949170    5032 patch_prober.go:29] interesting pod/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Oct 30 21:39:11.950397 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:11.949237    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7" podUID="0d5b3ad5-309e-4c34-89a3-034e4966e652" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Oct 30 21:39:12.949524 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:12.949484    5032 patch_prober.go:29] interesting pod/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Oct 30 21:39:12.949739 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:12.949537    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7" podUID="0d5b3ad5-309e-4c34-89a3-034e4966e652" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Oct 30 21:39:13.949425 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:13.949386    5032 patch_prober.go:29] interesting pod/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Oct 30 21:39:13.949934 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:13.949444    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7" podUID="0d5b3ad5-309e-4c34-89a3-034e4966e652" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Oct 30 21:39:14.275451 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:14.275418    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 21:39:14.949351 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:14.949317    5032 patch_prober.go:29] interesting pod/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Oct 30 21:39:14.949539 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:14.949411    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7" podUID="0d5b3ad5-309e-4c34-89a3-034e4966e652" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Oct 30 21:39:17.002534 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:17.002480    5032 patch_prober.go:29] interesting pod/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:39:17.003156 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:17.003120    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7" podUID="0d5b3ad5-309e-4c34-89a3-034e4966e652" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:39:17.140028 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:17.138434    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7"
Oct 30 21:39:17.140028 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:17.139862    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7"
Oct 30 21:39:17.153390 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:17.152830    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7"
Oct 30 21:39:18.196039 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:18.195630    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg"]
Oct 30 21:39:18.203769 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:18.203633    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg"]
Oct 30 21:39:18.203769 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:18.203756    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg"
Oct 30 21:39:18.237390 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:18.236632    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-1d8034\"/\"kube-root-ca.crt\""
Oct 30 21:39:18.237390 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:18.236729    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-1d8034\"/\"openshift-service-ca.crt\""
Oct 30 21:39:18.237390 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:18.236839    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-1d8034\"/\"default-dockercfg-n6qtb\""
Oct 30 21:39:18.237390 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:18.236910    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-1d8034\"/\"distinct-w3-1d8034-kserve-self-signed-certs\""
Oct 30 21:39:18.306899 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:18.306862    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9zz8\" (UniqueName: \"kubernetes.io/projected/2d0ea058-b6ca-4181-8823-2b719883e545-kube-api-access-t9zz8\") pod \"distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg\" (UID: \"2d0ea058-b6ca-4181-8823-2b719883e545\") " pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg"
Oct 30 21:39:18.307112 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:18.306907    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d0ea058-b6ca-4181-8823-2b719883e545-home\") pod \"distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg\" (UID: \"2d0ea058-b6ca-4181-8823-2b719883e545\") " pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg"
Oct 30 21:39:18.307112 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:18.307085    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d0ea058-b6ca-4181-8823-2b719883e545-model-cache\") pod \"distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg\" (UID: \"2d0ea058-b6ca-4181-8823-2b719883e545\") " pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg"
Oct 30 21:39:18.307258 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:18.307202    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d0ea058-b6ca-4181-8823-2b719883e545-tls-certs\") pod \"distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg\" (UID: \"2d0ea058-b6ca-4181-8823-2b719883e545\") " pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg"
Oct 30 21:39:18.307258 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:18.307245    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d0ea058-b6ca-4181-8823-2b719883e545-tmp-dir\") pod \"distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg\" (UID: \"2d0ea058-b6ca-4181-8823-2b719883e545\") " pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg"
Oct 30 21:39:18.307386 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:18.307309    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d0ea058-b6ca-4181-8823-2b719883e545-dshm\") pod \"distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg\" (UID: \"2d0ea058-b6ca-4181-8823-2b719883e545\") " pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg"
Oct 30 21:39:18.355772 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:39:18.355714    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 21:39:18.408171 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:18.408132    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9zz8\" (UniqueName: \"kubernetes.io/projected/2d0ea058-b6ca-4181-8823-2b719883e545-kube-api-access-t9zz8\") pod \"distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg\" (UID: \"2d0ea058-b6ca-4181-8823-2b719883e545\") " pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg"
Oct 30 21:39:18.408399 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:18.408184    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d0ea058-b6ca-4181-8823-2b719883e545-home\") pod \"distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg\" (UID: \"2d0ea058-b6ca-4181-8823-2b719883e545\") " pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg"
Oct 30 21:39:18.408399 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:18.408217    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d0ea058-b6ca-4181-8823-2b719883e545-model-cache\") pod \"distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg\" (UID: \"2d0ea058-b6ca-4181-8823-2b719883e545\") " pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg"
Oct 30 21:39:18.408399 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:18.408305    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d0ea058-b6ca-4181-8823-2b719883e545-tls-certs\") pod \"distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg\" (UID: \"2d0ea058-b6ca-4181-8823-2b719883e545\") " pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg"
Oct 30 21:39:18.408585 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:18.408477    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d0ea058-b6ca-4181-8823-2b719883e545-tmp-dir\") pod \"distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg\" (UID: \"2d0ea058-b6ca-4181-8823-2b719883e545\") " pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg"
Oct 30 21:39:18.408585 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:18.408545    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d0ea058-b6ca-4181-8823-2b719883e545-dshm\") pod \"distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg\" (UID: \"2d0ea058-b6ca-4181-8823-2b719883e545\") " pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg"
Oct 30 21:39:18.408697 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:18.408651    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d0ea058-b6ca-4181-8823-2b719883e545-home\") pod \"distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg\" (UID: \"2d0ea058-b6ca-4181-8823-2b719883e545\") " pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg"
Oct 30 21:39:18.408697 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:18.408666    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d0ea058-b6ca-4181-8823-2b719883e545-model-cache\") pod \"distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg\" (UID: \"2d0ea058-b6ca-4181-8823-2b719883e545\") " pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg"
Oct 30 21:39:18.408876 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:18.408858    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d0ea058-b6ca-4181-8823-2b719883e545-tmp-dir\") pod \"distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg\" (UID: \"2d0ea058-b6ca-4181-8823-2b719883e545\") " pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg"
Oct 30 21:39:18.411703 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:18.411676    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d0ea058-b6ca-4181-8823-2b719883e545-tls-certs\") pod \"distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg\" (UID: \"2d0ea058-b6ca-4181-8823-2b719883e545\") " pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg"
Oct 30 21:39:18.422854 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:18.422717    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9zz8\" (UniqueName: \"kubernetes.io/projected/2d0ea058-b6ca-4181-8823-2b719883e545-kube-api-access-t9zz8\") pod \"distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg\" (UID: \"2d0ea058-b6ca-4181-8823-2b719883e545\") " pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg"
Oct 30 21:39:18.425887 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:18.425847    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d0ea058-b6ca-4181-8823-2b719883e545-dshm\") pod \"distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg\" (UID: \"2d0ea058-b6ca-4181-8823-2b719883e545\") " pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg"
Oct 30 21:39:18.596684 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:18.596645    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg"
Oct 30 21:39:18.890981 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:18.890950    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg"]
Oct 30 21:39:18.892087 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:39:18.892061    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2d0ea058_b6ca_4181_8823_2b719883e545.slice/crio-87c54864389501b136ba8b2ea4d7da6a0880791e10ae9f83177efc1abfef2efa WatchSource:0}: Error finding container 87c54864389501b136ba8b2ea4d7da6a0880791e10ae9f83177efc1abfef2efa: Status 404 returned error can't find the container with id 87c54864389501b136ba8b2ea4d7da6a0880791e10ae9f83177efc1abfef2efa
Oct 30 21:39:19.318819 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:19.318776    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg" event={"ID":"2d0ea058-b6ca-4181-8823-2b719883e545","Type":"ContainerStarted","Data":"6797a8c03a4ba0206dd9a003cee3991819a6ab7cce24fc5acfb065cc7557ca8a"}
Oct 30 21:39:19.319320 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:19.318831    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg" event={"ID":"2d0ea058-b6ca-4181-8823-2b719883e545","Type":"ContainerStarted","Data":"87c54864389501b136ba8b2ea4d7da6a0880791e10ae9f83177efc1abfef2efa"}
Oct 30 21:39:19.597216 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:19.597140    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg"
Oct 30 21:39:23.214903 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:23.214842    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg" podStartSLOduration=5.214821376 podStartE2EDuration="5.214821376s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:39:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:39:19.341128801 +0000 UTC m=+672.142941264" watchObservedRunningTime="2026-10-30 21:39:23.214821376 +0000 UTC m=+676.016633839"
Oct 30 21:39:23.216997 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:23.216967    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq"]
Oct 30 21:39:23.221521 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:23.221498    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq"
Oct 30 21:39:23.225501 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:23.225471    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w5-caa3dc-openshift-default-dockercfg-5kw62\""
Oct 30 21:39:23.233686 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:23.233641    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq"]
Oct 30 21:39:23.370972 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:23.370941    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbbz2\" (UniqueName: \"kubernetes.io/projected/968e86d5-7b2e-4f83-ad1d-68b62aadfa18-kube-api-access-zbbz2\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq\" (UID: \"968e86d5-7b2e-4f83-ad1d-68b62aadfa18\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq"
Oct 30 21:39:23.371157 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:23.371002    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/968e86d5-7b2e-4f83-ad1d-68b62aadfa18-istio-token\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq\" (UID: \"968e86d5-7b2e-4f83-ad1d-68b62aadfa18\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq"
Oct 30 21:39:23.371157 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:23.371109    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/968e86d5-7b2e-4f83-ad1d-68b62aadfa18-istio-podinfo\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq\" (UID: \"968e86d5-7b2e-4f83-ad1d-68b62aadfa18\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq"
Oct 30 21:39:23.371157 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:23.371141    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/968e86d5-7b2e-4f83-ad1d-68b62aadfa18-workload-socket\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq\" (UID: \"968e86d5-7b2e-4f83-ad1d-68b62aadfa18\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq"
Oct 30 21:39:23.371312 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:23.371165    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/968e86d5-7b2e-4f83-ad1d-68b62aadfa18-credential-socket\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq\" (UID: \"968e86d5-7b2e-4f83-ad1d-68b62aadfa18\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq"
Oct 30 21:39:23.371312 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:23.371244    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/968e86d5-7b2e-4f83-ad1d-68b62aadfa18-istio-envoy\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq\" (UID: \"968e86d5-7b2e-4f83-ad1d-68b62aadfa18\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq"
Oct 30 21:39:23.371450 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:23.371432    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/968e86d5-7b2e-4f83-ad1d-68b62aadfa18-workload-certs\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq\" (UID: \"968e86d5-7b2e-4f83-ad1d-68b62aadfa18\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq"
Oct 30 21:39:23.371563 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:23.371546    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/968e86d5-7b2e-4f83-ad1d-68b62aadfa18-istio-data\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq\" (UID: \"968e86d5-7b2e-4f83-ad1d-68b62aadfa18\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq"
Oct 30 21:39:23.371623 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:23.371581    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/968e86d5-7b2e-4f83-ad1d-68b62aadfa18-istiod-ca-cert\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq\" (UID: \"968e86d5-7b2e-4f83-ad1d-68b62aadfa18\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq"
Oct 30 21:39:23.472937 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:23.472851    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/968e86d5-7b2e-4f83-ad1d-68b62aadfa18-istio-data\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq\" (UID: \"968e86d5-7b2e-4f83-ad1d-68b62aadfa18\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq"
Oct 30 21:39:23.472937 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:23.472892    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/968e86d5-7b2e-4f83-ad1d-68b62aadfa18-istiod-ca-cert\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq\" (UID: \"968e86d5-7b2e-4f83-ad1d-68b62aadfa18\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq"
Oct 30 21:39:23.472937 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:23.472937    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbbz2\" (UniqueName: \"kubernetes.io/projected/968e86d5-7b2e-4f83-ad1d-68b62aadfa18-kube-api-access-zbbz2\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq\" (UID: \"968e86d5-7b2e-4f83-ad1d-68b62aadfa18\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq"
Oct 30 21:39:23.473194 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:23.472975    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/968e86d5-7b2e-4f83-ad1d-68b62aadfa18-istio-token\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq\" (UID: \"968e86d5-7b2e-4f83-ad1d-68b62aadfa18\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq"
Oct 30 21:39:23.473194 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:23.473124    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/968e86d5-7b2e-4f83-ad1d-68b62aadfa18-istio-podinfo\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq\" (UID: \"968e86d5-7b2e-4f83-ad1d-68b62aadfa18\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq"
Oct 30 21:39:23.473194 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:23.473175    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/968e86d5-7b2e-4f83-ad1d-68b62aadfa18-workload-socket\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq\" (UID: \"968e86d5-7b2e-4f83-ad1d-68b62aadfa18\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq"
Oct 30 21:39:23.473349 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:23.473215    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/968e86d5-7b2e-4f83-ad1d-68b62aadfa18-credential-socket\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq\" (UID: \"968e86d5-7b2e-4f83-ad1d-68b62aadfa18\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq"
Oct 30 21:39:23.473349 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:23.473259    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/968e86d5-7b2e-4f83-ad1d-68b62aadfa18-istio-envoy\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq\" (UID: \"968e86d5-7b2e-4f83-ad1d-68b62aadfa18\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq"
Oct 30 21:39:23.473349 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:23.473285    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/968e86d5-7b2e-4f83-ad1d-68b62aadfa18-istio-data\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq\" (UID: \"968e86d5-7b2e-4f83-ad1d-68b62aadfa18\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq"
Oct 30 21:39:23.473349 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:23.473311    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/968e86d5-7b2e-4f83-ad1d-68b62aadfa18-workload-certs\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq\" (UID: \"968e86d5-7b2e-4f83-ad1d-68b62aadfa18\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq"
Oct 30 21:39:23.473586 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:23.473565    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/968e86d5-7b2e-4f83-ad1d-68b62aadfa18-workload-socket\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq\" (UID: \"968e86d5-7b2e-4f83-ad1d-68b62aadfa18\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq"
Oct 30 21:39:23.473676 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:23.473652    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/968e86d5-7b2e-4f83-ad1d-68b62aadfa18-credential-socket\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq\" (UID: \"968e86d5-7b2e-4f83-ad1d-68b62aadfa18\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq"
Oct 30 21:39:23.473737 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:23.473707    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/968e86d5-7b2e-4f83-ad1d-68b62aadfa18-istiod-ca-cert\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq\" (UID: \"968e86d5-7b2e-4f83-ad1d-68b62aadfa18\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq"
Oct 30 21:39:23.473737 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:23.473713    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/968e86d5-7b2e-4f83-ad1d-68b62aadfa18-workload-certs\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq\" (UID: \"968e86d5-7b2e-4f83-ad1d-68b62aadfa18\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq"
Oct 30 21:39:23.475508 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:23.475489    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/968e86d5-7b2e-4f83-ad1d-68b62aadfa18-istio-envoy\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq\" (UID: \"968e86d5-7b2e-4f83-ad1d-68b62aadfa18\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq"
Oct 30 21:39:23.475688 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:23.475670    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/968e86d5-7b2e-4f83-ad1d-68b62aadfa18-istio-podinfo\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq\" (UID: \"968e86d5-7b2e-4f83-ad1d-68b62aadfa18\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq"
Oct 30 21:39:23.485699 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:23.485671    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/968e86d5-7b2e-4f83-ad1d-68b62aadfa18-istio-token\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq\" (UID: \"968e86d5-7b2e-4f83-ad1d-68b62aadfa18\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq"
Oct 30 21:39:23.485917 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:23.485896    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbbz2\" (UniqueName: \"kubernetes.io/projected/968e86d5-7b2e-4f83-ad1d-68b62aadfa18-kube-api-access-zbbz2\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq\" (UID: \"968e86d5-7b2e-4f83-ad1d-68b62aadfa18\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq"
Oct 30 21:39:23.539541 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:23.539508    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq"
Oct 30 21:39:23.827409 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:23.827347    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq"]
Oct 30 21:39:23.828607 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:39:23.828574    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod968e86d5_7b2e_4f83_ad1d_68b62aadfa18.slice/crio-5d4d902a7f1deb25319533676bbfec2625aabaece20e66d678bddfad038e8422 WatchSource:0}: Error finding container 5d4d902a7f1deb25319533676bbfec2625aabaece20e66d678bddfad038e8422: Status 404 returned error can't find the container with id 5d4d902a7f1deb25319533676bbfec2625aabaece20e66d678bddfad038e8422
Oct 30 21:39:23.831383 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:23.831332    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:39:23.831481 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:23.831423    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:39:23.831481 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:23.831455    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:39:24.356962 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:24.356925    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq" event={"ID":"968e86d5-7b2e-4f83-ad1d-68b62aadfa18","Type":"ContainerStarted","Data":"ed9efed975857e81ed94a5d09e635a296b0334d2a218ab09ffc137bb102c3abc"}
Oct 30 21:39:24.357402 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:24.356979    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq" event={"ID":"968e86d5-7b2e-4f83-ad1d-68b62aadfa18","Type":"ContainerStarted","Data":"5d4d902a7f1deb25319533676bbfec2625aabaece20e66d678bddfad038e8422"}
Oct 30 21:39:24.382714 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:24.382663    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq" podStartSLOduration=1.382648992 podStartE2EDuration="1.382648992s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:39:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:39:24.379532942 +0000 UTC m=+677.181345406" watchObservedRunningTime="2026-10-30 21:39:24.382648992 +0000 UTC m=+677.184461453"
Oct 30 21:39:24.540564 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:24.540525    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq"
Oct 30 21:39:24.542319 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:24.542297    5032 patch_prober.go:29] interesting pod/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Oct 30 21:39:24.542403 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:24.542347    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq" podUID="968e86d5-7b2e-4f83-ad1d-68b62aadfa18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Oct 30 21:39:25.540647 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:25.540609    5032 patch_prober.go:29] interesting pod/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Oct 30 21:39:25.541041 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:25.540668    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq" podUID="968e86d5-7b2e-4f83-ad1d-68b62aadfa18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Oct 30 21:39:26.540384 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:26.540007    5032 patch_prober.go:29] interesting pod/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Oct 30 21:39:26.540384 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:26.540107    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq" podUID="968e86d5-7b2e-4f83-ad1d-68b62aadfa18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Oct 30 21:39:27.540384 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:27.540299    5032 patch_prober.go:29] interesting pod/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Oct 30 21:39:27.540937 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:27.540393    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq" podUID="968e86d5-7b2e-4f83-ad1d-68b62aadfa18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Oct 30 21:39:28.540728 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:28.540690    5032 patch_prober.go:29] interesting pod/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Oct 30 21:39:28.541240 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:28.540744    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq" podUID="968e86d5-7b2e-4f83-ad1d-68b62aadfa18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Oct 30 21:39:28.596905 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:28.596870    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg"
Oct 30 21:39:28.635148 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:28.635112    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg"
Oct 30 21:39:29.401096 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:29.401067    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg"
Oct 30 21:39:29.540949 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:29.540912    5032 patch_prober.go:29] interesting pod/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Oct 30 21:39:29.541542 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:29.541504    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq" podUID="968e86d5-7b2e-4f83-ad1d-68b62aadfa18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Oct 30 21:39:30.828893 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:30.828849    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq"
Oct 30 21:39:30.830141 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:30.830115    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq"
Oct 30 21:39:30.831208 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:30.831187    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq"
Oct 30 21:39:32.018526 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:32.018481    5032 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-c46gq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:39:32.019158 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:32.019125    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-c46gq" podUID="e3b353c1-3812-44aa-b77d-d7ddfc3b7057" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:39:33.747791 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:33.747741    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj"]
Oct 30 21:39:33.752891 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:33.752866    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj"
Oct 30 21:39:33.759048 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:33.759023    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-caa3dc\"/\"kube-root-ca.crt\""
Oct 30 21:39:33.767508 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:33.763631    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-caa3dc\"/\"distinct-w5-caa3dc-kserve-self-signed-certs\""
Oct 30 21:39:33.767508 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:33.763760    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-caa3dc\"/\"default-dockercfg-c2fpr\""
Oct 30 21:39:33.767508 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:33.763804    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-caa3dc\"/\"openshift-service-ca.crt\""
Oct 30 21:39:33.767508 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:33.766436    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj"]
Oct 30 21:39:33.851314 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:33.851278    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0ff5023-fd3f-4734-94c6-76b666010adf-tls-certs\") pod \"distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj\" (UID: \"d0ff5023-fd3f-4734-94c6-76b666010adf\") " pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj"
Oct 30 21:39:33.851522 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:33.851458    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0ff5023-fd3f-4734-94c6-76b666010adf-model-cache\") pod \"distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj\" (UID: \"d0ff5023-fd3f-4734-94c6-76b666010adf\") " pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj"
Oct 30 21:39:33.851522 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:33.851486    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0ff5023-fd3f-4734-94c6-76b666010adf-tmp-dir\") pod \"distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj\" (UID: \"d0ff5023-fd3f-4734-94c6-76b666010adf\") " pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj"
Oct 30 21:39:33.851645 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:33.851619    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0ff5023-fd3f-4734-94c6-76b666010adf-dshm\") pod \"distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj\" (UID: \"d0ff5023-fd3f-4734-94c6-76b666010adf\") " pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj"
Oct 30 21:39:33.851692 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:33.851676    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mfgq\" (UniqueName: \"kubernetes.io/projected/d0ff5023-fd3f-4734-94c6-76b666010adf-kube-api-access-7mfgq\") pod \"distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj\" (UID: \"d0ff5023-fd3f-4734-94c6-76b666010adf\") " pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj"
Oct 30 21:39:33.851741 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:33.851731    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0ff5023-fd3f-4734-94c6-76b666010adf-home\") pod \"distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj\" (UID: \"d0ff5023-fd3f-4734-94c6-76b666010adf\") " pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj"
Oct 30 21:39:33.952800 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:33.952759    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0ff5023-fd3f-4734-94c6-76b666010adf-model-cache\") pod \"distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj\" (UID: \"d0ff5023-fd3f-4734-94c6-76b666010adf\") " pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj"
Oct 30 21:39:33.952993 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:33.952812    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0ff5023-fd3f-4734-94c6-76b666010adf-tmp-dir\") pod \"distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj\" (UID: \"d0ff5023-fd3f-4734-94c6-76b666010adf\") " pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj"
Oct 30 21:39:33.952993 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:33.952900    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0ff5023-fd3f-4734-94c6-76b666010adf-dshm\") pod \"distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj\" (UID: \"d0ff5023-fd3f-4734-94c6-76b666010adf\") " pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj"
Oct 30 21:39:33.952993 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:33.952936    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mfgq\" (UniqueName: \"kubernetes.io/projected/d0ff5023-fd3f-4734-94c6-76b666010adf-kube-api-access-7mfgq\") pod \"distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj\" (UID: \"d0ff5023-fd3f-4734-94c6-76b666010adf\") " pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj"
Oct 30 21:39:33.952993 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:33.952978    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0ff5023-fd3f-4734-94c6-76b666010adf-home\") pod \"distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj\" (UID: \"d0ff5023-fd3f-4734-94c6-76b666010adf\") " pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj"
Oct 30 21:39:33.953209 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:33.953024    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0ff5023-fd3f-4734-94c6-76b666010adf-tls-certs\") pod \"distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj\" (UID: \"d0ff5023-fd3f-4734-94c6-76b666010adf\") " pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj"
Oct 30 21:39:33.953425 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:33.953396    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0ff5023-fd3f-4734-94c6-76b666010adf-model-cache\") pod \"distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj\" (UID: \"d0ff5023-fd3f-4734-94c6-76b666010adf\") " pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj"
Oct 30 21:39:33.953712 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:33.953687    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0ff5023-fd3f-4734-94c6-76b666010adf-home\") pod \"distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj\" (UID: \"d0ff5023-fd3f-4734-94c6-76b666010adf\") " pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj"
Oct 30 21:39:33.953947 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:33.953925    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0ff5023-fd3f-4734-94c6-76b666010adf-tmp-dir\") pod \"distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj\" (UID: \"d0ff5023-fd3f-4734-94c6-76b666010adf\") " pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj"
Oct 30 21:39:33.955808 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:33.955785    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0ff5023-fd3f-4734-94c6-76b666010adf-dshm\") pod \"distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj\" (UID: \"d0ff5023-fd3f-4734-94c6-76b666010adf\") " pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj"
Oct 30 21:39:33.959235 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:33.959189    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0ff5023-fd3f-4734-94c6-76b666010adf-tls-certs\") pod \"distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj\" (UID: \"d0ff5023-fd3f-4734-94c6-76b666010adf\") " pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj"
Oct 30 21:39:33.966274 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:33.966249    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mfgq\" (UniqueName: \"kubernetes.io/projected/d0ff5023-fd3f-4734-94c6-76b666010adf-kube-api-access-7mfgq\") pod \"distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj\" (UID: \"d0ff5023-fd3f-4734-94c6-76b666010adf\") " pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj"
Oct 30 21:39:34.074164 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:34.074079    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj"
Oct 30 21:39:34.362897 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:34.362866    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj"]
Oct 30 21:39:34.364559 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:39:34.364534    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd0ff5023_fd3f_4734_94c6_76b666010adf.slice/crio-118a0f2de3563d5cb90ed43134d621c9e958b41f296c5b96f0b0e85241a9b6bb WatchSource:0}: Error finding container 118a0f2de3563d5cb90ed43134d621c9e958b41f296c5b96f0b0e85241a9b6bb: Status 404 returned error can't find the container with id 118a0f2de3563d5cb90ed43134d621c9e958b41f296c5b96f0b0e85241a9b6bb
Oct 30 21:39:34.703724 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:34.703692    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj" event={"ID":"d0ff5023-fd3f-4734-94c6-76b666010adf","Type":"ContainerStarted","Data":"ea94fdb68f46ee7796c3cfa2a0f1a71db77b61558ac603043e078636e6316c03"}
Oct 30 21:39:34.703925 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:34.703737    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj" event={"ID":"d0ff5023-fd3f-4734-94c6-76b666010adf","Type":"ContainerStarted","Data":"118a0f2de3563d5cb90ed43134d621c9e958b41f296c5b96f0b0e85241a9b6bb"}
Oct 30 21:39:34.725641 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:34.725589    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj" podStartSLOduration=1.725574324 podStartE2EDuration="1.725574324s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:39:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:39:34.723265325 +0000 UTC m=+687.525077803" watchObservedRunningTime="2026-10-30 21:39:34.725574324 +0000 UTC m=+687.527386844"
Oct 30 21:39:35.074531 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:35.074444    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj"
Oct 30 21:39:38.114778 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.114739    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b"]
Oct 30 21:39:38.119180 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.119160    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b"
Oct 30 21:39:38.123491 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.123469    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-0fa864-openshift-default-dockercfg-bwzs2\""
Oct 30 21:39:38.131910 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.131883    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b"]
Oct 30 21:39:38.203027 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.202992    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3aee81c5-228b-4f21-b7d9-a0bd0822b4df-credential-socket\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b\" (UID: \"3aee81c5-228b-4f21-b7d9-a0bd0822b4df\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b"
Oct 30 21:39:38.203199 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.203072    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3aee81c5-228b-4f21-b7d9-a0bd0822b4df-istio-data\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b\" (UID: \"3aee81c5-228b-4f21-b7d9-a0bd0822b4df\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b"
Oct 30 21:39:38.203199 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.203136    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3aee81c5-228b-4f21-b7d9-a0bd0822b4df-istiod-ca-cert\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b\" (UID: \"3aee81c5-228b-4f21-b7d9-a0bd0822b4df\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b"
Oct 30 21:39:38.203199 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.203154    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5z877\" (UniqueName: \"kubernetes.io/projected/3aee81c5-228b-4f21-b7d9-a0bd0822b4df-kube-api-access-5z877\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b\" (UID: \"3aee81c5-228b-4f21-b7d9-a0bd0822b4df\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b"
Oct 30 21:39:38.203199 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.203186    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3aee81c5-228b-4f21-b7d9-a0bd0822b4df-istio-podinfo\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b\" (UID: \"3aee81c5-228b-4f21-b7d9-a0bd0822b4df\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b"
Oct 30 21:39:38.203350 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.203276    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3aee81c5-228b-4f21-b7d9-a0bd0822b4df-istio-token\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b\" (UID: \"3aee81c5-228b-4f21-b7d9-a0bd0822b4df\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b"
Oct 30 21:39:38.203350 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.203311    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3aee81c5-228b-4f21-b7d9-a0bd0822b4df-workload-socket\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b\" (UID: \"3aee81c5-228b-4f21-b7d9-a0bd0822b4df\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b"
Oct 30 21:39:38.203475 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.203373    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3aee81c5-228b-4f21-b7d9-a0bd0822b4df-workload-certs\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b\" (UID: \"3aee81c5-228b-4f21-b7d9-a0bd0822b4df\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b"
Oct 30 21:39:38.203475 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.203402    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3aee81c5-228b-4f21-b7d9-a0bd0822b4df-istio-envoy\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b\" (UID: \"3aee81c5-228b-4f21-b7d9-a0bd0822b4df\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b"
Oct 30 21:39:38.304194 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.304156    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3aee81c5-228b-4f21-b7d9-a0bd0822b4df-istiod-ca-cert\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b\" (UID: \"3aee81c5-228b-4f21-b7d9-a0bd0822b4df\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b"
Oct 30 21:39:38.304194 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.304193    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5z877\" (UniqueName: \"kubernetes.io/projected/3aee81c5-228b-4f21-b7d9-a0bd0822b4df-kube-api-access-5z877\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b\" (UID: \"3aee81c5-228b-4f21-b7d9-a0bd0822b4df\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b"
Oct 30 21:39:38.304440 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.304228    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3aee81c5-228b-4f21-b7d9-a0bd0822b4df-istio-podinfo\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b\" (UID: \"3aee81c5-228b-4f21-b7d9-a0bd0822b4df\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b"
Oct 30 21:39:38.304440 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.304255    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3aee81c5-228b-4f21-b7d9-a0bd0822b4df-istio-token\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b\" (UID: \"3aee81c5-228b-4f21-b7d9-a0bd0822b4df\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b"
Oct 30 21:39:38.304440 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.304283    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3aee81c5-228b-4f21-b7d9-a0bd0822b4df-workload-socket\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b\" (UID: \"3aee81c5-228b-4f21-b7d9-a0bd0822b4df\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b"
Oct 30 21:39:38.304605 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.304489    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3aee81c5-228b-4f21-b7d9-a0bd0822b4df-workload-certs\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b\" (UID: \"3aee81c5-228b-4f21-b7d9-a0bd0822b4df\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b"
Oct 30 21:39:38.304605 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.304559    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3aee81c5-228b-4f21-b7d9-a0bd0822b4df-istio-envoy\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b\" (UID: \"3aee81c5-228b-4f21-b7d9-a0bd0822b4df\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b"
Oct 30 21:39:38.304701 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.304681    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3aee81c5-228b-4f21-b7d9-a0bd0822b4df-credential-socket\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b\" (UID: \"3aee81c5-228b-4f21-b7d9-a0bd0822b4df\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b"
Oct 30 21:39:38.304772 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.304745    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3aee81c5-228b-4f21-b7d9-a0bd0822b4df-workload-socket\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b\" (UID: \"3aee81c5-228b-4f21-b7d9-a0bd0822b4df\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b"
Oct 30 21:39:38.304825 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.304799    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3aee81c5-228b-4f21-b7d9-a0bd0822b4df-istio-data\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b\" (UID: \"3aee81c5-228b-4f21-b7d9-a0bd0822b4df\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b"
Oct 30 21:39:38.304881 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.304851    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3aee81c5-228b-4f21-b7d9-a0bd0822b4df-workload-certs\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b\" (UID: \"3aee81c5-228b-4f21-b7d9-a0bd0822b4df\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b"
Oct 30 21:39:38.305054 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.305034    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3aee81c5-228b-4f21-b7d9-a0bd0822b4df-credential-socket\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b\" (UID: \"3aee81c5-228b-4f21-b7d9-a0bd0822b4df\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b"
Oct 30 21:39:38.305096 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.305057    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3aee81c5-228b-4f21-b7d9-a0bd0822b4df-istio-data\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b\" (UID: \"3aee81c5-228b-4f21-b7d9-a0bd0822b4df\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b"
Oct 30 21:39:38.305309 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.305294    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3aee81c5-228b-4f21-b7d9-a0bd0822b4df-istiod-ca-cert\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b\" (UID: \"3aee81c5-228b-4f21-b7d9-a0bd0822b4df\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b"
Oct 30 21:39:38.306664 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.306637    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3aee81c5-228b-4f21-b7d9-a0bd0822b4df-istio-envoy\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b\" (UID: \"3aee81c5-228b-4f21-b7d9-a0bd0822b4df\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b"
Oct 30 21:39:38.306774 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.306733    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3aee81c5-228b-4f21-b7d9-a0bd0822b4df-istio-podinfo\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b\" (UID: \"3aee81c5-228b-4f21-b7d9-a0bd0822b4df\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b"
Oct 30 21:39:38.318226 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.318201    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3aee81c5-228b-4f21-b7d9-a0bd0822b4df-istio-token\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b\" (UID: \"3aee81c5-228b-4f21-b7d9-a0bd0822b4df\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b"
Oct 30 21:39:38.318373 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.318339    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5z877\" (UniqueName: \"kubernetes.io/projected/3aee81c5-228b-4f21-b7d9-a0bd0822b4df-kube-api-access-5z877\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b\" (UID: \"3aee81c5-228b-4f21-b7d9-a0bd0822b4df\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b"
Oct 30 21:39:38.435673 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.435588    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b"
Oct 30 21:39:38.551008 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.550972    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf"]
Oct 30 21:39:38.557369 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.557332    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf"
Oct 30 21:39:38.563224 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.563200    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-1d8034-openshift-default-dockercfg-hl29t\""
Oct 30 21:39:38.568260 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.567350    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf"]
Oct 30 21:39:38.610931 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.610894    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b720f88-2fd8-4c4d-a300-941b1a9db6e2-workload-socket\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf\" (UID: \"2b720f88-2fd8-4c4d-a300-941b1a9db6e2\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf"
Oct 30 21:39:38.610931 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.610934    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b720f88-2fd8-4c4d-a300-941b1a9db6e2-workload-certs\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf\" (UID: \"2b720f88-2fd8-4c4d-a300-941b1a9db6e2\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf"
Oct 30 21:39:38.611140 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.611052    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b720f88-2fd8-4c4d-a300-941b1a9db6e2-istiod-ca-cert\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf\" (UID: \"2b720f88-2fd8-4c4d-a300-941b1a9db6e2\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf"
Oct 30 21:39:38.611140 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.611088    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b720f88-2fd8-4c4d-a300-941b1a9db6e2-istio-envoy\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf\" (UID: \"2b720f88-2fd8-4c4d-a300-941b1a9db6e2\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf"
Oct 30 21:39:38.611140 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.611113    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b720f88-2fd8-4c4d-a300-941b1a9db6e2-istio-podinfo\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf\" (UID: \"2b720f88-2fd8-4c4d-a300-941b1a9db6e2\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf"
Oct 30 21:39:38.611265 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.611155    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnl2w\" (UniqueName: \"kubernetes.io/projected/2b720f88-2fd8-4c4d-a300-941b1a9db6e2-kube-api-access-vnl2w\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf\" (UID: \"2b720f88-2fd8-4c4d-a300-941b1a9db6e2\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf"
Oct 30 21:39:38.611265 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.611248    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b720f88-2fd8-4c4d-a300-941b1a9db6e2-istio-data\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf\" (UID: \"2b720f88-2fd8-4c4d-a300-941b1a9db6e2\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf"
Oct 30 21:39:38.611344 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.611272    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b720f88-2fd8-4c4d-a300-941b1a9db6e2-istio-token\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf\" (UID: \"2b720f88-2fd8-4c4d-a300-941b1a9db6e2\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf"
Oct 30 21:39:38.611418 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.611378    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b720f88-2fd8-4c4d-a300-941b1a9db6e2-credential-socket\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf\" (UID: \"2b720f88-2fd8-4c4d-a300-941b1a9db6e2\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf"
Oct 30 21:39:38.712456 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.712403    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b720f88-2fd8-4c4d-a300-941b1a9db6e2-workload-certs\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf\" (UID: \"2b720f88-2fd8-4c4d-a300-941b1a9db6e2\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf"
Oct 30 21:39:38.712626 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.712501    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b720f88-2fd8-4c4d-a300-941b1a9db6e2-istiod-ca-cert\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf\" (UID: \"2b720f88-2fd8-4c4d-a300-941b1a9db6e2\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf"
Oct 30 21:39:38.712626 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.712528    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b720f88-2fd8-4c4d-a300-941b1a9db6e2-istio-envoy\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf\" (UID: \"2b720f88-2fd8-4c4d-a300-941b1a9db6e2\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf"
Oct 30 21:39:38.712626 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.712560    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b720f88-2fd8-4c4d-a300-941b1a9db6e2-istio-podinfo\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf\" (UID: \"2b720f88-2fd8-4c4d-a300-941b1a9db6e2\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf"
Oct 30 21:39:38.712626 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.712579    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnl2w\" (UniqueName: \"kubernetes.io/projected/2b720f88-2fd8-4c4d-a300-941b1a9db6e2-kube-api-access-vnl2w\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf\" (UID: \"2b720f88-2fd8-4c4d-a300-941b1a9db6e2\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf"
Oct 30 21:39:38.712872 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.712643    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b720f88-2fd8-4c4d-a300-941b1a9db6e2-istio-data\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf\" (UID: \"2b720f88-2fd8-4c4d-a300-941b1a9db6e2\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf"
Oct 30 21:39:38.712872 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.712665    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b720f88-2fd8-4c4d-a300-941b1a9db6e2-istio-token\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf\" (UID: \"2b720f88-2fd8-4c4d-a300-941b1a9db6e2\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf"
Oct 30 21:39:38.712872 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.712738    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b720f88-2fd8-4c4d-a300-941b1a9db6e2-credential-socket\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf\" (UID: \"2b720f88-2fd8-4c4d-a300-941b1a9db6e2\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf"
Oct 30 21:39:38.712872 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.712785    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b720f88-2fd8-4c4d-a300-941b1a9db6e2-workload-socket\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf\" (UID: \"2b720f88-2fd8-4c4d-a300-941b1a9db6e2\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf"
Oct 30 21:39:38.713057 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.712875    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b720f88-2fd8-4c4d-a300-941b1a9db6e2-workload-certs\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf\" (UID: \"2b720f88-2fd8-4c4d-a300-941b1a9db6e2\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf"
Oct 30 21:39:38.713104 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.713065    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b720f88-2fd8-4c4d-a300-941b1a9db6e2-workload-socket\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf\" (UID: \"2b720f88-2fd8-4c4d-a300-941b1a9db6e2\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf"
Oct 30 21:39:38.713280 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.713256    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b720f88-2fd8-4c4d-a300-941b1a9db6e2-istio-data\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf\" (UID: \"2b720f88-2fd8-4c4d-a300-941b1a9db6e2\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf"
Oct 30 21:39:38.713592 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.713564    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b720f88-2fd8-4c4d-a300-941b1a9db6e2-credential-socket\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf\" (UID: \"2b720f88-2fd8-4c4d-a300-941b1a9db6e2\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf"
Oct 30 21:39:38.713849 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.713831    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b720f88-2fd8-4c4d-a300-941b1a9db6e2-istiod-ca-cert\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf\" (UID: \"2b720f88-2fd8-4c4d-a300-941b1a9db6e2\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf"
Oct 30 21:39:38.715711 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.715691    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b720f88-2fd8-4c4d-a300-941b1a9db6e2-istio-envoy\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf\" (UID: \"2b720f88-2fd8-4c4d-a300-941b1a9db6e2\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf"
Oct 30 21:39:38.717217 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.717195    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b720f88-2fd8-4c4d-a300-941b1a9db6e2-istio-podinfo\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf\" (UID: \"2b720f88-2fd8-4c4d-a300-941b1a9db6e2\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf"
Oct 30 21:39:38.734281 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.734258    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b"]
Oct 30 21:39:38.737287 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:39:38.737260    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3aee81c5_228b_4f21_b7d9_a0bd0822b4df.slice/crio-adbf88a579c49b10072b440671d3d3caf1de0688f39ccc1d349261f6b595dff9 WatchSource:0}: Error finding container adbf88a579c49b10072b440671d3d3caf1de0688f39ccc1d349261f6b595dff9: Status 404 returned error can't find the container with id adbf88a579c49b10072b440671d3d3caf1de0688f39ccc1d349261f6b595dff9
Oct 30 21:39:38.737398 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.737320    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnl2w\" (UniqueName: \"kubernetes.io/projected/2b720f88-2fd8-4c4d-a300-941b1a9db6e2-kube-api-access-vnl2w\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf\" (UID: \"2b720f88-2fd8-4c4d-a300-941b1a9db6e2\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf"
Oct 30 21:39:38.737555 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.737539    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b720f88-2fd8-4c4d-a300-941b1a9db6e2-istio-token\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf\" (UID: \"2b720f88-2fd8-4c4d-a300-941b1a9db6e2\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf"
Oct 30 21:39:38.739949 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.739913    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:39:38.740080 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.739995    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:39:38.740080 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.740038    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:39:38.757212 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.757188    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf"]
Oct 30 21:39:38.762638 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.762615    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf"
Oct 30 21:39:38.773342 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.773315    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-db0b54-openshift-default-dockercfg-ltx2s\""
Oct 30 21:39:38.777321 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.777298    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf"]
Oct 30 21:39:38.814238 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.814215    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0ca92b2-a242-47f5-9cc6-ee04860b5692-credential-socket\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf\" (UID: \"c0ca92b2-a242-47f5-9cc6-ee04860b5692\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf"
Oct 30 21:39:38.814320 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.814253    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lw9n\" (UniqueName: \"kubernetes.io/projected/c0ca92b2-a242-47f5-9cc6-ee04860b5692-kube-api-access-6lw9n\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf\" (UID: \"c0ca92b2-a242-47f5-9cc6-ee04860b5692\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf"
Oct 30 21:39:38.814320 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.814280    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0ca92b2-a242-47f5-9cc6-ee04860b5692-istiod-ca-cert\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf\" (UID: \"c0ca92b2-a242-47f5-9cc6-ee04860b5692\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf"
Oct 30 21:39:38.814473 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.814322    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0ca92b2-a242-47f5-9cc6-ee04860b5692-workload-socket\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf\" (UID: \"c0ca92b2-a242-47f5-9cc6-ee04860b5692\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf"
Oct 30 21:39:38.814473 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.814347    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0ca92b2-a242-47f5-9cc6-ee04860b5692-istio-token\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf\" (UID: \"c0ca92b2-a242-47f5-9cc6-ee04860b5692\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf"
Oct 30 21:39:38.814473 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.814412    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0ca92b2-a242-47f5-9cc6-ee04860b5692-workload-certs\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf\" (UID: \"c0ca92b2-a242-47f5-9cc6-ee04860b5692\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf"
Oct 30 21:39:38.814473 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.814445    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0ca92b2-a242-47f5-9cc6-ee04860b5692-istio-envoy\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf\" (UID: \"c0ca92b2-a242-47f5-9cc6-ee04860b5692\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf"
Oct 30 21:39:38.814473 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.814473    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0ca92b2-a242-47f5-9cc6-ee04860b5692-istio-podinfo\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf\" (UID: \"c0ca92b2-a242-47f5-9cc6-ee04860b5692\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf"
Oct 30 21:39:38.814728 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.814547    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0ca92b2-a242-47f5-9cc6-ee04860b5692-istio-data\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf\" (UID: \"c0ca92b2-a242-47f5-9cc6-ee04860b5692\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf"
Oct 30 21:39:38.874919 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.874888    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf"
Oct 30 21:39:38.921328 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.915667    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0ca92b2-a242-47f5-9cc6-ee04860b5692-istio-podinfo\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf\" (UID: \"c0ca92b2-a242-47f5-9cc6-ee04860b5692\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf"
Oct 30 21:39:38.921328 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.915802    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0ca92b2-a242-47f5-9cc6-ee04860b5692-istio-data\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf\" (UID: \"c0ca92b2-a242-47f5-9cc6-ee04860b5692\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf"
Oct 30 21:39:38.921328 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.916508    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0ca92b2-a242-47f5-9cc6-ee04860b5692-credential-socket\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf\" (UID: \"c0ca92b2-a242-47f5-9cc6-ee04860b5692\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf"
Oct 30 21:39:38.921328 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.916557    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lw9n\" (UniqueName: \"kubernetes.io/projected/c0ca92b2-a242-47f5-9cc6-ee04860b5692-kube-api-access-6lw9n\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf\" (UID: \"c0ca92b2-a242-47f5-9cc6-ee04860b5692\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf"
Oct 30 21:39:38.921328 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.916591    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0ca92b2-a242-47f5-9cc6-ee04860b5692-istiod-ca-cert\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf\" (UID: \"c0ca92b2-a242-47f5-9cc6-ee04860b5692\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf"
Oct 30 21:39:38.921328 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.916644    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0ca92b2-a242-47f5-9cc6-ee04860b5692-workload-socket\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf\" (UID: \"c0ca92b2-a242-47f5-9cc6-ee04860b5692\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf"
Oct 30 21:39:38.921328 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.916686    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0ca92b2-a242-47f5-9cc6-ee04860b5692-istio-token\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf\" (UID: \"c0ca92b2-a242-47f5-9cc6-ee04860b5692\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf"
Oct 30 21:39:38.921328 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.916717    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0ca92b2-a242-47f5-9cc6-ee04860b5692-workload-certs\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf\" (UID: \"c0ca92b2-a242-47f5-9cc6-ee04860b5692\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf"
Oct 30 21:39:38.921328 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.916779    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0ca92b2-a242-47f5-9cc6-ee04860b5692-istio-envoy\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf\" (UID: \"c0ca92b2-a242-47f5-9cc6-ee04860b5692\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf"
Oct 30 21:39:38.921328 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.918395    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0ca92b2-a242-47f5-9cc6-ee04860b5692-istiod-ca-cert\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf\" (UID: \"c0ca92b2-a242-47f5-9cc6-ee04860b5692\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf"
Oct 30 21:39:38.921328 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.918808    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0ca92b2-a242-47f5-9cc6-ee04860b5692-credential-socket\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf\" (UID: \"c0ca92b2-a242-47f5-9cc6-ee04860b5692\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf"
Oct 30 21:39:38.921328 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.919096    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0ca92b2-a242-47f5-9cc6-ee04860b5692-workload-socket\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf\" (UID: \"c0ca92b2-a242-47f5-9cc6-ee04860b5692\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf"
Oct 30 21:39:38.921328 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.919122    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0ca92b2-a242-47f5-9cc6-ee04860b5692-istio-data\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf\" (UID: \"c0ca92b2-a242-47f5-9cc6-ee04860b5692\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf"
Oct 30 21:39:38.921328 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.919349    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0ca92b2-a242-47f5-9cc6-ee04860b5692-workload-certs\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf\" (UID: \"c0ca92b2-a242-47f5-9cc6-ee04860b5692\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf"
Oct 30 21:39:38.922452 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.922347    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0ca92b2-a242-47f5-9cc6-ee04860b5692-istio-envoy\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf\" (UID: \"c0ca92b2-a242-47f5-9cc6-ee04860b5692\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf"
Oct 30 21:39:38.922569 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.922517    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0ca92b2-a242-47f5-9cc6-ee04860b5692-istio-podinfo\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf\" (UID: \"c0ca92b2-a242-47f5-9cc6-ee04860b5692\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf"
Oct 30 21:39:38.937323 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.937264    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0ca92b2-a242-47f5-9cc6-ee04860b5692-istio-token\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf\" (UID: \"c0ca92b2-a242-47f5-9cc6-ee04860b5692\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf"
Oct 30 21:39:38.937629 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:38.937613    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lw9n\" (UniqueName: \"kubernetes.io/projected/c0ca92b2-a242-47f5-9cc6-ee04860b5692-kube-api-access-6lw9n\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf\" (UID: \"c0ca92b2-a242-47f5-9cc6-ee04860b5692\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf"
Oct 30 21:39:39.105032 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:39.104948    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf"
Oct 30 21:39:39.176846 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:39.176814    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf"]
Oct 30 21:39:39.181175 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:39:39.181144    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b720f88_2fd8_4c4d_a300_941b1a9db6e2.slice/crio-ff65bcc6977427c5ca3e2a446c7694e9dc79a1d484fdb0f82a88c54cb8be1c8c WatchSource:0}: Error finding container ff65bcc6977427c5ca3e2a446c7694e9dc79a1d484fdb0f82a88c54cb8be1c8c: Status 404 returned error can't find the container with id ff65bcc6977427c5ca3e2a446c7694e9dc79a1d484fdb0f82a88c54cb8be1c8c
Oct 30 21:39:39.184897 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:39.184853    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:39:39.184989 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:39.184949    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:39:39.185060 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:39.184995    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:39:39.399907 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:39.399821    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf"]
Oct 30 21:39:39.403871 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:39:39.403836    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0ca92b2_a242_47f5_9cc6_ee04860b5692.slice/crio-acef7605eda87b82a8fe8d41e5fd6806a5a7fa187c0d345a515247215f483e8e WatchSource:0}: Error finding container acef7605eda87b82a8fe8d41e5fd6806a5a7fa187c0d345a515247215f483e8e: Status 404 returned error can't find the container with id acef7605eda87b82a8fe8d41e5fd6806a5a7fa187c0d345a515247215f483e8e
Oct 30 21:39:39.406935 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:39.406899    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:39:39.407017 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:39.406968    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:39:39.407017 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:39.406999    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:39:39.757505 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:39.757476    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf" event={"ID":"2b720f88-2fd8-4c4d-a300-941b1a9db6e2","Type":"ContainerStarted","Data":"dcd9676b4bf074fc246addfff9469b2223555476176e11d6aa22a488effc8ac7"}
Oct 30 21:39:39.757860 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:39.757840    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf" event={"ID":"2b720f88-2fd8-4c4d-a300-941b1a9db6e2","Type":"ContainerStarted","Data":"ff65bcc6977427c5ca3e2a446c7694e9dc79a1d484fdb0f82a88c54cb8be1c8c"}
Oct 30 21:39:39.758620 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:39.758599    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf" event={"ID":"c0ca92b2-a242-47f5-9cc6-ee04860b5692","Type":"ContainerStarted","Data":"b2b97753a9fba276652fbba6d1514744ba74cfd3f791c70e1e8b7d7ec5c60cee"}
Oct 30 21:39:39.758756 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:39.758743    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf" event={"ID":"c0ca92b2-a242-47f5-9cc6-ee04860b5692","Type":"ContainerStarted","Data":"acef7605eda87b82a8fe8d41e5fd6806a5a7fa187c0d345a515247215f483e8e"}
Oct 30 21:39:39.762090 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:39.762070    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b" event={"ID":"3aee81c5-228b-4f21-b7d9-a0bd0822b4df","Type":"ContainerStarted","Data":"31706ca5ffc512e69734a1d499097c7cd0a124432df0d948d2d5ef41d7a406ad"}
Oct 30 21:39:39.762204 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:39.762193    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b" event={"ID":"3aee81c5-228b-4f21-b7d9-a0bd0822b4df","Type":"ContainerStarted","Data":"adbf88a579c49b10072b440671d3d3caf1de0688f39ccc1d349261f6b595dff9"}
Oct 30 21:39:39.783869 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:39.783811    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf" podStartSLOduration=1.783797997 podStartE2EDuration="1.783797997s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:39:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:39:39.781704252 +0000 UTC m=+692.583516730" watchObservedRunningTime="2026-10-30 21:39:39.783797997 +0000 UTC m=+692.585610460"
Oct 30 21:39:39.806598 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:39.806531    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf" podStartSLOduration=1.80650957 podStartE2EDuration="1.80650957s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:39:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:39:39.805141065 +0000 UTC m=+692.606953525" watchObservedRunningTime="2026-10-30 21:39:39.80650957 +0000 UTC m=+692.608322037"
Oct 30 21:39:39.876383 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:39.875816    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf"
Oct 30 21:39:39.879575 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:39.879510    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b" podStartSLOduration=1.8794929470000001 podStartE2EDuration="1.879492947s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:39:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:39:39.876224736 +0000 UTC m=+692.678037202" watchObservedRunningTime="2026-10-30 21:39:39.879492947 +0000 UTC m=+692.681305410"
Oct 30 21:39:40.106137 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:40.106048    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf"
Oct 30 21:39:40.106974 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:40.106949    5032 patch_prober.go:29] interesting pod/e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Oct 30 21:39:40.107078 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:40.107022    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf" podUID="c0ca92b2-a242-47f5-9cc6-ee04860b5692" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Oct 30 21:39:40.439459 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:40.439232    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b"
Oct 30 21:39:40.439459 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:40.439389    5032 patch_prober.go:29] interesting pod/e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Oct 30 21:39:40.440019 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:40.439455    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b" podUID="3aee81c5-228b-4f21-b7d9-a0bd0822b4df" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Oct 30 21:39:40.879387 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:40.878709    5032 patch_prober.go:29] interesting pod/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Oct 30 21:39:40.879387 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:40.878769    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf" podUID="2b720f88-2fd8-4c4d-a300-941b1a9db6e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Oct 30 21:39:41.106205 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:41.106169    5032 patch_prober.go:29] interesting pod/e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Oct 30 21:39:41.106355 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:41.106220    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf" podUID="c0ca92b2-a242-47f5-9cc6-ee04860b5692" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Oct 30 21:39:41.443388 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:41.442489    5032 patch_prober.go:29] interesting pod/e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Oct 30 21:39:41.443388 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:41.442558    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b" podUID="3aee81c5-228b-4f21-b7d9-a0bd0822b4df" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Oct 30 21:39:41.889339 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:41.888767    5032 patch_prober.go:29] interesting pod/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Oct 30 21:39:41.889339 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:41.888864    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf" podUID="2b720f88-2fd8-4c4d-a300-941b1a9db6e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Oct 30 21:39:42.106767 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:42.106734    5032 patch_prober.go:29] interesting pod/e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Oct 30 21:39:42.106913 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:42.106783    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf" podUID="c0ca92b2-a242-47f5-9cc6-ee04860b5692" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Oct 30 21:39:42.439384 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:42.438737    5032 patch_prober.go:29] interesting pod/e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Oct 30 21:39:42.439384 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:42.438816    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b" podUID="3aee81c5-228b-4f21-b7d9-a0bd0822b4df" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Oct 30 21:39:42.889350 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:42.878067    5032 patch_prober.go:29] interesting pod/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Oct 30 21:39:42.889350 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:42.878248    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf" podUID="2b720f88-2fd8-4c4d-a300-941b1a9db6e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Oct 30 21:39:43.106601 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:43.106385    5032 patch_prober.go:29] interesting pod/e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Oct 30 21:39:43.106601 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:43.106455    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf" podUID="c0ca92b2-a242-47f5-9cc6-ee04860b5692" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Oct 30 21:39:43.398174 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:43.398142    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 21:39:43.441822 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:43.441784    5032 patch_prober.go:29] interesting pod/e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Oct 30 21:39:43.443883 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:43.443846    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b" podUID="3aee81c5-228b-4f21-b7d9-a0bd0822b4df" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Oct 30 21:39:43.886767 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:43.885293    5032 patch_prober.go:29] interesting pod/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Oct 30 21:39:43.886767 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:43.885354    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf" podUID="2b720f88-2fd8-4c4d-a300-941b1a9db6e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Oct 30 21:39:44.098400 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:44.097795    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj"
Oct 30 21:39:44.105690 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:44.105641    5032 patch_prober.go:29] interesting pod/e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Oct 30 21:39:44.105883 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:44.105703    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf" podUID="c0ca92b2-a242-47f5-9cc6-ee04860b5692" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Oct 30 21:39:44.165936 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:44.165841    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj"
Oct 30 21:39:44.309469 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:44.309425    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw"]
Oct 30 21:39:44.556392 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:44.531666    5032 patch_prober.go:29] interesting pod/e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Oct 30 21:39:44.721132 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:44.720498    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b" podUID="3aee81c5-228b-4f21-b7d9-a0bd0822b4df" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Oct 30 21:39:44.906431 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:44.902305    5032 patch_prober.go:29] interesting pod/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Oct 30 21:39:44.954524 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:44.954471    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf" podUID="2b720f88-2fd8-4c4d-a300-941b1a9db6e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Oct 30 21:39:44.984177 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:44.984143    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw"
Oct 30 21:39:45.008707 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:45.008428    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-7dfff0\"/\"distinct-w6-7dfff0-kserve-self-signed-certs\""
Oct 30 21:39:45.034500 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:45.034469    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-7dfff0\"/\"kube-root-ca.crt\""
Oct 30 21:39:45.052697 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:45.034884    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-7dfff0\"/\"openshift-service-ca.crt\""
Oct 30 21:39:45.077130 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:45.077093    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw"]
Oct 30 21:39:45.101761 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:45.101731    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-7dfff0\"/\"default-dockercfg-xldhw\""
Oct 30 21:39:45.157678 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:45.113591    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c-dshm\") pod \"distinct-w6-7dfff0-kserve-674b4cf84b-qsclw\" (UID: \"2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c\") " pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw"
Oct 30 21:39:45.161509 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:45.127676    5032 patch_prober.go:29] interesting pod/e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Oct 30 21:39:45.161729 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:45.161687    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf" podUID="c0ca92b2-a242-47f5-9cc6-ee04860b5692" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Oct 30 21:39:45.186172 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:45.186127    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c-tls-certs\") pod \"distinct-w6-7dfff0-kserve-674b4cf84b-qsclw\" (UID: \"2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c\") " pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw"
Oct 30 21:39:45.186572 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:45.186546    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c-tmp-dir\") pod \"distinct-w6-7dfff0-kserve-674b4cf84b-qsclw\" (UID: \"2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c\") " pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw"
Oct 30 21:39:45.186906 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:45.186884    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c-home\") pod \"distinct-w6-7dfff0-kserve-674b4cf84b-qsclw\" (UID: \"2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c\") " pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw"
Oct 30 21:39:45.187097 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:45.187071    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hd2pl\" (UniqueName: \"kubernetes.io/projected/2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c-kube-api-access-hd2pl\") pod \"distinct-w6-7dfff0-kserve-674b4cf84b-qsclw\" (UID: \"2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c\") " pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw"
Oct 30 21:39:45.188643 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:45.188622    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c-model-cache\") pod \"distinct-w6-7dfff0-kserve-674b4cf84b-qsclw\" (UID: \"2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c\") " pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw"
Oct 30 21:39:45.292154 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:45.292109    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c-tmp-dir\") pod \"distinct-w6-7dfff0-kserve-674b4cf84b-qsclw\" (UID: \"2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c\") " pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw"
Oct 30 21:39:45.292987 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:45.292964    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c-home\") pod \"distinct-w6-7dfff0-kserve-674b4cf84b-qsclw\" (UID: \"2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c\") " pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw"
Oct 30 21:39:45.293173 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:45.293154    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hd2pl\" (UniqueName: \"kubernetes.io/projected/2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c-kube-api-access-hd2pl\") pod \"distinct-w6-7dfff0-kserve-674b4cf84b-qsclw\" (UID: \"2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c\") " pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw"
Oct 30 21:39:45.293348 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:45.293333    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c-model-cache\") pod \"distinct-w6-7dfff0-kserve-674b4cf84b-qsclw\" (UID: \"2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c\") " pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw"
Oct 30 21:39:45.293510 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:45.293495    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c-dshm\") pod \"distinct-w6-7dfff0-kserve-674b4cf84b-qsclw\" (UID: \"2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c\") " pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw"
Oct 30 21:39:45.293623 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:45.293609    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c-tls-certs\") pod \"distinct-w6-7dfff0-kserve-674b4cf84b-qsclw\" (UID: \"2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c\") " pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw"
Oct 30 21:39:45.296827 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:45.296801    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c-model-cache\") pod \"distinct-w6-7dfff0-kserve-674b4cf84b-qsclw\" (UID: \"2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c\") " pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw"
Oct 30 21:39:45.296982 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:45.292840    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c-tmp-dir\") pod \"distinct-w6-7dfff0-kserve-674b4cf84b-qsclw\" (UID: \"2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c\") " pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw"
Oct 30 21:39:45.297857 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:45.297826    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c-home\") pod \"distinct-w6-7dfff0-kserve-674b4cf84b-qsclw\" (UID: \"2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c\") " pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw"
Oct 30 21:39:45.323961 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:45.323879    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c-dshm\") pod \"distinct-w6-7dfff0-kserve-674b4cf84b-qsclw\" (UID: \"2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c\") " pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw"
Oct 30 21:39:45.380383 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:45.370399    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c-tls-certs\") pod \"distinct-w6-7dfff0-kserve-674b4cf84b-qsclw\" (UID: \"2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c\") " pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw"
Oct 30 21:39:45.380383 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:45.378909    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hd2pl\" (UniqueName: \"kubernetes.io/projected/2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c-kube-api-access-hd2pl\") pod \"distinct-w6-7dfff0-kserve-674b4cf84b-qsclw\" (UID: \"2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c\") " pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw"
Oct 30 21:39:45.445002 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:45.444472    5032 patch_prober.go:29] interesting pod/e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Oct 30 21:39:45.445002 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:45.444551    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b" podUID="3aee81c5-228b-4f21-b7d9-a0bd0822b4df" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Oct 30 21:39:45.571043 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:45.571009    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw"
Oct 30 21:39:45.657923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:45.628895    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj"
Oct 30 21:39:45.921386 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:45.883036    5032 patch_prober.go:29] interesting pod/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Oct 30 21:39:45.921386 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:45.883128    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf" podUID="2b720f88-2fd8-4c4d-a300-941b1a9db6e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Oct 30 21:39:46.106592 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:46.105711    5032 patch_prober.go:29] interesting pod/e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Oct 30 21:39:46.106592 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:46.105768    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf" podUID="c0ca92b2-a242-47f5-9cc6-ee04860b5692" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Oct 30 21:39:46.553506 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:46.553474    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw"]
Oct 30 21:39:46.557466 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:39:46.557416    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2ce9b0b7_ff4f_4bd4_ab29_18e2b340c50c.slice/crio-df140a9c931051d41094f139b02ca4a937a7ea41fa55f227332720391ecd1280 WatchSource:0}: Error finding container df140a9c931051d41094f139b02ca4a937a7ea41fa55f227332720391ecd1280: Status 404 returned error can't find the container with id df140a9c931051d41094f139b02ca4a937a7ea41fa55f227332720391ecd1280
Oct 30 21:39:46.875645 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:46.875556    5032 patch_prober.go:29] interesting pod/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Oct 30 21:39:46.875645 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:46.875626    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf" podUID="2b720f88-2fd8-4c4d-a300-941b1a9db6e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Oct 30 21:39:47.108005 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:47.106173    5032 patch_prober.go:29] interesting pod/e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Oct 30 21:39:47.108005 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:47.106236    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf" podUID="c0ca92b2-a242-47f5-9cc6-ee04860b5692" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Oct 30 21:39:47.224455 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:47.222556    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw" event={"ID":"2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c","Type":"ContainerStarted","Data":"51dc5925f7d3db9b4cf18e7e05f841ea126c46e10ceb6d77d137207f2c52ae78"}
Oct 30 21:39:47.224455 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:47.222613    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw" event={"ID":"2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c","Type":"ContainerStarted","Data":"df140a9c931051d41094f139b02ca4a937a7ea41fa55f227332720391ecd1280"}
Oct 30 21:39:47.261385 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:47.257745    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw" podStartSLOduration=3.257708793 podStartE2EDuration="3.257708793s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:39:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:39:47.252328145 +0000 UTC m=+700.054140621" watchObservedRunningTime="2026-10-30 21:39:47.257708793 +0000 UTC m=+700.059521259"
Oct 30 21:39:47.496287 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:47.485467    5032 patch_prober.go:29] interesting pod/e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:39:47.496287 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:47.485531    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b" podUID="3aee81c5-228b-4f21-b7d9-a0bd0822b4df" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:39:47.496287 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:47.492324    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc"]
Oct 30 21:39:47.502276 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:47.502245    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc"
Oct 30 21:39:47.504002 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:47.503544    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc"]
Oct 30 21:39:47.544039 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:47.543965    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-1d8034\"/\"distinct-2-w3-1d8034-kserve-self-signed-certs\""
Oct 30 21:39:47.571888 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:47.571832    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw"
Oct 30 21:39:47.590601 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:47.590573    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/23b97dbb-8488-4c44-a810-882ea18ed919-tls-certs\") pod \"distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc\" (UID: \"23b97dbb-8488-4c44-a810-882ea18ed919\") " pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc"
Oct 30 21:39:47.590821 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:47.590803    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/23b97dbb-8488-4c44-a810-882ea18ed919-home\") pod \"distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc\" (UID: \"23b97dbb-8488-4c44-a810-882ea18ed919\") " pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc"
Oct 30 21:39:47.590964 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:47.590950    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/23b97dbb-8488-4c44-a810-882ea18ed919-model-cache\") pod \"distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc\" (UID: \"23b97dbb-8488-4c44-a810-882ea18ed919\") " pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc"
Oct 30 21:39:47.591050 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:47.591040    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/23b97dbb-8488-4c44-a810-882ea18ed919-tmp-dir\") pod \"distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc\" (UID: \"23b97dbb-8488-4c44-a810-882ea18ed919\") " pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc"
Oct 30 21:39:47.591141 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:47.591126    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzpfw\" (UniqueName: \"kubernetes.io/projected/23b97dbb-8488-4c44-a810-882ea18ed919-kube-api-access-bzpfw\") pod \"distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc\" (UID: \"23b97dbb-8488-4c44-a810-882ea18ed919\") " pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc"
Oct 30 21:39:47.591302 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:47.591285    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/23b97dbb-8488-4c44-a810-882ea18ed919-dshm\") pod \"distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc\" (UID: \"23b97dbb-8488-4c44-a810-882ea18ed919\") " pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc"
Oct 30 21:39:47.692670 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:47.692428    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/23b97dbb-8488-4c44-a810-882ea18ed919-tls-certs\") pod \"distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc\" (UID: \"23b97dbb-8488-4c44-a810-882ea18ed919\") " pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc"
Oct 30 21:39:47.692814 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:47.692719    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/23b97dbb-8488-4c44-a810-882ea18ed919-home\") pod \"distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc\" (UID: \"23b97dbb-8488-4c44-a810-882ea18ed919\") " pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc"
Oct 30 21:39:47.693005 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:47.692983    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/23b97dbb-8488-4c44-a810-882ea18ed919-model-cache\") pod \"distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc\" (UID: \"23b97dbb-8488-4c44-a810-882ea18ed919\") " pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc"
Oct 30 21:39:47.693082 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:47.693067    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/23b97dbb-8488-4c44-a810-882ea18ed919-tmp-dir\") pod \"distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc\" (UID: \"23b97dbb-8488-4c44-a810-882ea18ed919\") " pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc"
Oct 30 21:39:47.693123 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:47.693097    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzpfw\" (UniqueName: \"kubernetes.io/projected/23b97dbb-8488-4c44-a810-882ea18ed919-kube-api-access-bzpfw\") pod \"distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc\" (UID: \"23b97dbb-8488-4c44-a810-882ea18ed919\") " pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc"
Oct 30 21:39:47.693221 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:47.693199    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/23b97dbb-8488-4c44-a810-882ea18ed919-dshm\") pod \"distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc\" (UID: \"23b97dbb-8488-4c44-a810-882ea18ed919\") " pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc"
Oct 30 21:39:47.693274 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:47.693238    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/23b97dbb-8488-4c44-a810-882ea18ed919-home\") pod \"distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc\" (UID: \"23b97dbb-8488-4c44-a810-882ea18ed919\") " pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc"
Oct 30 21:39:47.693746 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:47.693575    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/23b97dbb-8488-4c44-a810-882ea18ed919-model-cache\") pod \"distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc\" (UID: \"23b97dbb-8488-4c44-a810-882ea18ed919\") " pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc"
Oct 30 21:39:47.694101 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:47.694087    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/23b97dbb-8488-4c44-a810-882ea18ed919-tmp-dir\") pod \"distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc\" (UID: \"23b97dbb-8488-4c44-a810-882ea18ed919\") " pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc"
Oct 30 21:39:47.698404 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:47.696097    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/23b97dbb-8488-4c44-a810-882ea18ed919-tls-certs\") pod \"distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc\" (UID: \"23b97dbb-8488-4c44-a810-882ea18ed919\") " pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc"
Oct 30 21:39:47.700914 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:47.700701    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/23b97dbb-8488-4c44-a810-882ea18ed919-dshm\") pod \"distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc\" (UID: \"23b97dbb-8488-4c44-a810-882ea18ed919\") " pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc"
Oct 30 21:39:47.714131 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:47.714097    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzpfw\" (UniqueName: \"kubernetes.io/projected/23b97dbb-8488-4c44-a810-882ea18ed919-kube-api-access-bzpfw\") pod \"distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc\" (UID: \"23b97dbb-8488-4c44-a810-882ea18ed919\") " pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc"
Oct 30 21:39:47.770782 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:47.770700    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b"
Oct 30 21:39:47.771809 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:47.771785    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b"
Oct 30 21:39:47.772263 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:47.772241    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b"
Oct 30 21:39:47.873252 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:47.873217    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc"
Oct 30 21:39:47.892528 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:47.888732    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf"
Oct 30 21:39:47.892528 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:47.890289    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf"
Oct 30 21:39:47.892528 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:47.891448    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf"
Oct 30 21:39:48.191673 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:48.191642    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc"]
Oct 30 21:39:48.192953 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:39:48.192902    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod23b97dbb_8488_4c44_a810_882ea18ed919.slice/crio-4a379b2784d148084fae9bc331b46d462af255ccb510895bf36543e1f844579f WatchSource:0}: Error finding container 4a379b2784d148084fae9bc331b46d462af255ccb510895bf36543e1f844579f: Status 404 returned error can't find the container with id 4a379b2784d148084fae9bc331b46d462af255ccb510895bf36543e1f844579f
Oct 30 21:39:48.234101 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:48.234056    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc" event={"ID":"23b97dbb-8488-4c44-a810-882ea18ed919","Type":"ContainerStarted","Data":"4a379b2784d148084fae9bc331b46d462af255ccb510895bf36543e1f844579f"}
Oct 30 21:39:48.357395 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:39:48.355971    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 21:39:48.498596 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:48.498556    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf"
Oct 30 21:39:48.499450 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:48.499423    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf"
Oct 30 21:39:48.500354 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:48.500330    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf"
Oct 30 21:39:49.243182 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:49.243144    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc" event={"ID":"23b97dbb-8488-4c44-a810-882ea18ed919","Type":"ContainerStarted","Data":"55fec9098ae03c756d258df3c3d2d75a5db97bfdeaad74957831d75d1af4449b"}
Oct 30 21:39:49.266448 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:49.266395    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc" podStartSLOduration=2.266382024 podStartE2EDuration="2.266382024s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:39:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:39:49.26287101 +0000 UTC m=+702.064683489" watchObservedRunningTime="2026-10-30 21:39:49.266382024 +0000 UTC m=+702.068194517"
Oct 30 21:39:49.873767 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:49.873741    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc"
Oct 30 21:39:53.521752 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:53.521714    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2"]
Oct 30 21:39:53.526393 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:53.526355    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2"
Oct 30 21:39:53.539959 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:53.539932    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2"]
Oct 30 21:39:53.540968 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:53.540945    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w6-7dfff0-openshift-default-dockercfg-lrx4l\""
Oct 30 21:39:53.668956 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:53.668917    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62000861-c573-4dba-8bca-c6f962057f5b-istio-data\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2\" (UID: \"62000861-c573-4dba-8bca-c6f962057f5b\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2"
Oct 30 21:39:53.669130 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:53.668982    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62000861-c573-4dba-8bca-c6f962057f5b-workload-certs\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2\" (UID: \"62000861-c573-4dba-8bca-c6f962057f5b\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2"
Oct 30 21:39:53.669130 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:53.669037    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62000861-c573-4dba-8bca-c6f962057f5b-istio-podinfo\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2\" (UID: \"62000861-c573-4dba-8bca-c6f962057f5b\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2"
Oct 30 21:39:53.669130 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:53.669125    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62000861-c573-4dba-8bca-c6f962057f5b-credential-socket\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2\" (UID: \"62000861-c573-4dba-8bca-c6f962057f5b\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2"
Oct 30 21:39:53.669266 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:53.669227    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62000861-c573-4dba-8bca-c6f962057f5b-istio-envoy\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2\" (UID: \"62000861-c573-4dba-8bca-c6f962057f5b\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2"
Oct 30 21:39:53.669317 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:53.669304    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62000861-c573-4dba-8bca-c6f962057f5b-istiod-ca-cert\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2\" (UID: \"62000861-c573-4dba-8bca-c6f962057f5b\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2"
Oct 30 21:39:53.669354 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:53.669330    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62000861-c573-4dba-8bca-c6f962057f5b-istio-token\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2\" (UID: \"62000861-c573-4dba-8bca-c6f962057f5b\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2"
Oct 30 21:39:53.669456 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:53.669440    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62000861-c573-4dba-8bca-c6f962057f5b-workload-socket\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2\" (UID: \"62000861-c573-4dba-8bca-c6f962057f5b\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2"
Oct 30 21:39:53.669490 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:53.669467    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbcgk\" (UniqueName: \"kubernetes.io/projected/62000861-c573-4dba-8bca-c6f962057f5b-kube-api-access-rbcgk\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2\" (UID: \"62000861-c573-4dba-8bca-c6f962057f5b\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2"
Oct 30 21:39:53.703303 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:53.703264    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76"]
Oct 30 21:39:53.708946 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:53.708828    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76"
Oct 30 21:39:53.727486 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:53.727454    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76"]
Oct 30 21:39:53.770945 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:53.770906    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd4228de-c71f-4f88-a54e-9ebda8b91122-istio-token\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76\" (UID: \"cd4228de-c71f-4f88-a54e-9ebda8b91122\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76"
Oct 30 21:39:53.771142 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:53.770982    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd4228de-c71f-4f88-a54e-9ebda8b91122-istio-envoy\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76\" (UID: \"cd4228de-c71f-4f88-a54e-9ebda8b91122\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76"
Oct 30 21:39:53.771142 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:53.771094    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd4228de-c71f-4f88-a54e-9ebda8b91122-istiod-ca-cert\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76\" (UID: \"cd4228de-c71f-4f88-a54e-9ebda8b91122\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76"
Oct 30 21:39:53.771256 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:53.771142    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62000861-c573-4dba-8bca-c6f962057f5b-istio-envoy\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2\" (UID: \"62000861-c573-4dba-8bca-c6f962057f5b\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2"
Oct 30 21:39:53.771256 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:53.771172    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8v9qm\" (UniqueName: \"kubernetes.io/projected/cd4228de-c71f-4f88-a54e-9ebda8b91122-kube-api-access-8v9qm\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76\" (UID: \"cd4228de-c71f-4f88-a54e-9ebda8b91122\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76"
Oct 30 21:39:53.771395 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:53.771256    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd4228de-c71f-4f88-a54e-9ebda8b91122-istio-podinfo\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76\" (UID: \"cd4228de-c71f-4f88-a54e-9ebda8b91122\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76"
Oct 30 21:39:53.771395 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:53.771284    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd4228de-c71f-4f88-a54e-9ebda8b91122-credential-socket\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76\" (UID: \"cd4228de-c71f-4f88-a54e-9ebda8b91122\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76"
Oct 30 21:39:53.771395 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:53.771313    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd4228de-c71f-4f88-a54e-9ebda8b91122-workload-certs\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76\" (UID: \"cd4228de-c71f-4f88-a54e-9ebda8b91122\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76"
Oct 30 21:39:53.771395 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:53.771343    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62000861-c573-4dba-8bca-c6f962057f5b-istiod-ca-cert\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2\" (UID: \"62000861-c573-4dba-8bca-c6f962057f5b\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2"
Oct 30 21:39:53.771709 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:53.771420    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62000861-c573-4dba-8bca-c6f962057f5b-istio-token\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2\" (UID: \"62000861-c573-4dba-8bca-c6f962057f5b\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2"
Oct 30 21:39:53.771709 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:53.771486    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62000861-c573-4dba-8bca-c6f962057f5b-workload-socket\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2\" (UID: \"62000861-c573-4dba-8bca-c6f962057f5b\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2"
Oct 30 21:39:53.771709 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:53.771511    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbcgk\" (UniqueName: \"kubernetes.io/projected/62000861-c573-4dba-8bca-c6f962057f5b-kube-api-access-rbcgk\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2\" (UID: \"62000861-c573-4dba-8bca-c6f962057f5b\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2"
Oct 30 21:39:53.771709 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:53.771569    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62000861-c573-4dba-8bca-c6f962057f5b-istio-data\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2\" (UID: \"62000861-c573-4dba-8bca-c6f962057f5b\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2"
Oct 30 21:39:53.771709 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:53.771607    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd4228de-c71f-4f88-a54e-9ebda8b91122-istio-data\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76\" (UID: \"cd4228de-c71f-4f88-a54e-9ebda8b91122\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76"
Oct 30 21:39:53.771709 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:53.771669    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62000861-c573-4dba-8bca-c6f962057f5b-workload-certs\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2\" (UID: \"62000861-c573-4dba-8bca-c6f962057f5b\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2"
Oct 30 21:39:53.771709 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:53.771692    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62000861-c573-4dba-8bca-c6f962057f5b-istio-podinfo\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2\" (UID: \"62000861-c573-4dba-8bca-c6f962057f5b\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2"
Oct 30 21:39:53.772095 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:53.771723    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd4228de-c71f-4f88-a54e-9ebda8b91122-workload-socket\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76\" (UID: \"cd4228de-c71f-4f88-a54e-9ebda8b91122\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76"
Oct 30 21:39:53.772095 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:53.771783    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62000861-c573-4dba-8bca-c6f962057f5b-credential-socket\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2\" (UID: \"62000861-c573-4dba-8bca-c6f962057f5b\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2"
Oct 30 21:39:53.772393 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:53.772337    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62000861-c573-4dba-8bca-c6f962057f5b-credential-socket\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2\" (UID: \"62000861-c573-4dba-8bca-c6f962057f5b\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2"
Oct 30 21:39:53.772975 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:53.772952    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62000861-c573-4dba-8bca-c6f962057f5b-workload-socket\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2\" (UID: \"62000861-c573-4dba-8bca-c6f962057f5b\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2"
Oct 30 21:39:53.773559 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:53.773529    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62000861-c573-4dba-8bca-c6f962057f5b-istiod-ca-cert\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2\" (UID: \"62000861-c573-4dba-8bca-c6f962057f5b\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2"
Oct 30 21:39:53.773700 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:53.773566    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62000861-c573-4dba-8bca-c6f962057f5b-istio-data\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2\" (UID: \"62000861-c573-4dba-8bca-c6f962057f5b\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2"
Oct 30 21:39:53.773822 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:53.773795    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62000861-c573-4dba-8bca-c6f962057f5b-workload-certs\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2\" (UID: \"62000861-c573-4dba-8bca-c6f962057f5b\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2"
Oct 30 21:39:53.775428 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:53.775344    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62000861-c573-4dba-8bca-c6f962057f5b-istio-envoy\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2\" (UID: \"62000861-c573-4dba-8bca-c6f962057f5b\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2"
Oct 30 21:39:53.776610 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:53.776586    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62000861-c573-4dba-8bca-c6f962057f5b-istio-podinfo\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2\" (UID: \"62000861-c573-4dba-8bca-c6f962057f5b\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2"
Oct 30 21:39:53.792267 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:53.792244    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62000861-c573-4dba-8bca-c6f962057f5b-istio-token\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2\" (UID: \"62000861-c573-4dba-8bca-c6f962057f5b\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2"
Oct 30 21:39:53.792932 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:53.792887    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbcgk\" (UniqueName: \"kubernetes.io/projected/62000861-c573-4dba-8bca-c6f962057f5b-kube-api-access-rbcgk\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2\" (UID: \"62000861-c573-4dba-8bca-c6f962057f5b\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2"
Oct 30 21:39:53.844252 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:53.844213    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2"
Oct 30 21:39:53.873625 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:53.873592    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd4228de-c71f-4f88-a54e-9ebda8b91122-istio-envoy\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76\" (UID: \"cd4228de-c71f-4f88-a54e-9ebda8b91122\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76"
Oct 30 21:39:53.873761 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:53.873647    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd4228de-c71f-4f88-a54e-9ebda8b91122-istiod-ca-cert\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76\" (UID: \"cd4228de-c71f-4f88-a54e-9ebda8b91122\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76"
Oct 30 21:39:53.873761 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:53.873688    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8v9qm\" (UniqueName: \"kubernetes.io/projected/cd4228de-c71f-4f88-a54e-9ebda8b91122-kube-api-access-8v9qm\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76\" (UID: \"cd4228de-c71f-4f88-a54e-9ebda8b91122\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76"
Oct 30 21:39:53.873893 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:53.873783    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd4228de-c71f-4f88-a54e-9ebda8b91122-istio-podinfo\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76\" (UID: \"cd4228de-c71f-4f88-a54e-9ebda8b91122\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76"
Oct 30 21:39:53.873893 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:53.873814    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd4228de-c71f-4f88-a54e-9ebda8b91122-credential-socket\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76\" (UID: \"cd4228de-c71f-4f88-a54e-9ebda8b91122\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76"
Oct 30 21:39:53.873893 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:53.873837    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd4228de-c71f-4f88-a54e-9ebda8b91122-workload-certs\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76\" (UID: \"cd4228de-c71f-4f88-a54e-9ebda8b91122\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76"
Oct 30 21:39:53.874052 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:53.873948    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd4228de-c71f-4f88-a54e-9ebda8b91122-istio-data\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76\" (UID: \"cd4228de-c71f-4f88-a54e-9ebda8b91122\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76"
Oct 30 21:39:53.874052 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:53.874012    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd4228de-c71f-4f88-a54e-9ebda8b91122-workload-socket\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76\" (UID: \"cd4228de-c71f-4f88-a54e-9ebda8b91122\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76"
Oct 30 21:39:53.874164 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:53.874110    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd4228de-c71f-4f88-a54e-9ebda8b91122-istio-token\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76\" (UID: \"cd4228de-c71f-4f88-a54e-9ebda8b91122\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76"
Oct 30 21:39:53.874384 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:53.874330    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd4228de-c71f-4f88-a54e-9ebda8b91122-workload-certs\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76\" (UID: \"cd4228de-c71f-4f88-a54e-9ebda8b91122\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76"
Oct 30 21:39:53.874512 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:53.874467    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd4228de-c71f-4f88-a54e-9ebda8b91122-istiod-ca-cert\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76\" (UID: \"cd4228de-c71f-4f88-a54e-9ebda8b91122\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76"
Oct 30 21:39:53.874826 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:53.874795    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd4228de-c71f-4f88-a54e-9ebda8b91122-istio-data\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76\" (UID: \"cd4228de-c71f-4f88-a54e-9ebda8b91122\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76"
Oct 30 21:39:53.874826 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:53.874813    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd4228de-c71f-4f88-a54e-9ebda8b91122-workload-socket\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76\" (UID: \"cd4228de-c71f-4f88-a54e-9ebda8b91122\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76"
Oct 30 21:39:53.875075 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:53.875043    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd4228de-c71f-4f88-a54e-9ebda8b91122-credential-socket\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76\" (UID: \"cd4228de-c71f-4f88-a54e-9ebda8b91122\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76"
Oct 30 21:39:53.876239 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:53.876212    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd4228de-c71f-4f88-a54e-9ebda8b91122-istio-envoy\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76\" (UID: \"cd4228de-c71f-4f88-a54e-9ebda8b91122\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76"
Oct 30 21:39:53.877058 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:53.877036    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd4228de-c71f-4f88-a54e-9ebda8b91122-istio-podinfo\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76\" (UID: \"cd4228de-c71f-4f88-a54e-9ebda8b91122\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76"
Oct 30 21:39:53.888096 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:53.888060    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd4228de-c71f-4f88-a54e-9ebda8b91122-istio-token\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76\" (UID: \"cd4228de-c71f-4f88-a54e-9ebda8b91122\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76"
Oct 30 21:39:53.888229 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:53.888218    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8v9qm\" (UniqueName: \"kubernetes.io/projected/cd4228de-c71f-4f88-a54e-9ebda8b91122-kube-api-access-8v9qm\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76\" (UID: \"cd4228de-c71f-4f88-a54e-9ebda8b91122\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76"
Oct 30 21:39:53.994020 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:53.993982    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h"]
Oct 30 21:39:53.999413 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:53.999391    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h"
Oct 30 21:39:54.006644 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:54.006530    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-000f64-openshift-default-dockercfg-8l2xx\""
Oct 30 21:39:54.018273 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:54.018238    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h"]
Oct 30 21:39:54.028212 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:54.028128    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76"
Oct 30 21:39:54.076764 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:54.076726    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb5d58b6-89f9-4735-ba24-e4165dcb4c81-workload-certs\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h\" (UID: \"fb5d58b6-89f9-4735-ba24-e4165dcb4c81\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h"
Oct 30 21:39:54.076951 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:54.076776    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb5d58b6-89f9-4735-ba24-e4165dcb4c81-istio-envoy\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h\" (UID: \"fb5d58b6-89f9-4735-ba24-e4165dcb4c81\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h"
Oct 30 21:39:54.076951 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:54.076860    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb5d58b6-89f9-4735-ba24-e4165dcb4c81-istiod-ca-cert\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h\" (UID: \"fb5d58b6-89f9-4735-ba24-e4165dcb4c81\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h"
Oct 30 21:39:54.077071 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:54.076962    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb5d58b6-89f9-4735-ba24-e4165dcb4c81-istio-data\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h\" (UID: \"fb5d58b6-89f9-4735-ba24-e4165dcb4c81\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h"
Oct 30 21:39:54.077120 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:54.077081    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb5d58b6-89f9-4735-ba24-e4165dcb4c81-istio-podinfo\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h\" (UID: \"fb5d58b6-89f9-4735-ba24-e4165dcb4c81\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h"
Oct 30 21:39:54.077172 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:54.077122    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb5d58b6-89f9-4735-ba24-e4165dcb4c81-workload-socket\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h\" (UID: \"fb5d58b6-89f9-4735-ba24-e4165dcb4c81\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h"
Oct 30 21:39:54.077254 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:54.077238    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb5d58b6-89f9-4735-ba24-e4165dcb4c81-istio-token\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h\" (UID: \"fb5d58b6-89f9-4735-ba24-e4165dcb4c81\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h"
Oct 30 21:39:54.077308 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:54.077272    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqqfd\" (UniqueName: \"kubernetes.io/projected/fb5d58b6-89f9-4735-ba24-e4165dcb4c81-kube-api-access-wqqfd\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h\" (UID: \"fb5d58b6-89f9-4735-ba24-e4165dcb4c81\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h"
Oct 30 21:39:54.077375 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:54.077303    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb5d58b6-89f9-4735-ba24-e4165dcb4c81-credential-socket\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h\" (UID: \"fb5d58b6-89f9-4735-ba24-e4165dcb4c81\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h"
Oct 30 21:39:54.146441 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:54.143234    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2"]
Oct 30 21:39:54.152967 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:54.152901    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:39:54.153088 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:54.153068    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:39:54.153181 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:54.153156    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:39:54.178662 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:54.178604    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb5d58b6-89f9-4735-ba24-e4165dcb4c81-istio-token\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h\" (UID: \"fb5d58b6-89f9-4735-ba24-e4165dcb4c81\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h"
Oct 30 21:39:54.178662 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:54.178650    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqqfd\" (UniqueName: \"kubernetes.io/projected/fb5d58b6-89f9-4735-ba24-e4165dcb4c81-kube-api-access-wqqfd\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h\" (UID: \"fb5d58b6-89f9-4735-ba24-e4165dcb4c81\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h"
Oct 30 21:39:54.178799 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:54.178687    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb5d58b6-89f9-4735-ba24-e4165dcb4c81-credential-socket\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h\" (UID: \"fb5d58b6-89f9-4735-ba24-e4165dcb4c81\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h"
Oct 30 21:39:54.178799 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:54.178743    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb5d58b6-89f9-4735-ba24-e4165dcb4c81-workload-certs\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h\" (UID: \"fb5d58b6-89f9-4735-ba24-e4165dcb4c81\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h"
Oct 30 21:39:54.178799 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:54.178768    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb5d58b6-89f9-4735-ba24-e4165dcb4c81-istio-envoy\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h\" (UID: \"fb5d58b6-89f9-4735-ba24-e4165dcb4c81\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h"
Oct 30 21:39:54.178799 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:54.178789    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb5d58b6-89f9-4735-ba24-e4165dcb4c81-istiod-ca-cert\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h\" (UID: \"fb5d58b6-89f9-4735-ba24-e4165dcb4c81\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h"
Oct 30 21:39:54.179012 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:54.178880    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb5d58b6-89f9-4735-ba24-e4165dcb4c81-istio-data\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h\" (UID: \"fb5d58b6-89f9-4735-ba24-e4165dcb4c81\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h"
Oct 30 21:39:54.179012 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:54.178998    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb5d58b6-89f9-4735-ba24-e4165dcb4c81-istio-podinfo\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h\" (UID: \"fb5d58b6-89f9-4735-ba24-e4165dcb4c81\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h"
Oct 30 21:39:54.179124 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:54.179037    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb5d58b6-89f9-4735-ba24-e4165dcb4c81-workload-socket\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h\" (UID: \"fb5d58b6-89f9-4735-ba24-e4165dcb4c81\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h"
Oct 30 21:39:54.179466 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:54.179437    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb5d58b6-89f9-4735-ba24-e4165dcb4c81-workload-certs\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h\" (UID: \"fb5d58b6-89f9-4735-ba24-e4165dcb4c81\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h"
Oct 30 21:39:54.179564 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:54.179489    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb5d58b6-89f9-4735-ba24-e4165dcb4c81-workload-socket\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h\" (UID: \"fb5d58b6-89f9-4735-ba24-e4165dcb4c81\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h"
Oct 30 21:39:54.179774 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:54.179751    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb5d58b6-89f9-4735-ba24-e4165dcb4c81-istio-data\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h\" (UID: \"fb5d58b6-89f9-4735-ba24-e4165dcb4c81\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h"
Oct 30 21:39:54.179909 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:54.179885    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb5d58b6-89f9-4735-ba24-e4165dcb4c81-istiod-ca-cert\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h\" (UID: \"fb5d58b6-89f9-4735-ba24-e4165dcb4c81\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h"
Oct 30 21:39:54.180006 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:54.179979    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb5d58b6-89f9-4735-ba24-e4165dcb4c81-credential-socket\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h\" (UID: \"fb5d58b6-89f9-4735-ba24-e4165dcb4c81\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h"
Oct 30 21:39:54.182643 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:54.182617    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb5d58b6-89f9-4735-ba24-e4165dcb4c81-istio-envoy\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h\" (UID: \"fb5d58b6-89f9-4735-ba24-e4165dcb4c81\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h"
Oct 30 21:39:54.183035 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:54.183013    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb5d58b6-89f9-4735-ba24-e4165dcb4c81-istio-podinfo\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h\" (UID: \"fb5d58b6-89f9-4735-ba24-e4165dcb4c81\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h"
Oct 30 21:39:54.194437 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:54.194414    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb5d58b6-89f9-4735-ba24-e4165dcb4c81-istio-token\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h\" (UID: \"fb5d58b6-89f9-4735-ba24-e4165dcb4c81\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h"
Oct 30 21:39:54.194535 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:54.194494    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqqfd\" (UniqueName: \"kubernetes.io/projected/fb5d58b6-89f9-4735-ba24-e4165dcb4c81-kube-api-access-wqqfd\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h\" (UID: \"fb5d58b6-89f9-4735-ba24-e4165dcb4c81\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h"
Oct 30 21:39:54.279156 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:54.279053    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2" event={"ID":"62000861-c573-4dba-8bca-c6f962057f5b","Type":"ContainerStarted","Data":"ff7e7fa1ccecb3bf7d7b3431f975cbaf5fa1442a004336175284517e055f4b07"}
Oct 30 21:39:54.279156 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:54.279150    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2" event={"ID":"62000861-c573-4dba-8bca-c6f962057f5b","Type":"ContainerStarted","Data":"e79ccd071d9d11c318b375907cae2cdd951e23e7d5ff2def6f3b7bdd6274dadd"}
Oct 30 21:39:54.309248 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:54.309195    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2" podStartSLOduration=1.309182678 podStartE2EDuration="1.309182678s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:39:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:39:54.309114975 +0000 UTC m=+707.110927440" watchObservedRunningTime="2026-10-30 21:39:54.309182678 +0000 UTC m=+707.110995141"
Oct 30 21:39:54.322655 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:54.322511    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h"
Oct 30 21:39:54.330803 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:54.330769    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76"]
Oct 30 21:39:54.338468 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:39:54.338436    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd4228de_c71f_4f88_a54e_9ebda8b91122.slice/crio-519894c4789453a0a85fe8a84f06fb0900004326c673254d6c184489ca3b4f10 WatchSource:0}: Error finding container 519894c4789453a0a85fe8a84f06fb0900004326c673254d6c184489ca3b4f10: Status 404 returned error can't find the container with id 519894c4789453a0a85fe8a84f06fb0900004326c673254d6c184489ca3b4f10
Oct 30 21:39:54.340754 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:54.340718    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:39:54.340862 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:54.340787    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:39:54.340862 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:54.340816    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:39:54.641539 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:39:54.641506    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb5d58b6_89f9_4735_ba24_e4165dcb4c81.slice/crio-6f69c058ac9be551378f6159da0710dba8cab0b2e9594f93f845d57cf476ca34 WatchSource:0}: Error finding container 6f69c058ac9be551378f6159da0710dba8cab0b2e9594f93f845d57cf476ca34: Status 404 returned error can't find the container with id 6f69c058ac9be551378f6159da0710dba8cab0b2e9594f93f845d57cf476ca34
Oct 30 21:39:54.641927 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:54.641747    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h"]
Oct 30 21:39:54.644511 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:54.644479    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:39:54.644742 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:54.644713    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:39:54.644848 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:54.644782    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:39:54.850496 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:54.850458    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2"
Oct 30 21:39:55.304697 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:55.304656    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h" event={"ID":"fb5d58b6-89f9-4735-ba24-e4165dcb4c81","Type":"ContainerStarted","Data":"1c5a110507f29ec0cbeab84476c21dd0413d3278f930d7aa409b5b74f51449ab"}
Oct 30 21:39:55.305093 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:55.305058    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h" event={"ID":"fb5d58b6-89f9-4735-ba24-e4165dcb4c81","Type":"ContainerStarted","Data":"6f69c058ac9be551378f6159da0710dba8cab0b2e9594f93f845d57cf476ca34"}
Oct 30 21:39:55.308279 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:55.308249    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76" event={"ID":"cd4228de-c71f-4f88-a54e-9ebda8b91122","Type":"ContainerStarted","Data":"4637b417d0d41dd176aed51d61219db6bdd0debc511a69a2e445a093cd626e4d"}
Oct 30 21:39:55.308603 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:55.308582    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76" event={"ID":"cd4228de-c71f-4f88-a54e-9ebda8b91122","Type":"ContainerStarted","Data":"519894c4789453a0a85fe8a84f06fb0900004326c673254d6c184489ca3b4f10"}
Oct 30 21:39:55.324764 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:55.324735    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h"
Oct 30 21:39:55.331659 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:55.331621    5032 patch_prober.go:29] interesting pod/e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Oct 30 21:39:55.331967 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:55.331925    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h" podUID="fb5d58b6-89f9-4735-ba24-e4165dcb4c81" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Oct 30 21:39:55.377321 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:55.374098    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h" podStartSLOduration=2.374078715 podStartE2EDuration="2.374078715s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:39:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:39:55.343966015 +0000 UTC m=+708.145778505" watchObservedRunningTime="2026-10-30 21:39:55.374078715 +0000 UTC m=+708.175891180"
Oct 30 21:39:55.572894 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:55.572802    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw"
Oct 30 21:39:55.863942 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:55.863541    5032 patch_prober.go:29] interesting pod/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Oct 30 21:39:55.863942 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:55.863604    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2" podUID="62000861-c573-4dba-8bca-c6f962057f5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Oct 30 21:39:56.051390 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:56.043727    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw"
Oct 30 21:39:56.085395 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:56.070232    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76" podStartSLOduration=3.070211934 podStartE2EDuration="3.070211934s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:39:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:39:55.372569211 +0000 UTC m=+708.174381699" watchObservedRunningTime="2026-10-30 21:39:56.070211934 +0000 UTC m=+708.872024398"
Oct 30 21:39:56.085789 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:56.071485    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76"
Oct 30 21:39:56.099389 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:56.094645    5032 patch_prober.go:29] interesting pod/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Oct 30 21:39:56.099389 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:56.094705    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76" podUID="cd4228de-c71f-4f88-a54e-9ebda8b91122" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Oct 30 21:39:56.342386 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:56.338612    5032 patch_prober.go:29] interesting pod/e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Oct 30 21:39:56.342386 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:56.338687    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h" podUID="fb5d58b6-89f9-4735-ba24-e4165dcb4c81" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Oct 30 21:39:56.848385 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:56.846768    5032 patch_prober.go:29] interesting pod/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Oct 30 21:39:56.848385 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:56.846832    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2" podUID="62000861-c573-4dba-8bca-c6f962057f5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Oct 30 21:39:56.907551 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:56.907491    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx" podUID="34fdf57d-86f6-474f-a4f6-811e5063c405" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:39:57.041016 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:57.030758    5032 patch_prober.go:29] interesting pod/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Oct 30 21:39:57.041016 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:57.030820    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76" podUID="cd4228de-c71f-4f88-a54e-9ebda8b91122" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Oct 30 21:39:57.168240 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:57.167861    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw"
Oct 30 21:39:57.420444 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:57.398671    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg" podUID="2d0ea058-b6ca-4181-8823-2b719883e545" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:39:57.487635 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:57.421199    5032 patch_prober.go:29] interesting pod/e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Oct 30 21:39:57.487635 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:57.421259    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h" podUID="fb5d58b6-89f9-4735-ba24-e4165dcb4c81" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Oct 30 21:39:57.764494 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:57.695493    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj" podUID="d0ff5023-fd3f-4734-94c6-76b666010adf" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:39:58.222073 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:58.206695    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc"
Oct 30 21:39:58.299207 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:58.299171    5032 patch_prober.go:29] interesting pod/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Oct 30 21:39:58.312699 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:58.312662    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76" podUID="cd4228de-c71f-4f88-a54e-9ebda8b91122" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Oct 30 21:39:58.315248 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:58.300332    5032 patch_prober.go:29] interesting pod/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Oct 30 21:39:58.424996 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:58.424951    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2" podUID="62000861-c573-4dba-8bca-c6f962057f5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Oct 30 21:39:58.425523 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:58.349706    5032 patch_prober.go:29] interesting pod/e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Oct 30 21:39:58.425737 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:58.425709    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h" podUID="fb5d58b6-89f9-4735-ba24-e4165dcb4c81" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Oct 30 21:39:58.699079 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:58.698992    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc"
Oct 30 21:39:58.865429 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:58.846145    5032 patch_prober.go:29] interesting pod/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Oct 30 21:39:58.865429 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:58.846202    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2" podUID="62000861-c573-4dba-8bca-c6f962057f5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Oct 30 21:39:59.007106 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:59.007077    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc"
Oct 30 21:39:59.034387 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:59.029721    5032 patch_prober.go:29] interesting pod/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Oct 30 21:39:59.034387 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:59.029794    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76" podUID="cd4228de-c71f-4f88-a54e-9ebda8b91122" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Oct 30 21:39:59.325853 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:59.325760    5032 patch_prober.go:29] interesting pod/e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Oct 30 21:39:59.326457 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:59.326430    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h" podUID="fb5d58b6-89f9-4735-ba24-e4165dcb4c81" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Oct 30 21:39:59.846822 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:59.846777    5032 patch_prober.go:29] interesting pod/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Oct 30 21:39:59.847094 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:39:59.847068    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2" podUID="62000861-c573-4dba-8bca-c6f962057f5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Oct 30 21:40:00.040515 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:00.040479    5032 patch_prober.go:29] interesting pod/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Oct 30 21:40:00.040797 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:00.040765    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76" podUID="cd4228de-c71f-4f88-a54e-9ebda8b91122" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Oct 30 21:40:00.326525 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:00.326490    5032 patch_prober.go:29] interesting pod/e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Oct 30 21:40:00.327099 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:00.327069    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h" podUID="fb5d58b6-89f9-4735-ba24-e4165dcb4c81" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Oct 30 21:40:00.858668 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:00.858631    5032 patch_prober.go:29] interesting pod/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Oct 30 21:40:00.858947 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:00.858918    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2" podUID="62000861-c573-4dba-8bca-c6f962057f5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Oct 30 21:40:01.043384 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:01.042811    5032 patch_prober.go:29] interesting pod/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Oct 30 21:40:01.043384 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:01.042875    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76" podUID="cd4228de-c71f-4f88-a54e-9ebda8b91122" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Oct 30 21:40:01.330384 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:01.328783    5032 patch_prober.go:29] interesting pod/e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Oct 30 21:40:01.330384 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:01.328845    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h" podUID="fb5d58b6-89f9-4735-ba24-e4165dcb4c81" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Oct 30 21:40:02.323806 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:02.323761    5032 patch_prober.go:29] interesting pod/e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Oct 30 21:40:02.324073 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:02.323853    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h" podUID="fb5d58b6-89f9-4735-ba24-e4165dcb4c81" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Oct 30 21:40:02.941387 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:02.927541    5032 patch_prober.go:29] interesting pod/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:40:02.941387 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:02.927602    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2" podUID="62000861-c573-4dba-8bca-c6f962057f5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:40:03.032871 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:03.032834    5032 patch_prober.go:29] interesting pod/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:40:03.033172 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:03.033134    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76" podUID="cd4228de-c71f-4f88-a54e-9ebda8b91122" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:40:03.089399 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:03.088820    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76"
Oct 30 21:40:03.090419 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:03.090398    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76"
Oct 30 21:40:03.092665 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:03.092637    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76"
Oct 30 21:40:03.116088 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:03.116065    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2"
Oct 30 21:40:03.117285 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:03.117252    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2"
Oct 30 21:40:03.121191 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:03.121165    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2"
Oct 30 21:40:03.471842 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:03.471809    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h"
Oct 30 21:40:03.472338 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:03.472299    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h"
Oct 30 21:40:03.473175 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:03.473153    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h"
Oct 30 21:40:09.843072 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:09.842996    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw"]
Oct 30 21:40:09.850854 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:09.850829    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw"
Oct 30 21:40:09.860034 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:09.860011    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-0fa864\"/\"distinct-2-w4-0fa864-kserve-self-signed-certs\""
Oct 30 21:40:09.865058 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:09.865016    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-0fa864\"/\"openshift-service-ca.crt\""
Oct 30 21:40:09.865524 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:09.865506    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-0fa864\"/\"kube-root-ca.crt\""
Oct 30 21:40:09.869025 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:09.869003    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw"]
Oct 30 21:40:09.907432 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:09.907393    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/914f7544-6ef8-4496-b98d-b7ebc4123cda-tls-certs\") pod \"distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw\" (UID: \"914f7544-6ef8-4496-b98d-b7ebc4123cda\") " pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw"
Oct 30 21:40:09.907606 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:09.907514    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hsrd\" (UniqueName: \"kubernetes.io/projected/914f7544-6ef8-4496-b98d-b7ebc4123cda-kube-api-access-9hsrd\") pod \"distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw\" (UID: \"914f7544-6ef8-4496-b98d-b7ebc4123cda\") " pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw"
Oct 30 21:40:09.907606 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:09.907555    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/914f7544-6ef8-4496-b98d-b7ebc4123cda-model-cache\") pod \"distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw\" (UID: \"914f7544-6ef8-4496-b98d-b7ebc4123cda\") " pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw"
Oct 30 21:40:09.907683 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:09.907612    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/914f7544-6ef8-4496-b98d-b7ebc4123cda-home\") pod \"distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw\" (UID: \"914f7544-6ef8-4496-b98d-b7ebc4123cda\") " pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw"
Oct 30 21:40:09.907683 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:09.907627    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/914f7544-6ef8-4496-b98d-b7ebc4123cda-dshm\") pod \"distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw\" (UID: \"914f7544-6ef8-4496-b98d-b7ebc4123cda\") " pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw"
Oct 30 21:40:09.907683 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:09.907664    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/914f7544-6ef8-4496-b98d-b7ebc4123cda-tmp-dir\") pod \"distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw\" (UID: \"914f7544-6ef8-4496-b98d-b7ebc4123cda\") " pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw"
Oct 30 21:40:09.960103 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:09.960065    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-0fa864\"/\"default-dockercfg-jldb5\""
Oct 30 21:40:10.009344 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:10.009291    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hsrd\" (UniqueName: \"kubernetes.io/projected/914f7544-6ef8-4496-b98d-b7ebc4123cda-kube-api-access-9hsrd\") pod \"distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw\" (UID: \"914f7544-6ef8-4496-b98d-b7ebc4123cda\") " pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw"
Oct 30 21:40:10.009554 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:10.009383    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/914f7544-6ef8-4496-b98d-b7ebc4123cda-model-cache\") pod \"distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw\" (UID: \"914f7544-6ef8-4496-b98d-b7ebc4123cda\") " pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw"
Oct 30 21:40:10.009554 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:10.009469    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/914f7544-6ef8-4496-b98d-b7ebc4123cda-home\") pod \"distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw\" (UID: \"914f7544-6ef8-4496-b98d-b7ebc4123cda\") " pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw"
Oct 30 21:40:10.009554 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:10.009492    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/914f7544-6ef8-4496-b98d-b7ebc4123cda-dshm\") pod \"distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw\" (UID: \"914f7544-6ef8-4496-b98d-b7ebc4123cda\") " pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw"
Oct 30 21:40:10.009554 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:10.009546    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/914f7544-6ef8-4496-b98d-b7ebc4123cda-tmp-dir\") pod \"distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw\" (UID: \"914f7544-6ef8-4496-b98d-b7ebc4123cda\") " pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw"
Oct 30 21:40:10.009782 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:10.009710    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/914f7544-6ef8-4496-b98d-b7ebc4123cda-tls-certs\") pod \"distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw\" (UID: \"914f7544-6ef8-4496-b98d-b7ebc4123cda\") " pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw"
Oct 30 21:40:10.009908 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:10.009886    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/914f7544-6ef8-4496-b98d-b7ebc4123cda-model-cache\") pod \"distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw\" (UID: \"914f7544-6ef8-4496-b98d-b7ebc4123cda\") " pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw"
Oct 30 21:40:10.010191 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:10.010167    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/914f7544-6ef8-4496-b98d-b7ebc4123cda-tmp-dir\") pod \"distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw\" (UID: \"914f7544-6ef8-4496-b98d-b7ebc4123cda\") " pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw"
Oct 30 21:40:10.010289 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:10.010228    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/914f7544-6ef8-4496-b98d-b7ebc4123cda-home\") pod \"distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw\" (UID: \"914f7544-6ef8-4496-b98d-b7ebc4123cda\") " pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw"
Oct 30 21:40:10.012193 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:10.012170    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/914f7544-6ef8-4496-b98d-b7ebc4123cda-dshm\") pod \"distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw\" (UID: \"914f7544-6ef8-4496-b98d-b7ebc4123cda\") " pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw"
Oct 30 21:40:10.012375 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:10.012346    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/914f7544-6ef8-4496-b98d-b7ebc4123cda-tls-certs\") pod \"distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw\" (UID: \"914f7544-6ef8-4496-b98d-b7ebc4123cda\") " pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw"
Oct 30 21:40:10.022954 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:10.022922    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hsrd\" (UniqueName: \"kubernetes.io/projected/914f7544-6ef8-4496-b98d-b7ebc4123cda-kube-api-access-9hsrd\") pod \"distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw\" (UID: \"914f7544-6ef8-4496-b98d-b7ebc4123cda\") " pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw"
Oct 30 21:40:10.163642 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:10.163557    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw"
Oct 30 21:40:10.447448 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:10.447420    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw"]
Oct 30 21:40:10.448624 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:40:10.448596    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod914f7544_6ef8_4496_b98d_b7ebc4123cda.slice/crio-3bf1d930d5f2c23358af0342c0fd8205b60e23ab25f6251002d4947cb989ec81 WatchSource:0}: Error finding container 3bf1d930d5f2c23358af0342c0fd8205b60e23ab25f6251002d4947cb989ec81: Status 404 returned error can't find the container with id 3bf1d930d5f2c23358af0342c0fd8205b60e23ab25f6251002d4947cb989ec81
Oct 30 21:40:11.166129 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:11.166088    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw" event={"ID":"914f7544-6ef8-4496-b98d-b7ebc4123cda","Type":"ContainerStarted","Data":"a76aa49cf87523727b92511ed06e0157f5e68250327deea402ed3da6905be90e"}
Oct 30 21:40:11.166552 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:11.166138    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw" event={"ID":"914f7544-6ef8-4496-b98d-b7ebc4123cda","Type":"ContainerStarted","Data":"3bf1d930d5f2c23358af0342c0fd8205b60e23ab25f6251002d4947cb989ec81"}
Oct 30 21:40:11.189135 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:11.189089    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw" podStartSLOduration=2.189075263 podStartE2EDuration="2.189075263s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:40:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:40:11.187295005 +0000 UTC m=+723.989107467" watchObservedRunningTime="2026-10-30 21:40:11.189075263 +0000 UTC m=+723.990887727"
Oct 30 21:40:12.163848 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:12.163811    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw"
Oct 30 21:40:16.318575 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:16.318539    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-v4k4f"]
Oct 30 21:40:16.323162 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:16.323141    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-v4k4f"
Oct 30 21:40:16.333199 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:16.333170    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Oct 30 21:40:16.334289 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:16.334264    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Oct 30 21:40:16.334416 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:16.334265    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Oct 30 21:40:16.334416 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:16.334390    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-747gg\""
Oct 30 21:40:16.339493 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:16.339473    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-v4k4f"]
Oct 30 21:40:16.498978 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:16.498936    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9b50efc9-2123-439d-9a07-e027fe6a6bfa-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-v4k4f\" (UID: \"9b50efc9-2123-439d-9a07-e027fe6a6bfa\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-v4k4f"
Oct 30 21:40:16.499164 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:16.499086    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b50efc9-2123-439d-9a07-e027fe6a6bfa-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-v4k4f\" (UID: \"9b50efc9-2123-439d-9a07-e027fe6a6bfa\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-v4k4f"
Oct 30 21:40:16.499164 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:16.499133    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84xzr\" (UniqueName: \"kubernetes.io/projected/9b50efc9-2123-439d-9a07-e027fe6a6bfa-kube-api-access-84xzr\") pod \"istiod-openshift-gateway-7ddd9cfdc8-v4k4f\" (UID: \"9b50efc9-2123-439d-9a07-e027fe6a6bfa\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-v4k4f"
Oct 30 21:40:16.499239 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:16.499165    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9b50efc9-2123-439d-9a07-e027fe6a6bfa-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-v4k4f\" (UID: \"9b50efc9-2123-439d-9a07-e027fe6a6bfa\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-v4k4f"
Oct 30 21:40:16.499292 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:16.499278    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9b50efc9-2123-439d-9a07-e027fe6a6bfa-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-v4k4f\" (UID: \"9b50efc9-2123-439d-9a07-e027fe6a6bfa\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-v4k4f"
Oct 30 21:40:16.499333 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:16.499312    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b50efc9-2123-439d-9a07-e027fe6a6bfa-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-v4k4f\" (UID: \"9b50efc9-2123-439d-9a07-e027fe6a6bfa\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-v4k4f"
Oct 30 21:40:16.499692 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:16.499673    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9b50efc9-2123-439d-9a07-e027fe6a6bfa-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-v4k4f\" (UID: \"9b50efc9-2123-439d-9a07-e027fe6a6bfa\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-v4k4f"
Oct 30 21:40:16.601293 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:16.601209    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9b50efc9-2123-439d-9a07-e027fe6a6bfa-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-v4k4f\" (UID: \"9b50efc9-2123-439d-9a07-e027fe6a6bfa\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-v4k4f"
Oct 30 21:40:16.601293 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:16.601266    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b50efc9-2123-439d-9a07-e027fe6a6bfa-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-v4k4f\" (UID: \"9b50efc9-2123-439d-9a07-e027fe6a6bfa\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-v4k4f"
Oct 30 21:40:16.601293 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:16.601290    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84xzr\" (UniqueName: \"kubernetes.io/projected/9b50efc9-2123-439d-9a07-e027fe6a6bfa-kube-api-access-84xzr\") pod \"istiod-openshift-gateway-7ddd9cfdc8-v4k4f\" (UID: \"9b50efc9-2123-439d-9a07-e027fe6a6bfa\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-v4k4f"
Oct 30 21:40:16.601670 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:16.601318    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9b50efc9-2123-439d-9a07-e027fe6a6bfa-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-v4k4f\" (UID: \"9b50efc9-2123-439d-9a07-e027fe6a6bfa\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-v4k4f"
Oct 30 21:40:16.601670 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:16.601342    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9b50efc9-2123-439d-9a07-e027fe6a6bfa-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-v4k4f\" (UID: \"9b50efc9-2123-439d-9a07-e027fe6a6bfa\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-v4k4f"
Oct 30 21:40:16.601670 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:16.601432    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b50efc9-2123-439d-9a07-e027fe6a6bfa-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-v4k4f\" (UID: \"9b50efc9-2123-439d-9a07-e027fe6a6bfa\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-v4k4f"
Oct 30 21:40:16.601670 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:16.601541    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9b50efc9-2123-439d-9a07-e027fe6a6bfa-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-v4k4f\" (UID: \"9b50efc9-2123-439d-9a07-e027fe6a6bfa\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-v4k4f"
Oct 30 21:40:16.602276 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:16.602250    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9b50efc9-2123-439d-9a07-e027fe6a6bfa-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-v4k4f\" (UID: \"9b50efc9-2123-439d-9a07-e027fe6a6bfa\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-v4k4f"
Oct 30 21:40:16.603899 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:16.603871    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9b50efc9-2123-439d-9a07-e027fe6a6bfa-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-v4k4f\" (UID: \"9b50efc9-2123-439d-9a07-e027fe6a6bfa\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-v4k4f"
Oct 30 21:40:16.603991 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:16.603904    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b50efc9-2123-439d-9a07-e027fe6a6bfa-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-v4k4f\" (UID: \"9b50efc9-2123-439d-9a07-e027fe6a6bfa\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-v4k4f"
Oct 30 21:40:16.603991 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:16.603954    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9b50efc9-2123-439d-9a07-e027fe6a6bfa-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-v4k4f\" (UID: \"9b50efc9-2123-439d-9a07-e027fe6a6bfa\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-v4k4f"
Oct 30 21:40:16.604117 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:16.604098    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9b50efc9-2123-439d-9a07-e027fe6a6bfa-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-v4k4f\" (UID: \"9b50efc9-2123-439d-9a07-e027fe6a6bfa\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-v4k4f"
Oct 30 21:40:16.629851 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:16.629819    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b50efc9-2123-439d-9a07-e027fe6a6bfa-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-v4k4f\" (UID: \"9b50efc9-2123-439d-9a07-e027fe6a6bfa\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-v4k4f"
Oct 30 21:40:16.638952 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:16.638925    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-84xzr\" (UniqueName: \"kubernetes.io/projected/9b50efc9-2123-439d-9a07-e027fe6a6bfa-kube-api-access-84xzr\") pod \"istiod-openshift-gateway-7ddd9cfdc8-v4k4f\" (UID: \"9b50efc9-2123-439d-9a07-e027fe6a6bfa\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-v4k4f"
Oct 30 21:40:16.937987 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:16.937887    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-v4k4f"
Oct 30 21:40:16.974750 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:16.974718    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 21:40:17.244890 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:17.244863    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-v4k4f"]
Oct 30 21:40:17.245991 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:40:17.245960    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b50efc9_2123_439d_9a07_e027fe6a6bfa.slice/crio-c349f7c92ba9f1ba8b52ccd87eb229b978549af068812237b4227198a211468a WatchSource:0}: Error finding container c349f7c92ba9f1ba8b52ccd87eb229b978549af068812237b4227198a211468a: Status 404 returned error can't find the container with id c349f7c92ba9f1ba8b52ccd87eb229b978549af068812237b4227198a211468a
Oct 30 21:40:17.248045 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:17.248029    5032 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 21:40:18.242382 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:18.242029    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-v4k4f" event={"ID":"9b50efc9-2123-439d-9a07-e027fe6a6bfa","Type":"ContainerStarted","Data":"c349f7c92ba9f1ba8b52ccd87eb229b978549af068812237b4227198a211468a"}
Oct 30 21:40:18.383386 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:40:18.382882    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 21:40:20.164870 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:20.164446    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw"
Oct 30 21:40:20.185078 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:20.185048    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw"
Oct 30 21:40:20.277332 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:20.277297    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw"
Oct 30 21:40:20.928214 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:20.928183    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:40:20.928284 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:20.928248    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:40:21.275919 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:21.275878    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-v4k4f" event={"ID":"9b50efc9-2123-439d-9a07-e027fe6a6bfa","Type":"ContainerStarted","Data":"3dd3fd2a5d100e6265c56eb20ed982a2abe7964a65c3d4b0485f87a5c891010a"}
Oct 30 21:40:21.276458 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:21.275985    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-v4k4f"
Oct 30 21:40:21.278252 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:21.278225    5032 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-v4k4f container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:40:21.278400 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:21.278272    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-v4k4f" podUID="9b50efc9-2123-439d-9a07-e027fe6a6bfa" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:40:21.303908 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:21.303845    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-v4k4f" podStartSLOduration=1.6250058859999998 podStartE2EDuration="5.303826286s" totalImagesPullingTime="3.6788204s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:40:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:40:21.3016049 +0000 UTC m=+734.103417363" watchObservedRunningTime="2026-10-30 21:40:21.303826286 +0000 UTC m=+734.105638733"
Oct 30 21:40:22.283893 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:22.283866    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-v4k4f"
Oct 30 21:40:48.346609 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:40:48.346520    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 21:40:49.048293 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:40:49.048253    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 21:41:08.333293 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.333249    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8"]
Oct 30 21:41:08.339598 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.339558    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8"
Oct 30 21:41:08.358117 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.358086    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8"]
Oct 30 21:41:08.427441 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.427411    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv"]
Oct 30 21:41:08.431743 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.431723    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv"
Oct 30 21:41:08.445851 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.445826    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv"]
Oct 30 21:41:08.459422 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.459395    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7679391-8575-43d7-b7f1-5b1ab0bcee5f-workload-socket\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8\" (UID: \"e7679391-8575-43d7-b7f1-5b1ab0bcee5f\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8"
Oct 30 21:41:08.459579 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.459467    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hbd2\" (UniqueName: \"kubernetes.io/projected/e7679391-8575-43d7-b7f1-5b1ab0bcee5f-kube-api-access-2hbd2\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8\" (UID: \"e7679391-8575-43d7-b7f1-5b1ab0bcee5f\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8"
Oct 30 21:41:08.459721 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.459703    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7679391-8575-43d7-b7f1-5b1ab0bcee5f-credential-socket\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8\" (UID: \"e7679391-8575-43d7-b7f1-5b1ab0bcee5f\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8"
Oct 30 21:41:08.459788 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.459751    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7679391-8575-43d7-b7f1-5b1ab0bcee5f-istio-token\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8\" (UID: \"e7679391-8575-43d7-b7f1-5b1ab0bcee5f\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8"
Oct 30 21:41:08.459788 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.459779    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e7679391-8575-43d7-b7f1-5b1ab0bcee5f-istiod-ca-cert\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8\" (UID: \"e7679391-8575-43d7-b7f1-5b1ab0bcee5f\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8"
Oct 30 21:41:08.459877 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.459860    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e7679391-8575-43d7-b7f1-5b1ab0bcee5f-istio-envoy\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8\" (UID: \"e7679391-8575-43d7-b7f1-5b1ab0bcee5f\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8"
Oct 30 21:41:08.459912 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.459887    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e7679391-8575-43d7-b7f1-5b1ab0bcee5f-istio-podinfo\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8\" (UID: \"e7679391-8575-43d7-b7f1-5b1ab0bcee5f\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8"
Oct 30 21:41:08.459989 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.459976    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7679391-8575-43d7-b7f1-5b1ab0bcee5f-workload-certs\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8\" (UID: \"e7679391-8575-43d7-b7f1-5b1ab0bcee5f\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8"
Oct 30 21:41:08.460039 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.460024    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e7679391-8575-43d7-b7f1-5b1ab0bcee5f-istio-data\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8\" (UID: \"e7679391-8575-43d7-b7f1-5b1ab0bcee5f\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8"
Oct 30 21:41:08.471564 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.468253    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj"]
Oct 30 21:41:08.473746 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.473711    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj"
Oct 30 21:41:08.489143 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.488801    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj"]
Oct 30 21:41:08.561263 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.561222    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30ea822a-85f1-4db5-a3a2-b8edd78d1200-credential-socket\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv\" (UID: \"30ea822a-85f1-4db5-a3a2-b8edd78d1200\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv"
Oct 30 21:41:08.561459 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.561272    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e7679391-8575-43d7-b7f1-5b1ab0bcee5f-istio-data\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8\" (UID: \"e7679391-8575-43d7-b7f1-5b1ab0bcee5f\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8"
Oct 30 21:41:08.561459 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.561300    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhkx9\" (UniqueName: \"kubernetes.io/projected/30ea822a-85f1-4db5-a3a2-b8edd78d1200-kube-api-access-xhkx9\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv\" (UID: \"30ea822a-85f1-4db5-a3a2-b8edd78d1200\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv"
Oct 30 21:41:08.561553 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.561467    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4577b3de-b9e4-47a7-87e0-bcba13b11bbe-istio-token\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj\" (UID: \"4577b3de-b9e4-47a7-87e0-bcba13b11bbe\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj"
Oct 30 21:41:08.561553 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.561538    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30ea822a-85f1-4db5-a3a2-b8edd78d1200-istio-data\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv\" (UID: \"30ea822a-85f1-4db5-a3a2-b8edd78d1200\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv"
Oct 30 21:41:08.561662 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.561641    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7679391-8575-43d7-b7f1-5b1ab0bcee5f-workload-socket\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8\" (UID: \"e7679391-8575-43d7-b7f1-5b1ab0bcee5f\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8"
Oct 30 21:41:08.561731 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.561710    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hbd2\" (UniqueName: \"kubernetes.io/projected/e7679391-8575-43d7-b7f1-5b1ab0bcee5f-kube-api-access-2hbd2\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8\" (UID: \"e7679391-8575-43d7-b7f1-5b1ab0bcee5f\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8"
Oct 30 21:41:08.561785 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.561734    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e7679391-8575-43d7-b7f1-5b1ab0bcee5f-istio-data\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8\" (UID: \"e7679391-8575-43d7-b7f1-5b1ab0bcee5f\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8"
Oct 30 21:41:08.561837 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.561805    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4577b3de-b9e4-47a7-87e0-bcba13b11bbe-workload-socket\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj\" (UID: \"4577b3de-b9e4-47a7-87e0-bcba13b11bbe\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj"
Oct 30 21:41:08.561896 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.561859    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7679391-8575-43d7-b7f1-5b1ab0bcee5f-credential-socket\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8\" (UID: \"e7679391-8575-43d7-b7f1-5b1ab0bcee5f\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8"
Oct 30 21:41:08.561949 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.561913    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7679391-8575-43d7-b7f1-5b1ab0bcee5f-workload-socket\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8\" (UID: \"e7679391-8575-43d7-b7f1-5b1ab0bcee5f\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8"
Oct 30 21:41:08.561949 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.561919    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4577b3de-b9e4-47a7-87e0-bcba13b11bbe-istiod-ca-cert\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj\" (UID: \"4577b3de-b9e4-47a7-87e0-bcba13b11bbe\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj"
Oct 30 21:41:08.562052 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.561988    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30ea822a-85f1-4db5-a3a2-b8edd78d1200-istio-token\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv\" (UID: \"30ea822a-85f1-4db5-a3a2-b8edd78d1200\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv"
Oct 30 21:41:08.562052 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.562027    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7679391-8575-43d7-b7f1-5b1ab0bcee5f-istio-token\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8\" (UID: \"e7679391-8575-43d7-b7f1-5b1ab0bcee5f\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8"
Oct 30 21:41:08.562160 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.562053    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e7679391-8575-43d7-b7f1-5b1ab0bcee5f-istiod-ca-cert\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8\" (UID: \"e7679391-8575-43d7-b7f1-5b1ab0bcee5f\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8"
Oct 30 21:41:08.562160 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.562092    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwr7d\" (UniqueName: \"kubernetes.io/projected/4577b3de-b9e4-47a7-87e0-bcba13b11bbe-kube-api-access-nwr7d\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj\" (UID: \"4577b3de-b9e4-47a7-87e0-bcba13b11bbe\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj"
Oct 30 21:41:08.562160 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.562147    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7679391-8575-43d7-b7f1-5b1ab0bcee5f-credential-socket\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8\" (UID: \"e7679391-8575-43d7-b7f1-5b1ab0bcee5f\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8"
Oct 30 21:41:08.562314 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.562213    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4577b3de-b9e4-47a7-87e0-bcba13b11bbe-istio-data\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj\" (UID: \"4577b3de-b9e4-47a7-87e0-bcba13b11bbe\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj"
Oct 30 21:41:08.562314 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.562261    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e7679391-8575-43d7-b7f1-5b1ab0bcee5f-istio-envoy\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8\" (UID: \"e7679391-8575-43d7-b7f1-5b1ab0bcee5f\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8"
Oct 30 21:41:08.562314 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.562306    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e7679391-8575-43d7-b7f1-5b1ab0bcee5f-istio-podinfo\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8\" (UID: \"e7679391-8575-43d7-b7f1-5b1ab0bcee5f\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8"
Oct 30 21:41:08.562502 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.562329    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30ea822a-85f1-4db5-a3a2-b8edd78d1200-istio-podinfo\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv\" (UID: \"30ea822a-85f1-4db5-a3a2-b8edd78d1200\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv"
Oct 30 21:41:08.562502 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.562419    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30ea822a-85f1-4db5-a3a2-b8edd78d1200-istio-envoy\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv\" (UID: \"30ea822a-85f1-4db5-a3a2-b8edd78d1200\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv"
Oct 30 21:41:08.562609 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.562515    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30ea822a-85f1-4db5-a3a2-b8edd78d1200-workload-certs\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv\" (UID: \"30ea822a-85f1-4db5-a3a2-b8edd78d1200\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv"
Oct 30 21:41:08.562609 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.562548    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4577b3de-b9e4-47a7-87e0-bcba13b11bbe-workload-certs\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj\" (UID: \"4577b3de-b9e4-47a7-87e0-bcba13b11bbe\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj"
Oct 30 21:41:08.562609 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.562573    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4577b3de-b9e4-47a7-87e0-bcba13b11bbe-istio-envoy\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj\" (UID: \"4577b3de-b9e4-47a7-87e0-bcba13b11bbe\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj"
Oct 30 21:41:08.562771 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.562633    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4577b3de-b9e4-47a7-87e0-bcba13b11bbe-istio-podinfo\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj\" (UID: \"4577b3de-b9e4-47a7-87e0-bcba13b11bbe\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj"
Oct 30 21:41:08.562771 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.562665    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30ea822a-85f1-4db5-a3a2-b8edd78d1200-istiod-ca-cert\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv\" (UID: \"30ea822a-85f1-4db5-a3a2-b8edd78d1200\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv"
Oct 30 21:41:08.562771 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.562688    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7679391-8575-43d7-b7f1-5b1ab0bcee5f-workload-certs\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8\" (UID: \"e7679391-8575-43d7-b7f1-5b1ab0bcee5f\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8"
Oct 30 21:41:08.562771 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.562751    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30ea822a-85f1-4db5-a3a2-b8edd78d1200-workload-socket\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv\" (UID: \"30ea822a-85f1-4db5-a3a2-b8edd78d1200\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv"
Oct 30 21:41:08.562946 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.562791    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4577b3de-b9e4-47a7-87e0-bcba13b11bbe-credential-socket\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj\" (UID: \"4577b3de-b9e4-47a7-87e0-bcba13b11bbe\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj"
Oct 30 21:41:08.562946 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.562822    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e7679391-8575-43d7-b7f1-5b1ab0bcee5f-istiod-ca-cert\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8\" (UID: \"e7679391-8575-43d7-b7f1-5b1ab0bcee5f\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8"
Oct 30 21:41:08.562946 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.562911    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7679391-8575-43d7-b7f1-5b1ab0bcee5f-workload-certs\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8\" (UID: \"e7679391-8575-43d7-b7f1-5b1ab0bcee5f\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8"
Oct 30 21:41:08.564635 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.564611    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e7679391-8575-43d7-b7f1-5b1ab0bcee5f-istio-envoy\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8\" (UID: \"e7679391-8575-43d7-b7f1-5b1ab0bcee5f\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8"
Oct 30 21:41:08.564885 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.564869    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e7679391-8575-43d7-b7f1-5b1ab0bcee5f-istio-podinfo\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8\" (UID: \"e7679391-8575-43d7-b7f1-5b1ab0bcee5f\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8"
Oct 30 21:41:08.574536 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.574505    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7679391-8575-43d7-b7f1-5b1ab0bcee5f-istio-token\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8\" (UID: \"e7679391-8575-43d7-b7f1-5b1ab0bcee5f\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8"
Oct 30 21:41:08.574650 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.574636    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hbd2\" (UniqueName: \"kubernetes.io/projected/e7679391-8575-43d7-b7f1-5b1ab0bcee5f-kube-api-access-2hbd2\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8\" (UID: \"e7679391-8575-43d7-b7f1-5b1ab0bcee5f\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8"
Oct 30 21:41:08.664533 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.664436    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4577b3de-b9e4-47a7-87e0-bcba13b11bbe-workload-socket\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj\" (UID: \"4577b3de-b9e4-47a7-87e0-bcba13b11bbe\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj"
Oct 30 21:41:08.664716 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.664533    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4577b3de-b9e4-47a7-87e0-bcba13b11bbe-istiod-ca-cert\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj\" (UID: \"4577b3de-b9e4-47a7-87e0-bcba13b11bbe\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj"
Oct 30 21:41:08.664716 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.664563    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30ea822a-85f1-4db5-a3a2-b8edd78d1200-istio-token\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv\" (UID: \"30ea822a-85f1-4db5-a3a2-b8edd78d1200\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv"
Oct 30 21:41:08.664716 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.664593    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwr7d\" (UniqueName: \"kubernetes.io/projected/4577b3de-b9e4-47a7-87e0-bcba13b11bbe-kube-api-access-nwr7d\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj\" (UID: \"4577b3de-b9e4-47a7-87e0-bcba13b11bbe\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj"
Oct 30 21:41:08.664716 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.664620    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4577b3de-b9e4-47a7-87e0-bcba13b11bbe-istio-data\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj\" (UID: \"4577b3de-b9e4-47a7-87e0-bcba13b11bbe\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj"
Oct 30 21:41:08.664716 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.664657    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30ea822a-85f1-4db5-a3a2-b8edd78d1200-istio-podinfo\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv\" (UID: \"30ea822a-85f1-4db5-a3a2-b8edd78d1200\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv"
Oct 30 21:41:08.664978 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.664729    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30ea822a-85f1-4db5-a3a2-b8edd78d1200-istio-envoy\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv\" (UID: \"30ea822a-85f1-4db5-a3a2-b8edd78d1200\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv"
Oct 30 21:41:08.664978 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.664762    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30ea822a-85f1-4db5-a3a2-b8edd78d1200-workload-certs\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv\" (UID: \"30ea822a-85f1-4db5-a3a2-b8edd78d1200\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv"
Oct 30 21:41:08.664978 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.664790    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4577b3de-b9e4-47a7-87e0-bcba13b11bbe-workload-certs\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj\" (UID: \"4577b3de-b9e4-47a7-87e0-bcba13b11bbe\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj"
Oct 30 21:41:08.664978 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.664815    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4577b3de-b9e4-47a7-87e0-bcba13b11bbe-istio-envoy\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj\" (UID: \"4577b3de-b9e4-47a7-87e0-bcba13b11bbe\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj"
Oct 30 21:41:08.664978 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.664845    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4577b3de-b9e4-47a7-87e0-bcba13b11bbe-istio-podinfo\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj\" (UID: \"4577b3de-b9e4-47a7-87e0-bcba13b11bbe\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj"
Oct 30 21:41:08.664978 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.664875    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30ea822a-85f1-4db5-a3a2-b8edd78d1200-istiod-ca-cert\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv\" (UID: \"30ea822a-85f1-4db5-a3a2-b8edd78d1200\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv"
Oct 30 21:41:08.664978 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.664915    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30ea822a-85f1-4db5-a3a2-b8edd78d1200-workload-socket\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv\" (UID: \"30ea822a-85f1-4db5-a3a2-b8edd78d1200\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv"
Oct 30 21:41:08.664978 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.664942    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4577b3de-b9e4-47a7-87e0-bcba13b11bbe-credential-socket\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj\" (UID: \"4577b3de-b9e4-47a7-87e0-bcba13b11bbe\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj"
Oct 30 21:41:08.664978 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.664949    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4577b3de-b9e4-47a7-87e0-bcba13b11bbe-workload-socket\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj\" (UID: \"4577b3de-b9e4-47a7-87e0-bcba13b11bbe\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj"
Oct 30 21:41:08.665466 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.664995    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30ea822a-85f1-4db5-a3a2-b8edd78d1200-credential-socket\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv\" (UID: \"30ea822a-85f1-4db5-a3a2-b8edd78d1200\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv"
Oct 30 21:41:08.665466 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.665043    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhkx9\" (UniqueName: \"kubernetes.io/projected/30ea822a-85f1-4db5-a3a2-b8edd78d1200-kube-api-access-xhkx9\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv\" (UID: \"30ea822a-85f1-4db5-a3a2-b8edd78d1200\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv"
Oct 30 21:41:08.665466 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.665074    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4577b3de-b9e4-47a7-87e0-bcba13b11bbe-istio-token\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj\" (UID: \"4577b3de-b9e4-47a7-87e0-bcba13b11bbe\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj"
Oct 30 21:41:08.665466 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.665140    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30ea822a-85f1-4db5-a3a2-b8edd78d1200-istio-data\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv\" (UID: \"30ea822a-85f1-4db5-a3a2-b8edd78d1200\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv"
Oct 30 21:41:08.665466 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.665229    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4577b3de-b9e4-47a7-87e0-bcba13b11bbe-istio-data\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj\" (UID: \"4577b3de-b9e4-47a7-87e0-bcba13b11bbe\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj"
Oct 30 21:41:08.665466 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.665309    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4577b3de-b9e4-47a7-87e0-bcba13b11bbe-istiod-ca-cert\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj\" (UID: \"4577b3de-b9e4-47a7-87e0-bcba13b11bbe\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj"
Oct 30 21:41:08.665821 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.665659    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30ea822a-85f1-4db5-a3a2-b8edd78d1200-istio-data\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv\" (UID: \"30ea822a-85f1-4db5-a3a2-b8edd78d1200\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv"
Oct 30 21:41:08.665821 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.665740    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30ea822a-85f1-4db5-a3a2-b8edd78d1200-workload-certs\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv\" (UID: \"30ea822a-85f1-4db5-a3a2-b8edd78d1200\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv"
Oct 30 21:41:08.665936 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.665842    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30ea822a-85f1-4db5-a3a2-b8edd78d1200-credential-socket\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv\" (UID: \"30ea822a-85f1-4db5-a3a2-b8edd78d1200\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv"
Oct 30 21:41:08.665936 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.665886    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30ea822a-85f1-4db5-a3a2-b8edd78d1200-workload-socket\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv\" (UID: \"30ea822a-85f1-4db5-a3a2-b8edd78d1200\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv"
Oct 30 21:41:08.666044 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.665949    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4577b3de-b9e4-47a7-87e0-bcba13b11bbe-workload-certs\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj\" (UID: \"4577b3de-b9e4-47a7-87e0-bcba13b11bbe\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj"
Oct 30 21:41:08.666280 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.666255    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4577b3de-b9e4-47a7-87e0-bcba13b11bbe-credential-socket\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj\" (UID: \"4577b3de-b9e4-47a7-87e0-bcba13b11bbe\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj"
Oct 30 21:41:08.666406 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.666289    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30ea822a-85f1-4db5-a3a2-b8edd78d1200-istiod-ca-cert\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv\" (UID: \"30ea822a-85f1-4db5-a3a2-b8edd78d1200\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv"
Oct 30 21:41:08.667906 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.667883    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8"
Oct 30 21:41:08.668118 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.668094    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4577b3de-b9e4-47a7-87e0-bcba13b11bbe-istio-envoy\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj\" (UID: \"4577b3de-b9e4-47a7-87e0-bcba13b11bbe\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj"
Oct 30 21:41:08.668231 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.668200    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30ea822a-85f1-4db5-a3a2-b8edd78d1200-istio-envoy\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv\" (UID: \"30ea822a-85f1-4db5-a3a2-b8edd78d1200\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv"
Oct 30 21:41:08.668293 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.668209    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4577b3de-b9e4-47a7-87e0-bcba13b11bbe-istio-podinfo\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj\" (UID: \"4577b3de-b9e4-47a7-87e0-bcba13b11bbe\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj"
Oct 30 21:41:08.668342 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.668326    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30ea822a-85f1-4db5-a3a2-b8edd78d1200-istio-podinfo\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv\" (UID: \"30ea822a-85f1-4db5-a3a2-b8edd78d1200\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv"
Oct 30 21:41:08.678545 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.678520    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30ea822a-85f1-4db5-a3a2-b8edd78d1200-istio-token\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv\" (UID: \"30ea822a-85f1-4db5-a3a2-b8edd78d1200\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv"
Oct 30 21:41:08.678775 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.678754    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwr7d\" (UniqueName: \"kubernetes.io/projected/4577b3de-b9e4-47a7-87e0-bcba13b11bbe-kube-api-access-nwr7d\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj\" (UID: \"4577b3de-b9e4-47a7-87e0-bcba13b11bbe\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj"
Oct 30 21:41:08.678893 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.678876    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4577b3de-b9e4-47a7-87e0-bcba13b11bbe-istio-token\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj\" (UID: \"4577b3de-b9e4-47a7-87e0-bcba13b11bbe\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj"
Oct 30 21:41:08.680815 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.680791    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhkx9\" (UniqueName: \"kubernetes.io/projected/30ea822a-85f1-4db5-a3a2-b8edd78d1200-kube-api-access-xhkx9\") pod \"e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv\" (UID: \"30ea822a-85f1-4db5-a3a2-b8edd78d1200\") " pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv"
Oct 30 21:41:08.748334 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.747793    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv"
Oct 30 21:41:08.776191 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.776159    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz"]
Oct 30 21:41:08.781835 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.781802    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz"
Oct 30 21:41:08.796330 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.796300    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj"
Oct 30 21:41:08.809628 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.803152    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz"]
Oct 30 21:41:08.820674 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.820295    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv"]
Oct 30 21:41:08.838056 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.829602    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv"
Oct 30 21:41:08.849183 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.848401    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv"]
Oct 30 21:41:08.868603 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.868175    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74-istio-podinfo\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz\" (UID: \"ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz"
Oct 30 21:41:08.868603 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.868286    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pk7x\" (UniqueName: \"kubernetes.io/projected/ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74-kube-api-access-2pk7x\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz\" (UID: \"ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz"
Oct 30 21:41:08.868603 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.868316    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njq55\" (UniqueName: \"kubernetes.io/projected/aed33297-3e27-4d4a-a767-4191e8bc5127-kube-api-access-njq55\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv\" (UID: \"aed33297-3e27-4d4a-a767-4191e8bc5127\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv"
Oct 30 21:41:08.868603 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.868337    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74-workload-certs\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz\" (UID: \"ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz"
Oct 30 21:41:08.868603 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.868372    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aed33297-3e27-4d4a-a767-4191e8bc5127-workload-certs\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv\" (UID: \"aed33297-3e27-4d4a-a767-4191e8bc5127\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv"
Oct 30 21:41:08.868603 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.868410    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aed33297-3e27-4d4a-a767-4191e8bc5127-istio-token\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv\" (UID: \"aed33297-3e27-4d4a-a767-4191e8bc5127\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv"
Oct 30 21:41:08.868603 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.868439    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aed33297-3e27-4d4a-a767-4191e8bc5127-istiod-ca-cert\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv\" (UID: \"aed33297-3e27-4d4a-a767-4191e8bc5127\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv"
Oct 30 21:41:08.868603 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.868483    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74-workload-socket\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz\" (UID: \"ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz"
Oct 30 21:41:08.868603 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.868544    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aed33297-3e27-4d4a-a767-4191e8bc5127-istio-podinfo\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv\" (UID: \"aed33297-3e27-4d4a-a767-4191e8bc5127\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv"
Oct 30 21:41:08.868603 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.868604    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aed33297-3e27-4d4a-a767-4191e8bc5127-istio-data\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv\" (UID: \"aed33297-3e27-4d4a-a767-4191e8bc5127\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv"
Oct 30 21:41:08.869284 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.868636    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74-istio-data\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz\" (UID: \"ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz"
Oct 30 21:41:08.869284 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.868674    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aed33297-3e27-4d4a-a767-4191e8bc5127-credential-socket\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv\" (UID: \"aed33297-3e27-4d4a-a767-4191e8bc5127\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv"
Oct 30 21:41:08.869284 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.868754    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aed33297-3e27-4d4a-a767-4191e8bc5127-workload-socket\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv\" (UID: \"aed33297-3e27-4d4a-a767-4191e8bc5127\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv"
Oct 30 21:41:08.869284 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.868795    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74-istio-envoy\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz\" (UID: \"ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz"
Oct 30 21:41:08.869284 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.868853    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74-istiod-ca-cert\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz\" (UID: \"ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz"
Oct 30 21:41:08.869284 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.868887    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74-istio-token\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz\" (UID: \"ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz"
Oct 30 21:41:08.869284 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.868910    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aed33297-3e27-4d4a-a767-4191e8bc5127-istio-envoy\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv\" (UID: \"aed33297-3e27-4d4a-a767-4191e8bc5127\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv"
Oct 30 21:41:08.869284 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.868939    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74-credential-socket\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz\" (UID: \"ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz"
Oct 30 21:41:08.964235 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.964198    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8"]
Oct 30 21:41:08.965347 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:41:08.965310    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7679391_8575_43d7_b7f1_5b1ab0bcee5f.slice/crio-7d71a3dc31c1ffee7eaa24ca73eb24e042d8c060cb046b61f75b46bbb650df2f WatchSource:0}: Error finding container 7d71a3dc31c1ffee7eaa24ca73eb24e042d8c060cb046b61f75b46bbb650df2f: Status 404 returned error can't find the container with id 7d71a3dc31c1ffee7eaa24ca73eb24e042d8c060cb046b61f75b46bbb650df2f
Oct 30 21:41:08.968436 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.968395    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:41:08.968551 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.968487    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:41:08.968551 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.968535    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:41:08.971958 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.970241    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74-istio-podinfo\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz\" (UID: \"ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz"
Oct 30 21:41:08.971958 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.970303    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pk7x\" (UniqueName: \"kubernetes.io/projected/ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74-kube-api-access-2pk7x\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz\" (UID: \"ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz"
Oct 30 21:41:08.971958 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.970340    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njq55\" (UniqueName: \"kubernetes.io/projected/aed33297-3e27-4d4a-a767-4191e8bc5127-kube-api-access-njq55\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv\" (UID: \"aed33297-3e27-4d4a-a767-4191e8bc5127\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv"
Oct 30 21:41:08.971958 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.970396    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74-workload-certs\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz\" (UID: \"ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz"
Oct 30 21:41:08.971958 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.970425    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aed33297-3e27-4d4a-a767-4191e8bc5127-workload-certs\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv\" (UID: \"aed33297-3e27-4d4a-a767-4191e8bc5127\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv"
Oct 30 21:41:08.971958 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.970463    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aed33297-3e27-4d4a-a767-4191e8bc5127-istio-token\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv\" (UID: \"aed33297-3e27-4d4a-a767-4191e8bc5127\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv"
Oct 30 21:41:08.971958 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.970490    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aed33297-3e27-4d4a-a767-4191e8bc5127-istiod-ca-cert\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv\" (UID: \"aed33297-3e27-4d4a-a767-4191e8bc5127\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv"
Oct 30 21:41:08.971958 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.970534    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74-workload-socket\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz\" (UID: \"ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz"
Oct 30 21:41:08.971958 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.970586    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aed33297-3e27-4d4a-a767-4191e8bc5127-istio-podinfo\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv\" (UID: \"aed33297-3e27-4d4a-a767-4191e8bc5127\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv"
Oct 30 21:41:08.971958 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.970641    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aed33297-3e27-4d4a-a767-4191e8bc5127-istio-data\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv\" (UID: \"aed33297-3e27-4d4a-a767-4191e8bc5127\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv"
Oct 30 21:41:08.971958 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.970671    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74-istio-data\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz\" (UID: \"ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz"
Oct 30 21:41:08.971958 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.970706    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aed33297-3e27-4d4a-a767-4191e8bc5127-credential-socket\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv\" (UID: \"aed33297-3e27-4d4a-a767-4191e8bc5127\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv"
Oct 30 21:41:08.971958 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.970770    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aed33297-3e27-4d4a-a767-4191e8bc5127-workload-socket\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv\" (UID: \"aed33297-3e27-4d4a-a767-4191e8bc5127\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv"
Oct 30 21:41:08.971958 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.970804    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74-istio-envoy\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz\" (UID: \"ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz"
Oct 30 21:41:08.971958 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.970854    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74-istiod-ca-cert\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz\" (UID: \"ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz"
Oct 30 21:41:08.971958 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.970886    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74-istio-token\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz\" (UID: \"ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz"
Oct 30 21:41:08.971958 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.970912    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aed33297-3e27-4d4a-a767-4191e8bc5127-istio-envoy\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv\" (UID: \"aed33297-3e27-4d4a-a767-4191e8bc5127\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv"
Oct 30 21:41:08.971958 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.970952    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74-credential-socket\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz\" (UID: \"ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz"
Oct 30 21:41:08.971958 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.971385    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74-credential-socket\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz\" (UID: \"ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz"
Oct 30 21:41:08.971958 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.971677    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aed33297-3e27-4d4a-a767-4191e8bc5127-istio-data\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv\" (UID: \"aed33297-3e27-4d4a-a767-4191e8bc5127\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv"
Oct 30 21:41:08.973034 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.972469    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74-istiod-ca-cert\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz\" (UID: \"ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz"
Oct 30 21:41:08.974460 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.973301    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74-workload-certs\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz\" (UID: \"ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz"
Oct 30 21:41:08.974460 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.973958    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aed33297-3e27-4d4a-a767-4191e8bc5127-workload-certs\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv\" (UID: \"aed33297-3e27-4d4a-a767-4191e8bc5127\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv"
Oct 30 21:41:08.974460 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.974226    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aed33297-3e27-4d4a-a767-4191e8bc5127-credential-socket\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv\" (UID: \"aed33297-3e27-4d4a-a767-4191e8bc5127\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv"
Oct 30 21:41:08.974813 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.974501    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74-istio-data\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz\" (UID: \"ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz"
Oct 30 21:41:08.974882 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.974812    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aed33297-3e27-4d4a-a767-4191e8bc5127-workload-socket\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv\" (UID: \"aed33297-3e27-4d4a-a767-4191e8bc5127\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv"
Oct 30 21:41:08.977413 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.975552    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74-workload-socket\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz\" (UID: \"ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz"
Oct 30 21:41:08.977413 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.976850    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aed33297-3e27-4d4a-a767-4191e8bc5127-istiod-ca-cert\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv\" (UID: \"aed33297-3e27-4d4a-a767-4191e8bc5127\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv"
Oct 30 21:41:08.977657 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.977523    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aed33297-3e27-4d4a-a767-4191e8bc5127-istio-envoy\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv\" (UID: \"aed33297-3e27-4d4a-a767-4191e8bc5127\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv"
Oct 30 21:41:08.986497 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.979095    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74-istio-podinfo\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz\" (UID: \"ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz"
Oct 30 21:41:08.986497 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.981315    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aed33297-3e27-4d4a-a767-4191e8bc5127-istio-podinfo\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv\" (UID: \"aed33297-3e27-4d4a-a767-4191e8bc5127\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv"
Oct 30 21:41:08.986497 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.981859    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74-istio-envoy\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz\" (UID: \"ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz"
Oct 30 21:41:08.986497 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.985044    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74-istio-token\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz\" (UID: \"ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz"
Oct 30 21:41:08.986784 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.986646    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aed33297-3e27-4d4a-a767-4191e8bc5127-istio-token\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv\" (UID: \"aed33297-3e27-4d4a-a767-4191e8bc5127\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv"
Oct 30 21:41:08.989128 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.989104    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pk7x\" (UniqueName: \"kubernetes.io/projected/ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74-kube-api-access-2pk7x\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz\" (UID: \"ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz"
Oct 30 21:41:08.989286 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:08.989268    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-njq55\" (UniqueName: \"kubernetes.io/projected/aed33297-3e27-4d4a-a767-4191e8bc5127-kube-api-access-njq55\") pod \"e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv\" (UID: \"aed33297-3e27-4d4a-a767-4191e8bc5127\") " pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv"
Oct 30 21:41:09.074510 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:09.074480    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv"]
Oct 30 21:41:09.076368 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:41:09.076318    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30ea822a_85f1_4db5_a3a2_b8edd78d1200.slice/crio-59e71aa45ab38a5e1a2b02fcba4ea14c4391970e9bed69c55de03cad879144c2 WatchSource:0}: Error finding container 59e71aa45ab38a5e1a2b02fcba4ea14c4391970e9bed69c55de03cad879144c2: Status 404 returned error can't find the container with id 59e71aa45ab38a5e1a2b02fcba4ea14c4391970e9bed69c55de03cad879144c2
Oct 30 21:41:09.079311 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:09.079267    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:41:09.079432 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:09.079413    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:41:09.079490 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:09.079462    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:41:09.119099 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:09.118870    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz"
Oct 30 21:41:09.156615 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:09.156583    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj"]
Oct 30 21:41:09.158974 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:41:09.158938    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4577b3de_b9e4_47a7_87e0_bcba13b11bbe.slice/crio-57c7c539da9fc4d0ce4e272f8504d17cfc38051e78794d02ced2ca7bc8643e96 WatchSource:0}: Error finding container 57c7c539da9fc4d0ce4e272f8504d17cfc38051e78794d02ced2ca7bc8643e96: Status 404 returned error can't find the container with id 57c7c539da9fc4d0ce4e272f8504d17cfc38051e78794d02ced2ca7bc8643e96
Oct 30 21:41:09.162403 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:09.162331    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:41:09.162512 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:09.162432    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:41:09.162512 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:09.162488    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:41:09.164205 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:09.164132    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv"
Oct 30 21:41:09.475218 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:09.475179    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz"]
Oct 30 21:41:09.495778 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:09.495727    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:41:09.495990 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:09.495957    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:41:09.496109 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:09.496083    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:41:09.755611 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:09.752353    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8" event={"ID":"e7679391-8575-43d7-b7f1-5b1ab0bcee5f","Type":"ContainerStarted","Data":"b026d760849187704d451a5830315d5a4a7556c362335e17873b515117d83d93"}
Oct 30 21:41:09.755611 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:09.752446    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8" event={"ID":"e7679391-8575-43d7-b7f1-5b1ab0bcee5f","Type":"ContainerStarted","Data":"7d71a3dc31c1ffee7eaa24ca73eb24e042d8c060cb046b61f75b46bbb650df2f"}
Oct 30 21:41:09.755611 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:09.754874    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj" event={"ID":"4577b3de-b9e4-47a7-87e0-bcba13b11bbe","Type":"ContainerStarted","Data":"e61b8147c0670a991ce18515008b69f74f5279f964883befb24bdc04a625a2b7"}
Oct 30 21:41:09.755611 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:09.754924    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj" event={"ID":"4577b3de-b9e4-47a7-87e0-bcba13b11bbe","Type":"ContainerStarted","Data":"57c7c539da9fc4d0ce4e272f8504d17cfc38051e78794d02ced2ca7bc8643e96"}
Oct 30 21:41:09.762647 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:09.762617    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv"]
Oct 30 21:41:09.773428 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:09.771091    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz" event={"ID":"ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74","Type":"ContainerStarted","Data":"eb74413af42ee9771451cc538cb1e339d53b7020e6d1fa2ef1baf2e878f9817b"}
Oct 30 21:41:09.773428 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:09.771155    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz" event={"ID":"ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74","Type":"ContainerStarted","Data":"20bc2e525eb6ca86d1306ddee03586f0f76681d114d0f6401decfd69f90c7148"}
Oct 30 21:41:09.777391 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:09.774951    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:41:09.777391 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:09.775066    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:41:09.777391 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:09.775112    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:41:09.777733 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:09.777387    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv" event={"ID":"30ea822a-85f1-4db5-a3a2-b8edd78d1200","Type":"ContainerStarted","Data":"ca5eafd98d90fdc3a6c340da0657c9c5ef55304bd7ea8e22277dc6bb83be92d0"}
Oct 30 21:41:09.777733 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:09.777533    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv" event={"ID":"30ea822a-85f1-4db5-a3a2-b8edd78d1200","Type":"ContainerStarted","Data":"59e71aa45ab38a5e1a2b02fcba4ea14c4391970e9bed69c55de03cad879144c2"}
Oct 30 21:41:09.809391 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:09.794978    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8" podStartSLOduration=1.7949547479999999 podStartE2EDuration="1.794954748s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:41:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:41:09.782009895 +0000 UTC m=+782.583822382" watchObservedRunningTime="2026-10-30 21:41:09.794954748 +0000 UTC m=+782.596767204"
Oct 30 21:41:09.809391 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:09.798744    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj"
Oct 30 21:41:09.895418 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:09.879104    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj" podStartSLOduration=1.879080002 podStartE2EDuration="1.879080002s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:41:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:41:09.86522909 +0000 UTC m=+782.667041568" watchObservedRunningTime="2026-10-30 21:41:09.879080002 +0000 UTC m=+782.680892468"
Oct 30 21:41:09.907417 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:09.902343    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz" podStartSLOduration=1.902323993 podStartE2EDuration="1.902323993s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:41:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:41:09.901041459 +0000 UTC m=+782.702853924" watchObservedRunningTime="2026-10-30 21:41:09.902323993 +0000 UTC m=+782.704136458"
Oct 30 21:41:09.986380 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:09.986308    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv" podStartSLOduration=1.98627623 podStartE2EDuration="1.98627623s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:41:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:41:09.940673457 +0000 UTC m=+782.742485934" watchObservedRunningTime="2026-10-30 21:41:09.98627623 +0000 UTC m=+782.788088695"
Oct 30 21:41:10.154433 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:10.153757    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz"
Oct 30 21:41:10.183550 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:10.183182    5032 patch_prober.go:29] interesting pod/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Oct 30 21:41:10.183550 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:10.183241    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz" podUID="ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Oct 30 21:41:10.709839 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:10.699747    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8"
Oct 30 21:41:10.709839 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:10.699875    5032 patch_prober.go:29] interesting pod/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Oct 30 21:41:10.709839 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:10.699916    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8" podUID="e7679391-8575-43d7-b7f1-5b1ab0bcee5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Oct 30 21:41:10.766432 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:10.763989    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv"
Oct 30 21:41:10.766432 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:10.765860    5032 patch_prober.go:29] interesting pod/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Oct 30 21:41:10.766432 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:10.765912    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv" podUID="30ea822a-85f1-4db5-a3a2-b8edd78d1200" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Oct 30 21:41:10.800160 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:10.799816    5032 patch_prober.go:29] interesting pod/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Oct 30 21:41:10.800160 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:10.799874    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj" podUID="4577b3de-b9e4-47a7-87e0-bcba13b11bbe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Oct 30 21:41:10.925420 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:10.912268    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv" event={"ID":"aed33297-3e27-4d4a-a767-4191e8bc5127","Type":"ContainerStarted","Data":"996f7341f0007a4f636c0f27df8ba37ce893d6675da2f33bffc987c040b54633"}
Oct 30 21:41:10.941346 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:10.941294    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv" event={"ID":"aed33297-3e27-4d4a-a767-4191e8bc5127","Type":"ContainerStarted","Data":"09224aafbc69a125d0efe7f890ef8f01c13ecffe3b072db75c11066ffc7402cf"}
Oct 30 21:41:10.959577 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:10.959503    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv" podStartSLOduration=2.959480526 podStartE2EDuration="2.959480526s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:41:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:41:10.941172914 +0000 UTC m=+783.742985404" watchObservedRunningTime="2026-10-30 21:41:10.959480526 +0000 UTC m=+783.761292991"
Oct 30 21:41:11.166306 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:11.166211    5032 patch_prober.go:29] interesting pod/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Oct 30 21:41:11.167917 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:11.167883    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz" podUID="ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Oct 30 21:41:11.168431 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:11.166691    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv"
Oct 30 21:41:11.168583 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:11.166767    5032 patch_prober.go:29] interesting pod/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body=
Oct 30 21:41:11.168714 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:11.168696    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv" podUID="aed33297-3e27-4d4a-a767-4191e8bc5127" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused"
Oct 30 21:41:11.695978 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:11.695494    5032 patch_prober.go:29] interesting pod/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Oct 30 21:41:11.695978 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:11.695563    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8" podUID="e7679391-8575-43d7-b7f1-5b1ab0bcee5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Oct 30 21:41:11.760430 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:11.759538    5032 patch_prober.go:29] interesting pod/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Oct 30 21:41:11.760430 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:11.759606    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv" podUID="30ea822a-85f1-4db5-a3a2-b8edd78d1200" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Oct 30 21:41:11.815549 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:11.815510    5032 patch_prober.go:29] interesting pod/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Oct 30 21:41:11.816180 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:11.816146    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj" podUID="4577b3de-b9e4-47a7-87e0-bcba13b11bbe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Oct 30 21:41:11.888517 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:11.888464    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx" podUID="34fdf57d-86f6-474f-a4f6-811e5063c405" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:11.944386 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:11.926742    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc" podUID="23b97dbb-8488-4c44-a810-882ea18ed919" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": context deadline exceeded"
Oct 30 21:41:12.124384 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:12.123794    5032 patch_prober.go:29] interesting pod/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Oct 30 21:41:12.124384 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:12.123885    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz" podUID="ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Oct 30 21:41:12.210730 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:12.210690    5032 patch_prober.go:29] interesting pod/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body=
Oct 30 21:41:12.211005 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:12.210975    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv" podUID="aed33297-3e27-4d4a-a767-4191e8bc5127" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused"
Oct 30 21:41:12.299389 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:12.291423    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw" podUID="914f7544-6ef8-4496-b98d-b7ebc4123cda" containerName="main" probeResult="failure" output="Get \"https://10.132.0.86:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:12.360043 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:12.359995    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw" podUID="2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded"
Oct 30 21:41:12.407326 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:12.407175    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg" podUID="2d0ea058-b6ca-4181-8823-2b719883e545" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:12.673896 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:12.673788    5032 patch_prober.go:29] interesting pod/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Oct 30 21:41:12.673896 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:12.673858    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8" podUID="e7679391-8575-43d7-b7f1-5b1ab0bcee5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Oct 30 21:41:12.701393 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:12.699541    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj" podUID="d0ff5023-fd3f-4734-94c6-76b666010adf" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:12.756420 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:12.750693    5032 patch_prober.go:29] interesting pod/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Oct 30 21:41:12.756420 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:12.750749    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv" podUID="30ea822a-85f1-4db5-a3a2-b8edd78d1200" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Oct 30 21:41:12.799397 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:12.797898    5032 patch_prober.go:29] interesting pod/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Oct 30 21:41:12.799397 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:12.797956    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj" podUID="4577b3de-b9e4-47a7-87e0-bcba13b11bbe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Oct 30 21:41:12.909447 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:12.907756    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx" podUID="34fdf57d-86f6-474f-a4f6-811e5063c405" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": context deadline exceeded"
Oct 30 21:41:12.909447 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:12.908423    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx"
Oct 30 21:41:12.998521 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:12.984056    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc" podUID="23b97dbb-8488-4c44-a810-882ea18ed919" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:12.998521 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:12.984865    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc"
Oct 30 21:41:13.199435 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:13.169824    5032 patch_prober.go:29] interesting pod/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Oct 30 21:41:13.199765 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:13.199717    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz" podUID="ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Oct 30 21:41:13.226387 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:13.202113    5032 patch_prober.go:29] interesting pod/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body=
Oct 30 21:41:13.226387 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:13.202164    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv" podUID="aed33297-3e27-4d4a-a767-4191e8bc5127" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused"
Oct 30 21:41:13.376215 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:13.333528    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw" podUID="914f7544-6ef8-4496-b98d-b7ebc4123cda" containerName="main" probeResult="failure" output="Get \"https://10.132.0.86:8000/ready\": context deadline exceeded"
Oct 30 21:41:13.376215 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:13.333965    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw"
Oct 30 21:41:13.436452 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:13.436406    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg" podUID="2d0ea058-b6ca-4181-8823-2b719883e545" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": context deadline exceeded"
Oct 30 21:41:13.486050 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:13.437293    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw" podUID="2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:13.749386 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:13.741536    5032 patch_prober.go:29] interesting pod/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Oct 30 21:41:13.858383 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:13.770375    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8" podUID="e7679391-8575-43d7-b7f1-5b1ab0bcee5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Oct 30 21:41:13.998527 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:13.997132    5032 patch_prober.go:29] interesting pod/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Oct 30 21:41:13.998527 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:13.997196    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv" podUID="30ea822a-85f1-4db5-a3a2-b8edd78d1200" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Oct 30 21:41:14.050469 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:14.047537    5032 patch_prober.go:29] interesting pod/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Oct 30 21:41:14.050469 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:14.047622    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj" podUID="4577b3de-b9e4-47a7-87e0-bcba13b11bbe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Oct 30 21:41:14.050469 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:14.049797    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc" podUID="23b97dbb-8488-4c44-a810-882ea18ed919" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:14.195420 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:14.155307    5032 patch_prober.go:29] interesting pod/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Oct 30 21:41:14.195420 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:14.155385    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz" podUID="ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Oct 30 21:41:14.195420 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:14.180558    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx" podUID="34fdf57d-86f6-474f-a4f6-811e5063c405" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:14.449443 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:14.438427    5032 patch_prober.go:29] interesting pod/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body=
Oct 30 21:41:14.449443 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:14.438486    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv" podUID="aed33297-3e27-4d4a-a767-4191e8bc5127" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused"
Oct 30 21:41:14.500476 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:14.499640    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw" podUID="914f7544-6ef8-4496-b98d-b7ebc4123cda" containerName="main" probeResult="failure" output="Get \"https://10.132.0.86:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:14.539859 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:14.539813    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw" podUID="2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:14.557568 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:14.541985    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg" podUID="2d0ea058-b6ca-4181-8823-2b719883e545" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": context deadline exceeded"
Oct 30 21:41:14.557568 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:41:14.545936    5032 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.115s"
Oct 30 21:41:14.557568 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:14.546248    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg"
Oct 30 21:41:14.557568 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:14.546321    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw"
Oct 30 21:41:14.658386 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:14.650191    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj" podUID="d0ff5023-fd3f-4734-94c6-76b666010adf" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:14.734208 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:14.734171    5032 patch_prober.go:29] interesting pod/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Oct 30 21:41:14.915483 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:14.891712    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8" podUID="e7679391-8575-43d7-b7f1-5b1ab0bcee5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Oct 30 21:41:14.915483 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:14.780744    5032 patch_prober.go:29] interesting pod/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Oct 30 21:41:14.915483 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:14.891896    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv" podUID="30ea822a-85f1-4db5-a3a2-b8edd78d1200" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Oct 30 21:41:14.915483 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:14.807467    5032 patch_prober.go:29] interesting pod/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Oct 30 21:41:14.915483 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:14.891940    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj" podUID="4577b3de-b9e4-47a7-87e0-bcba13b11bbe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Oct 30 21:41:15.079571 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:15.079467    5032 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj" podUID="d0ff5023-fd3f-4734-94c6-76b666010adf" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:15.153612 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:15.153572    5032 patch_prober.go:29] interesting pod/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Oct 30 21:41:15.174602 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:15.174553    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz" podUID="ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Oct 30 21:41:15.234482 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:15.225877    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc" podUID="23b97dbb-8488-4c44-a810-882ea18ed919" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:15.440437 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:15.293373    5032 patch_prober.go:29] interesting pod/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body=
Oct 30 21:41:15.440437 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:15.293433    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv" podUID="aed33297-3e27-4d4a-a767-4191e8bc5127" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused"
Oct 30 21:41:15.440437 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:15.344329    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx" podUID="34fdf57d-86f6-474f-a4f6-811e5063c405" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": context deadline exceeded"
Oct 30 21:41:15.511461 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:15.510595    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw" podUID="914f7544-6ef8-4496-b98d-b7ebc4123cda" containerName="main" probeResult="failure" output="Get \"https://10.132.0.86:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:15.568391 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:15.546884    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw" podUID="2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:15.570479 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:15.570410    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg" podUID="2d0ea058-b6ca-4181-8823-2b719883e545" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:15.706469 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:15.666465    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj" podUID="d0ff5023-fd3f-4734-94c6-76b666010adf" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": context deadline exceeded"
Oct 30 21:41:15.706469 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:15.675465    5032 patch_prober.go:29] interesting pod/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Oct 30 21:41:15.706469 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:15.675523    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8" podUID="e7679391-8575-43d7-b7f1-5b1ab0bcee5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Oct 30 21:41:15.792384 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:15.754331    5032 patch_prober.go:29] interesting pod/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Oct 30 21:41:15.890927 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:15.890890    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv" podUID="30ea822a-85f1-4db5-a3a2-b8edd78d1200" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Oct 30 21:41:15.891165 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:15.890513    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj"
Oct 30 21:41:15.892152 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:15.840711    5032 patch_prober.go:29] interesting pod/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Oct 30 21:41:15.931802 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:15.931763    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj" podUID="4577b3de-b9e4-47a7-87e0-bcba13b11bbe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Oct 30 21:41:16.136448 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:16.125863    5032 patch_prober.go:29] interesting pod/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Oct 30 21:41:16.136448 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:16.125928    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz" podUID="ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Oct 30 21:41:16.197541 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:16.197489    5032 patch_prober.go:29] interesting pod/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body=
Oct 30 21:41:16.197818 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:16.197789    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv" podUID="aed33297-3e27-4d4a-a767-4191e8bc5127" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused"
Oct 30 21:41:16.239388 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:16.236531    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc" podUID="23b97dbb-8488-4c44-a810-882ea18ed919" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:16.371410 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:16.363457    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx" podUID="34fdf57d-86f6-474f-a4f6-811e5063c405" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": context deadline exceeded"
Oct 30 21:41:16.555387 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:16.523543    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw" podUID="914f7544-6ef8-4496-b98d-b7ebc4123cda" containerName="main" probeResult="failure" output="Get \"https://10.132.0.86:8000/ready\": context deadline exceeded"
Oct 30 21:41:16.581501 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:16.581458    5032 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw" podUID="2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/health\": context deadline exceeded"
Oct 30 21:41:16.582052 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:16.582015    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg" podUID="2d0ea058-b6ca-4181-8823-2b719883e545" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": context deadline exceeded"
Oct 30 21:41:16.612393 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:16.602509    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw" podUID="2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:16.714431 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:16.693094    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj" podUID="d0ff5023-fd3f-4734-94c6-76b666010adf" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:16.733841 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:16.733192    5032 patch_prober.go:29] interesting pod/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Oct 30 21:41:16.733841 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:16.733273    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8" podUID="e7679391-8575-43d7-b7f1-5b1ab0bcee5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Oct 30 21:41:16.800204 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:16.800165    5032 patch_prober.go:29] interesting pod/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Oct 30 21:41:16.800508 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:16.800478    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj" podUID="4577b3de-b9e4-47a7-87e0-bcba13b11bbe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Oct 30 21:41:16.932439 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:16.929935    5032 patch_prober.go:29] interesting pod/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Oct 30 21:41:16.932439 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:16.930023    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv" podUID="30ea822a-85f1-4db5-a3a2-b8edd78d1200" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Oct 30 21:41:17.119830 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:17.119791    5032 patch_prober.go:29] interesting pod/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Oct 30 21:41:17.120099 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:17.120058    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz" podUID="ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Oct 30 21:41:17.190663 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:17.175424    5032 patch_prober.go:29] interesting pod/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body=
Oct 30 21:41:17.190663 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:17.175487    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv" podUID="aed33297-3e27-4d4a-a767-4191e8bc5127" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused"
Oct 30 21:41:17.290425 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:17.264374    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc" podUID="23b97dbb-8488-4c44-a810-882ea18ed919" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": context deadline exceeded"
Oct 30 21:41:17.373907 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:17.373857    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx" podUID="34fdf57d-86f6-474f-a4f6-811e5063c405" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:17.536381 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:17.526510    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw" podUID="914f7544-6ef8-4496-b98d-b7ebc4123cda" containerName="main" probeResult="failure" output="Get \"https://10.132.0.86:8000/ready\": context deadline exceeded"
Oct 30 21:41:17.705701 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:17.704440    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj" podUID="d0ff5023-fd3f-4734-94c6-76b666010adf" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:17.737397 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:17.716481    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg" podUID="2d0ea058-b6ca-4181-8823-2b719883e545" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:17.741392 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:17.740944    5032 patch_prober.go:29] interesting pod/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Oct 30 21:41:17.741392 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:17.740997    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8" podUID="e7679391-8575-43d7-b7f1-5b1ab0bcee5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Oct 30 21:41:17.840458 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:17.827943    5032 patch_prober.go:29] interesting pod/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Oct 30 21:41:17.840458 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:17.828021    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj" podUID="4577b3de-b9e4-47a7-87e0-bcba13b11bbe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Oct 30 21:41:17.843563 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:17.843182    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw" podUID="2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:17.866677 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:17.866641    5032 patch_prober.go:29] interesting pod/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Oct 30 21:41:17.867009 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:17.866980    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv" podUID="30ea822a-85f1-4db5-a3a2-b8edd78d1200" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Oct 30 21:41:18.347409 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:18.347354    5032 patch_prober.go:29] interesting pod/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Oct 30 21:41:18.348066 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:18.348021    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz" podUID="ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Oct 30 21:41:18.562842 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:18.348711    5032 patch_prober.go:29] interesting pod/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body=
Oct 30 21:41:18.563160 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:18.563125    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv" podUID="aed33297-3e27-4d4a-a767-4191e8bc5127" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused"
Oct 30 21:41:18.563338 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:18.406426    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc" podUID="23b97dbb-8488-4c44-a810-882ea18ed919" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": context deadline exceeded"
Oct 30 21:41:18.573540 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:18.573494    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx" podUID="34fdf57d-86f6-474f-a4f6-811e5063c405" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:18.590687 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:18.590632    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw" podUID="914f7544-6ef8-4496-b98d-b7ebc4123cda" containerName="main" probeResult="failure" output="Get \"https://10.132.0.86:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:18.749396 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:18.747440    5032 patch_prober.go:29] interesting pod/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Oct 30 21:41:18.749396 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:18.747552    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8" podUID="e7679391-8575-43d7-b7f1-5b1ab0bcee5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Oct 30 21:41:18.839521 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:18.792434    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg" podUID="2d0ea058-b6ca-4181-8823-2b719883e545" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:18.842765 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:18.842733    5032 patch_prober.go:29] interesting pod/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Oct 30 21:41:18.843049 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:18.842998    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj" podUID="4577b3de-b9e4-47a7-87e0-bcba13b11bbe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Oct 30 21:41:18.843440 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:18.843415    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw" podUID="2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded"
Oct 30 21:41:18.940837 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:18.888166    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj" podUID="d0ff5023-fd3f-4734-94c6-76b666010adf" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": context deadline exceeded"
Oct 30 21:41:18.947930 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:18.901180    5032 patch_prober.go:29] interesting pod/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Oct 30 21:41:18.957723 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:18.957668    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv" podUID="30ea822a-85f1-4db5-a3a2-b8edd78d1200" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Oct 30 21:41:18.965834 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:18.901518    5032 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc" podUID="23b97dbb-8488-4c44-a810-882ea18ed919" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:19.122580 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:19.120684    5032 patch_prober.go:29] interesting pod/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Oct 30 21:41:19.218387 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:19.195170    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz" podUID="ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Oct 30 21:41:19.279385 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:19.224876    5032 patch_prober.go:29] interesting pod/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body=
Oct 30 21:41:19.352478 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:19.352439    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv" podUID="aed33297-3e27-4d4a-a767-4191e8bc5127" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused"
Oct 30 21:41:19.573978 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:19.573938    5032 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx" podUID="34fdf57d-86f6-474f-a4f6-811e5063c405" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:19.578529 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:19.575354    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx" podUID="34fdf57d-86f6-474f-a4f6-811e5063c405" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:19.585099 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:19.585064    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc" podUID="23b97dbb-8488-4c44-a810-882ea18ed919" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:19.599597 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:19.599557    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw" podUID="914f7544-6ef8-4496-b98d-b7ebc4123cda" containerName="main" probeResult="failure" output="Get \"https://10.132.0.86:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:19.607323 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:19.607278    5032 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg" podUID="2d0ea058-b6ca-4181-8823-2b719883e545" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:19.678387 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:19.678337    5032 patch_prober.go:29] interesting pod/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Oct 30 21:41:19.678661 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:19.678631    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8" podUID="e7679391-8575-43d7-b7f1-5b1ab0bcee5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Oct 30 21:41:19.865029 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:19.862737    5032 patch_prober.go:29] interesting pod/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Oct 30 21:41:19.865029 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:19.862795    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj" podUID="4577b3de-b9e4-47a7-87e0-bcba13b11bbe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Oct 30 21:41:19.894137 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:41:19.894103    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 21:41:19.920760 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:19.920735    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw"
Oct 30 21:41:19.921010 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:19.920999    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc"
Oct 30 21:41:19.921128 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:19.921118    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw"
Oct 30 21:41:19.921228 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:19.921220    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg"
Oct 30 21:41:19.921324 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:19.921315    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx"
Oct 30 21:41:19.948529 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:19.948469    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj" podUID="d0ff5023-fd3f-4734-94c6-76b666010adf" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:20.081245 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:20.081220    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj"
Oct 30 21:41:20.142743 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:20.127816    5032 patch_prober.go:29] interesting pod/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Oct 30 21:41:20.143054 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:20.143017    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz" podUID="ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Oct 30 21:41:20.179221 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:20.179189    5032 patch_prober.go:29] interesting pod/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body=
Oct 30 21:41:20.179541 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:20.179498    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv" podUID="aed33297-3e27-4d4a-a767-4191e8bc5127" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused"
Oct 30 21:41:20.597385 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:20.584197    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv"
Oct 30 21:41:20.597385 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:20.585864    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv"
Oct 30 21:41:20.597385 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:20.594048    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv"
Oct 30 21:41:20.981153 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:20.981124    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8"
Oct 30 21:41:20.982941 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:20.982913    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8"
Oct 30 21:41:20.987382 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:20.984750    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8"
Oct 30 21:41:21.145386 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:21.137702    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj"
Oct 30 21:41:21.145386 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:21.143392    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj"
Oct 30 21:41:21.149338 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:21.149249    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj"
Oct 30 21:41:21.771751 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:21.771709    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz"
Oct 30 21:41:21.773635 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:21.773580    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz"
Oct 30 21:41:21.774993 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:21.774969    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz"
Oct 30 21:41:21.808500 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:21.808464    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv"
Oct 30 21:41:21.809128 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:21.809102    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv"
Oct 30 21:41:21.810788 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:21.810765    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv"
Oct 30 21:41:23.633382 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:23.631464    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc"]
Oct 30 21:41:23.818050 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:23.797798    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc"]
Oct 30 21:41:23.818050 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:23.797935    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc"
Oct 30 21:41:23.970571 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:23.970530    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b92d898-765d-445d-9056-9f256030162e-istio-podinfo\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc\" (UID: \"3b92d898-765d-445d-9056-9f256030162e\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc"
Oct 30 21:41:23.970873 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:23.970851    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b92d898-765d-445d-9056-9f256030162e-workload-socket\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc\" (UID: \"3b92d898-765d-445d-9056-9f256030162e\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc"
Oct 30 21:41:23.971058 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:23.971038    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hvh5\" (UniqueName: \"kubernetes.io/projected/3b92d898-765d-445d-9056-9f256030162e-kube-api-access-7hvh5\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc\" (UID: \"3b92d898-765d-445d-9056-9f256030162e\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc"
Oct 30 21:41:23.971224 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:23.971207    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b92d898-765d-445d-9056-9f256030162e-credential-socket\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc\" (UID: \"3b92d898-765d-445d-9056-9f256030162e\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc"
Oct 30 21:41:23.971346 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:23.971331    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b92d898-765d-445d-9056-9f256030162e-istio-envoy\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc\" (UID: \"3b92d898-765d-445d-9056-9f256030162e\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc"
Oct 30 21:41:23.971556 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:23.971536    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b92d898-765d-445d-9056-9f256030162e-istiod-ca-cert\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc\" (UID: \"3b92d898-765d-445d-9056-9f256030162e\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc"
Oct 30 21:41:23.971719 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:23.971701    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b92d898-765d-445d-9056-9f256030162e-workload-certs\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc\" (UID: \"3b92d898-765d-445d-9056-9f256030162e\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc"
Oct 30 21:41:23.972037 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:23.971989    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b92d898-765d-445d-9056-9f256030162e-istio-data\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc\" (UID: \"3b92d898-765d-445d-9056-9f256030162e\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc"
Oct 30 21:41:23.972175 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:23.972157    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b92d898-765d-445d-9056-9f256030162e-istio-token\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc\" (UID: \"3b92d898-765d-445d-9056-9f256030162e\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc"
Oct 30 21:41:23.982579 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:23.982548    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww"]
Oct 30 21:41:24.072382 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.067826    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww"]
Oct 30 21:41:24.072382 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.067885    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s"]
Oct 30 21:41:24.079141 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.079108    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww"
Oct 30 21:41:24.110817 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.110199    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s"
Oct 30 21:41:24.116152 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.116122    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s"]
Oct 30 21:41:24.148396 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.133026    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b92d898-765d-445d-9056-9f256030162e-istio-podinfo\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc\" (UID: \"3b92d898-765d-445d-9056-9f256030162e\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc"
Oct 30 21:41:24.217386 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.188718    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b92d898-765d-445d-9056-9f256030162e-workload-socket\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc\" (UID: \"3b92d898-765d-445d-9056-9f256030162e\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc"
Oct 30 21:41:24.217386 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.192520    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hvh5\" (UniqueName: \"kubernetes.io/projected/3b92d898-765d-445d-9056-9f256030162e-kube-api-access-7hvh5\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc\" (UID: \"3b92d898-765d-445d-9056-9f256030162e\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc"
Oct 30 21:41:24.217386 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.192593    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b92d898-765d-445d-9056-9f256030162e-workload-socket\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc\" (UID: \"3b92d898-765d-445d-9056-9f256030162e\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc"
Oct 30 21:41:24.217386 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.192630    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b92d898-765d-445d-9056-9f256030162e-credential-socket\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc\" (UID: \"3b92d898-765d-445d-9056-9f256030162e\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc"
Oct 30 21:41:24.217386 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.192660    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b92d898-765d-445d-9056-9f256030162e-istio-envoy\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc\" (UID: \"3b92d898-765d-445d-9056-9f256030162e\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc"
Oct 30 21:41:24.217386 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.192694    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b92d898-765d-445d-9056-9f256030162e-istiod-ca-cert\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc\" (UID: \"3b92d898-765d-445d-9056-9f256030162e\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc"
Oct 30 21:41:24.217386 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.192759    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b92d898-765d-445d-9056-9f256030162e-workload-certs\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc\" (UID: \"3b92d898-765d-445d-9056-9f256030162e\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc"
Oct 30 21:41:24.217386 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.183122    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b92d898-765d-445d-9056-9f256030162e-istio-podinfo\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc\" (UID: \"3b92d898-765d-445d-9056-9f256030162e\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc"
Oct 30 21:41:24.217386 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.193238    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b92d898-765d-445d-9056-9f256030162e-credential-socket\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc\" (UID: \"3b92d898-765d-445d-9056-9f256030162e\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc"
Oct 30 21:41:24.217386 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.161085    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm"]
Oct 30 21:41:24.217386 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.216549    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b92d898-765d-445d-9056-9f256030162e-workload-certs\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc\" (UID: \"3b92d898-765d-445d-9056-9f256030162e\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc"
Oct 30 21:41:24.217386 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.217198    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b92d898-765d-445d-9056-9f256030162e-istiod-ca-cert\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc\" (UID: \"3b92d898-765d-445d-9056-9f256030162e\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc"
Oct 30 21:41:24.217386 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.217317    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b92d898-765d-445d-9056-9f256030162e-istio-data\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc\" (UID: \"3b92d898-765d-445d-9056-9f256030162e\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc"
Oct 30 21:41:24.225905 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.225825    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b92d898-765d-445d-9056-9f256030162e-istio-envoy\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc\" (UID: \"3b92d898-765d-445d-9056-9f256030162e\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc"
Oct 30 21:41:24.230383 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.226658    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b92d898-765d-445d-9056-9f256030162e-istio-data\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc\" (UID: \"3b92d898-765d-445d-9056-9f256030162e\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc"
Oct 30 21:41:24.269391 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.251457    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc" podUID="23b97dbb-8488-4c44-a810-882ea18ed919" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:24.329288 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.217349    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b92d898-765d-445d-9056-9f256030162e-istio-token\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc\" (UID: \"3b92d898-765d-445d-9056-9f256030162e\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc"
Oct 30 21:41:24.370655 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.369014    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b92d898-765d-445d-9056-9f256030162e-istio-token\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc\" (UID: \"3b92d898-765d-445d-9056-9f256030162e\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc"
Oct 30 21:41:24.385804 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.385763    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm"
Oct 30 21:41:24.389383 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.386792    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm"]
Oct 30 21:41:24.448380 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.437395    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/222c96b4-9c61-49c0-a3ed-450ea7d17b67-istiod-ca-cert\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s\" (UID: \"222c96b4-9c61-49c0-a3ed-450ea7d17b67\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s"
Oct 30 21:41:24.448380 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.437461    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/222c96b4-9c61-49c0-a3ed-450ea7d17b67-istio-data\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s\" (UID: \"222c96b4-9c61-49c0-a3ed-450ea7d17b67\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s"
Oct 30 21:41:24.448380 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.437489    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spnlf\" (UniqueName: \"kubernetes.io/projected/222c96b4-9c61-49c0-a3ed-450ea7d17b67-kube-api-access-spnlf\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s\" (UID: \"222c96b4-9c61-49c0-a3ed-450ea7d17b67\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s"
Oct 30 21:41:24.448380 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.437541    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63f89cda-14de-416c-9263-00ffdb8c9ec9-istio-token\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww\" (UID: \"63f89cda-14de-416c-9263-00ffdb8c9ec9\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww"
Oct 30 21:41:24.448380 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.437593    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/222c96b4-9c61-49c0-a3ed-450ea7d17b67-workload-socket\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s\" (UID: \"222c96b4-9c61-49c0-a3ed-450ea7d17b67\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s"
Oct 30 21:41:24.448380 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.437729    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63f89cda-14de-416c-9263-00ffdb8c9ec9-credential-socket\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww\" (UID: \"63f89cda-14de-416c-9263-00ffdb8c9ec9\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww"
Oct 30 21:41:24.448380 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.437823    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/222c96b4-9c61-49c0-a3ed-450ea7d17b67-istio-token\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s\" (UID: \"222c96b4-9c61-49c0-a3ed-450ea7d17b67\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s"
Oct 30 21:41:24.448380 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.437888    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63f89cda-14de-416c-9263-00ffdb8c9ec9-workload-certs\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww\" (UID: \"63f89cda-14de-416c-9263-00ffdb8c9ec9\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww"
Oct 30 21:41:24.448380 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.437932    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/222c96b4-9c61-49c0-a3ed-450ea7d17b67-istio-podinfo\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s\" (UID: \"222c96b4-9c61-49c0-a3ed-450ea7d17b67\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s"
Oct 30 21:41:24.448380 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.437962    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63f89cda-14de-416c-9263-00ffdb8c9ec9-istiod-ca-cert\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww\" (UID: \"63f89cda-14de-416c-9263-00ffdb8c9ec9\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww"
Oct 30 21:41:24.448380 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.438002    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/222c96b4-9c61-49c0-a3ed-450ea7d17b67-workload-certs\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s\" (UID: \"222c96b4-9c61-49c0-a3ed-450ea7d17b67\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s"
Oct 30 21:41:24.448380 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.438060    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63f89cda-14de-416c-9263-00ffdb8c9ec9-istio-data\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww\" (UID: \"63f89cda-14de-416c-9263-00ffdb8c9ec9\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww"
Oct 30 21:41:24.448380 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.438199    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/222c96b4-9c61-49c0-a3ed-450ea7d17b67-credential-socket\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s\" (UID: \"222c96b4-9c61-49c0-a3ed-450ea7d17b67\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s"
Oct 30 21:41:24.448380 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.438238    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63f89cda-14de-416c-9263-00ffdb8c9ec9-istio-envoy\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww\" (UID: \"63f89cda-14de-416c-9263-00ffdb8c9ec9\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww"
Oct 30 21:41:24.448380 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.438267    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63f89cda-14de-416c-9263-00ffdb8c9ec9-workload-socket\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww\" (UID: \"63f89cda-14de-416c-9263-00ffdb8c9ec9\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww"
Oct 30 21:41:24.448380 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.438306    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spprn\" (UniqueName: \"kubernetes.io/projected/63f89cda-14de-416c-9263-00ffdb8c9ec9-kube-api-access-spprn\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww\" (UID: \"63f89cda-14de-416c-9263-00ffdb8c9ec9\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww"
Oct 30 21:41:24.470481 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.450203    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hvh5\" (UniqueName: \"kubernetes.io/projected/3b92d898-765d-445d-9056-9f256030162e-kube-api-access-7hvh5\") pod \"e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc\" (UID: \"3b92d898-765d-445d-9056-9f256030162e\") " pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc"
Oct 30 21:41:24.575567 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.570471    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw" podUID="914f7544-6ef8-4496-b98d-b7ebc4123cda" containerName="main" probeResult="failure" output="Get \"https://10.132.0.86:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:24.575567 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.438355    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/222c96b4-9c61-49c0-a3ed-450ea7d17b67-istio-envoy\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s\" (UID: \"222c96b4-9c61-49c0-a3ed-450ea7d17b67\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s"
Oct 30 21:41:24.575567 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.571114    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63f89cda-14de-416c-9263-00ffdb8c9ec9-istio-podinfo\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww\" (UID: \"63f89cda-14de-416c-9263-00ffdb8c9ec9\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww"
Oct 30 21:41:24.602387 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.598573    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg" podUID="2d0ea058-b6ca-4181-8823-2b719883e545" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": context deadline exceeded"
Oct 30 21:41:24.627388 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.623952    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw" podUID="2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:24.647382 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.645414    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2"]
Oct 30 21:41:24.715774 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.657919    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc"
Oct 30 21:41:24.721618 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.721552    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2"
Oct 30 21:41:24.801478 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.798149    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b99e2de-e8a0-4566-b14c-dd627c74edf3-credential-socket\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm\" (UID: \"6b99e2de-e8a0-4566-b14c-dd627c74edf3\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm"
Oct 30 21:41:24.801478 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.798242    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/222c96b4-9c61-49c0-a3ed-450ea7d17b67-istio-token\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s\" (UID: \"222c96b4-9c61-49c0-a3ed-450ea7d17b67\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s"
Oct 30 21:41:24.801478 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.798279    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b99e2de-e8a0-4566-b14c-dd627c74edf3-workload-socket\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm\" (UID: \"6b99e2de-e8a0-4566-b14c-dd627c74edf3\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm"
Oct 30 21:41:24.801478 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.798311    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63f89cda-14de-416c-9263-00ffdb8c9ec9-workload-certs\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww\" (UID: \"63f89cda-14de-416c-9263-00ffdb8c9ec9\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww"
Oct 30 21:41:24.801478 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.798340    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b99e2de-e8a0-4566-b14c-dd627c74edf3-istio-podinfo\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm\" (UID: \"6b99e2de-e8a0-4566-b14c-dd627c74edf3\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm"
Oct 30 21:41:24.801478 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.798387    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/222c96b4-9c61-49c0-a3ed-450ea7d17b67-istio-podinfo\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s\" (UID: \"222c96b4-9c61-49c0-a3ed-450ea7d17b67\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s"
Oct 30 21:41:24.801478 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.798416    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63f89cda-14de-416c-9263-00ffdb8c9ec9-istiod-ca-cert\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww\" (UID: \"63f89cda-14de-416c-9263-00ffdb8c9ec9\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww"
Oct 30 21:41:24.801478 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.798440    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/57dfd470-9c35-404f-b967-14a9103adeea-workload-certs\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2\" (UID: \"57dfd470-9c35-404f-b967-14a9103adeea\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2"
Oct 30 21:41:24.801478 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.798477    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/222c96b4-9c61-49c0-a3ed-450ea7d17b67-workload-certs\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s\" (UID: \"222c96b4-9c61-49c0-a3ed-450ea7d17b67\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s"
Oct 30 21:41:24.801478 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.798514    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63f89cda-14de-416c-9263-00ffdb8c9ec9-istio-data\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww\" (UID: \"63f89cda-14de-416c-9263-00ffdb8c9ec9\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww"
Oct 30 21:41:24.801478 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.798541    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57dfd470-9c35-404f-b967-14a9103adeea-istio-token\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2\" (UID: \"57dfd470-9c35-404f-b967-14a9103adeea\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2"
Oct 30 21:41:24.801478 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.798590    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/57dfd470-9c35-404f-b967-14a9103adeea-istio-podinfo\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2\" (UID: \"57dfd470-9c35-404f-b967-14a9103adeea\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2"
Oct 30 21:41:24.801478 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.798645    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/222c96b4-9c61-49c0-a3ed-450ea7d17b67-credential-socket\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s\" (UID: \"222c96b4-9c61-49c0-a3ed-450ea7d17b67\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s"
Oct 30 21:41:24.801478 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.798672    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btml4\" (UniqueName: \"kubernetes.io/projected/6b99e2de-e8a0-4566-b14c-dd627c74edf3-kube-api-access-btml4\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm\" (UID: \"6b99e2de-e8a0-4566-b14c-dd627c74edf3\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm"
Oct 30 21:41:24.801478 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.798706    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63f89cda-14de-416c-9263-00ffdb8c9ec9-istio-envoy\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww\" (UID: \"63f89cda-14de-416c-9263-00ffdb8c9ec9\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww"
Oct 30 21:41:24.801478 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.798735    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63f89cda-14de-416c-9263-00ffdb8c9ec9-workload-socket\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww\" (UID: \"63f89cda-14de-416c-9263-00ffdb8c9ec9\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww"
Oct 30 21:41:24.801478 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.798771    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spprn\" (UniqueName: \"kubernetes.io/projected/63f89cda-14de-416c-9263-00ffdb8c9ec9-kube-api-access-spprn\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww\" (UID: \"63f89cda-14de-416c-9263-00ffdb8c9ec9\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww"
Oct 30 21:41:24.801478 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.798816    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/222c96b4-9c61-49c0-a3ed-450ea7d17b67-istio-envoy\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s\" (UID: \"222c96b4-9c61-49c0-a3ed-450ea7d17b67\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s"
Oct 30 21:41:24.801478 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.798847    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63f89cda-14de-416c-9263-00ffdb8c9ec9-istio-podinfo\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww\" (UID: \"63f89cda-14de-416c-9263-00ffdb8c9ec9\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww"
Oct 30 21:41:24.801478 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.798947    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/57dfd470-9c35-404f-b967-14a9103adeea-workload-socket\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2\" (UID: \"57dfd470-9c35-404f-b967-14a9103adeea\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2"
Oct 30 21:41:24.801478 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.798988    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnvsp\" (UniqueName: \"kubernetes.io/projected/57dfd470-9c35-404f-b967-14a9103adeea-kube-api-access-mnvsp\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2\" (UID: \"57dfd470-9c35-404f-b967-14a9103adeea\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2"
Oct 30 21:41:24.801478 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.799018    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/222c96b4-9c61-49c0-a3ed-450ea7d17b67-istiod-ca-cert\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s\" (UID: \"222c96b4-9c61-49c0-a3ed-450ea7d17b67\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s"
Oct 30 21:41:24.801478 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.799046    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/57dfd470-9c35-404f-b967-14a9103adeea-istiod-ca-cert\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2\" (UID: \"57dfd470-9c35-404f-b967-14a9103adeea\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2"
Oct 30 21:41:24.801478 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.799088    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b99e2de-e8a0-4566-b14c-dd627c74edf3-istio-token\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm\" (UID: \"6b99e2de-e8a0-4566-b14c-dd627c74edf3\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm"
Oct 30 21:41:24.801478 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.799115    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b99e2de-e8a0-4566-b14c-dd627c74edf3-istiod-ca-cert\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm\" (UID: \"6b99e2de-e8a0-4566-b14c-dd627c74edf3\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm"
Oct 30 21:41:24.801478 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.799143    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/222c96b4-9c61-49c0-a3ed-450ea7d17b67-istio-data\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s\" (UID: \"222c96b4-9c61-49c0-a3ed-450ea7d17b67\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s"
Oct 30 21:41:24.801478 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.799167    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spnlf\" (UniqueName: \"kubernetes.io/projected/222c96b4-9c61-49c0-a3ed-450ea7d17b67-kube-api-access-spnlf\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s\" (UID: \"222c96b4-9c61-49c0-a3ed-450ea7d17b67\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s"
Oct 30 21:41:24.801478 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.799238    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b99e2de-e8a0-4566-b14c-dd627c74edf3-istio-data\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm\" (UID: \"6b99e2de-e8a0-4566-b14c-dd627c74edf3\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm"
Oct 30 21:41:24.801478 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.799265    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63f89cda-14de-416c-9263-00ffdb8c9ec9-istio-token\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww\" (UID: \"63f89cda-14de-416c-9263-00ffdb8c9ec9\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww"
Oct 30 21:41:24.801478 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.799301    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/222c96b4-9c61-49c0-a3ed-450ea7d17b67-workload-socket\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s\" (UID: \"222c96b4-9c61-49c0-a3ed-450ea7d17b67\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s"
Oct 30 21:41:24.801478 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.799339    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b99e2de-e8a0-4566-b14c-dd627c74edf3-workload-certs\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm\" (UID: \"6b99e2de-e8a0-4566-b14c-dd627c74edf3\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm"
Oct 30 21:41:24.801478 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.799443    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/57dfd470-9c35-404f-b967-14a9103adeea-istio-data\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2\" (UID: \"57dfd470-9c35-404f-b967-14a9103adeea\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2"
Oct 30 21:41:24.801478 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.799471    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/57dfd470-9c35-404f-b967-14a9103adeea-istio-envoy\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2\" (UID: \"57dfd470-9c35-404f-b967-14a9103adeea\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2"
Oct 30 21:41:24.801478 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.799565    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b99e2de-e8a0-4566-b14c-dd627c74edf3-istio-envoy\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm\" (UID: \"6b99e2de-e8a0-4566-b14c-dd627c74edf3\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm"
Oct 30 21:41:24.801478 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.799596    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/57dfd470-9c35-404f-b967-14a9103adeea-credential-socket\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2\" (UID: \"57dfd470-9c35-404f-b967-14a9103adeea\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2"
Oct 30 21:41:24.801478 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.799637    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63f89cda-14de-416c-9263-00ffdb8c9ec9-credential-socket\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww\" (UID: \"63f89cda-14de-416c-9263-00ffdb8c9ec9\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww"
Oct 30 21:41:24.801478 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.800124    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63f89cda-14de-416c-9263-00ffdb8c9ec9-credential-socket\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww\" (UID: \"63f89cda-14de-416c-9263-00ffdb8c9ec9\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww"
Oct 30 21:41:24.821276 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.821233    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63f89cda-14de-416c-9263-00ffdb8c9ec9-workload-certs\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww\" (UID: \"63f89cda-14de-416c-9263-00ffdb8c9ec9\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww"
Oct 30 21:41:24.862385 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.858407    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/222c96b4-9c61-49c0-a3ed-450ea7d17b67-istio-podinfo\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s\" (UID: \"222c96b4-9c61-49c0-a3ed-450ea7d17b67\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s"
Oct 30 21:41:24.982385 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.869248    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/222c96b4-9c61-49c0-a3ed-450ea7d17b67-istio-token\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s\" (UID: \"222c96b4-9c61-49c0-a3ed-450ea7d17b67\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s"
Oct 30 21:41:24.982385 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.969926    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b99e2de-e8a0-4566-b14c-dd627c74edf3-credential-socket\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm\" (UID: \"6b99e2de-e8a0-4566-b14c-dd627c74edf3\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm"
Oct 30 21:41:24.982385 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.970062    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b99e2de-e8a0-4566-b14c-dd627c74edf3-workload-socket\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm\" (UID: \"6b99e2de-e8a0-4566-b14c-dd627c74edf3\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm"
Oct 30 21:41:24.982385 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:24.970142    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b99e2de-e8a0-4566-b14c-dd627c74edf3-istio-podinfo\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm\" (UID: \"6b99e2de-e8a0-4566-b14c-dd627c74edf3\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm"
Oct 30 21:41:25.159604 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:25.125005    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63f89cda-14de-416c-9263-00ffdb8c9ec9-istiod-ca-cert\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww\" (UID: \"63f89cda-14de-416c-9263-00ffdb8c9ec9\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww"
Oct 30 21:41:25.185385 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:25.126664    5032 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj" podUID="d0ff5023-fd3f-4734-94c6-76b666010adf" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/health\": context deadline exceeded"
Oct 30 21:41:25.185385 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:25.137752    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/222c96b4-9c61-49c0-a3ed-450ea7d17b67-workload-certs\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s\" (UID: \"222c96b4-9c61-49c0-a3ed-450ea7d17b67\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s"
Oct 30 21:41:25.193404 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:25.193352    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/222c96b4-9c61-49c0-a3ed-450ea7d17b67-credential-socket\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s\" (UID: \"222c96b4-9c61-49c0-a3ed-450ea7d17b67\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s"
Oct 30 21:41:25.207392 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:25.203845    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2"]
Oct 30 21:41:25.355624 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:25.230441    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63f89cda-14de-416c-9263-00ffdb8c9ec9-istio-data\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww\" (UID: \"63f89cda-14de-416c-9263-00ffdb8c9ec9\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww"
Oct 30 21:41:25.355929 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:25.256041    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63f89cda-14de-416c-9263-00ffdb8c9ec9-workload-socket\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww\" (UID: \"63f89cda-14de-416c-9263-00ffdb8c9ec9\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww"
Oct 30 21:41:25.356053 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:25.269804    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc" podUID="23b97dbb-8488-4c44-a810-882ea18ed919" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": context deadline exceeded"
Oct 30 21:41:25.363556 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:25.363522    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b99e2de-e8a0-4566-b14c-dd627c74edf3-istio-envoy\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm\" (UID: \"6b99e2de-e8a0-4566-b14c-dd627c74edf3\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm"
Oct 30 21:41:25.363733 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:25.363583    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/57dfd470-9c35-404f-b967-14a9103adeea-credential-socket\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2\" (UID: \"57dfd470-9c35-404f-b967-14a9103adeea\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2"
Oct 30 21:41:25.363861 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:25.363823    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/57dfd470-9c35-404f-b967-14a9103adeea-workload-certs\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2\" (UID: \"57dfd470-9c35-404f-b967-14a9103adeea\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2"
Oct 30 21:41:25.363940 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:25.363917    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57dfd470-9c35-404f-b967-14a9103adeea-istio-token\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2\" (UID: \"57dfd470-9c35-404f-b967-14a9103adeea\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2"
Oct 30 21:41:25.364020 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:25.364005    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/57dfd470-9c35-404f-b967-14a9103adeea-istio-podinfo\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2\" (UID: \"57dfd470-9c35-404f-b967-14a9103adeea\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2"
Oct 30 21:41:25.364106 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:25.364082    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btml4\" (UniqueName: \"kubernetes.io/projected/6b99e2de-e8a0-4566-b14c-dd627c74edf3-kube-api-access-btml4\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm\" (UID: \"6b99e2de-e8a0-4566-b14c-dd627c74edf3\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm"
Oct 30 21:41:25.364391 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:25.364351    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b99e2de-e8a0-4566-b14c-dd627c74edf3-istio-data\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm\" (UID: \"6b99e2de-e8a0-4566-b14c-dd627c74edf3\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm"
Oct 30 21:41:25.364489 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:25.364436    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b99e2de-e8a0-4566-b14c-dd627c74edf3-workload-certs\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm\" (UID: \"6b99e2de-e8a0-4566-b14c-dd627c74edf3\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm"
Oct 30 21:41:25.364568 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:25.364518    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/57dfd470-9c35-404f-b967-14a9103adeea-istio-data\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2\" (UID: \"57dfd470-9c35-404f-b967-14a9103adeea\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2"
Oct 30 21:41:25.364627 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:25.364556    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/57dfd470-9c35-404f-b967-14a9103adeea-istio-envoy\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2\" (UID: \"57dfd470-9c35-404f-b967-14a9103adeea\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2"
Oct 30 21:41:25.380414 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:25.270212    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/222c96b4-9c61-49c0-a3ed-450ea7d17b67-workload-socket\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s\" (UID: \"222c96b4-9c61-49c0-a3ed-450ea7d17b67\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s"
Oct 30 21:41:25.381282 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:25.381253    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc"
Oct 30 21:41:25.665932 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:25.568330    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/222c96b4-9c61-49c0-a3ed-450ea7d17b67-istiod-ca-cert\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s\" (UID: \"222c96b4-9c61-49c0-a3ed-450ea7d17b67\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s"
Oct 30 21:41:25.916424 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:25.582476    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw" podUID="914f7544-6ef8-4496-b98d-b7ebc4123cda" containerName="main" probeResult="failure" output="Get \"https://10.132.0.86:8000/ready\": context deadline exceeded"
Oct 30 21:41:25.945821 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:25.582967    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/222c96b4-9c61-49c0-a3ed-450ea7d17b67-istio-data\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s\" (UID: \"222c96b4-9c61-49c0-a3ed-450ea7d17b67\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s"
Oct 30 21:41:25.983388 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:25.639445    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw" podUID="2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:26.126751 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:25.157320    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b99e2de-e8a0-4566-b14c-dd627c74edf3-istio-podinfo\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm\" (UID: \"6b99e2de-e8a0-4566-b14c-dd627c74edf3\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm"
Oct 30 21:41:26.306501 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:25.675344    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-spprn\" (UniqueName: \"kubernetes.io/projected/63f89cda-14de-416c-9263-00ffdb8c9ec9-kube-api-access-spprn\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww\" (UID: \"63f89cda-14de-416c-9263-00ffdb8c9ec9\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww"
Oct 30 21:41:26.306782 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:25.690306    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/57dfd470-9c35-404f-b967-14a9103adeea-workload-socket\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2\" (UID: \"57dfd470-9c35-404f-b967-14a9103adeea\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2"
Oct 30 21:41:26.306968 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:26.306944    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnvsp\" (UniqueName: \"kubernetes.io/projected/57dfd470-9c35-404f-b967-14a9103adeea-kube-api-access-mnvsp\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2\" (UID: \"57dfd470-9c35-404f-b967-14a9103adeea\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2"
Oct 30 21:41:26.307113 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:26.307096    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/57dfd470-9c35-404f-b967-14a9103adeea-istiod-ca-cert\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2\" (UID: \"57dfd470-9c35-404f-b967-14a9103adeea\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2"
Oct 30 21:41:26.336536 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:26.336493    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b99e2de-e8a0-4566-b14c-dd627c74edf3-istio-token\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm\" (UID: \"6b99e2de-e8a0-4566-b14c-dd627c74edf3\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm"
Oct 30 21:41:26.336794 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:26.336776    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b99e2de-e8a0-4566-b14c-dd627c74edf3-istiod-ca-cert\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm\" (UID: \"6b99e2de-e8a0-4566-b14c-dd627c74edf3\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm"
Oct 30 21:41:26.362389 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:25.762970    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/57dfd470-9c35-404f-b967-14a9103adeea-workload-certs\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2\" (UID: \"57dfd470-9c35-404f-b967-14a9103adeea\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2"
Oct 30 21:41:26.554387 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:25.964821    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63f89cda-14de-416c-9263-00ffdb8c9ec9-istio-envoy\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww\" (UID: \"63f89cda-14de-416c-9263-00ffdb8c9ec9\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww"
Oct 30 21:41:26.554387 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:25.965016    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/222c96b4-9c61-49c0-a3ed-450ea7d17b67-istio-envoy\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s\" (UID: \"222c96b4-9c61-49c0-a3ed-450ea7d17b67\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s"
Oct 30 21:41:26.554387 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:25.997610    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b99e2de-e8a0-4566-b14c-dd627c74edf3-istio-data\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm\" (UID: \"6b99e2de-e8a0-4566-b14c-dd627c74edf3\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm"
Oct 30 21:41:26.554387 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:26.024196    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/57dfd470-9c35-404f-b967-14a9103adeea-istio-data\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2\" (UID: \"57dfd470-9c35-404f-b967-14a9103adeea\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2"
Oct 30 21:41:26.554387 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:26.149310    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63f89cda-14de-416c-9263-00ffdb8c9ec9-istio-podinfo\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww\" (UID: \"63f89cda-14de-416c-9263-00ffdb8c9ec9\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww"
Oct 30 21:41:26.554387 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:25.674525    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg" podUID="2d0ea058-b6ca-4181-8823-2b719883e545" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": context deadline exceeded"
Oct 30 21:41:26.554387 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:25.361854    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b99e2de-e8a0-4566-b14c-dd627c74edf3-credential-socket\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm\" (UID: \"6b99e2de-e8a0-4566-b14c-dd627c74edf3\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm"
Oct 30 21:41:26.554387 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:26.283668    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63f89cda-14de-416c-9263-00ffdb8c9ec9-istio-token\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww\" (UID: \"63f89cda-14de-416c-9263-00ffdb8c9ec9\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww"
Oct 30 21:41:26.554387 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:25.760071    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-spnlf\" (UniqueName: \"kubernetes.io/projected/222c96b4-9c61-49c0-a3ed-450ea7d17b67-kube-api-access-spnlf\") pod \"e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s\" (UID: \"222c96b4-9c61-49c0-a3ed-450ea7d17b67\") " pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s"
Oct 30 21:41:26.554387 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:25.762031    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/57dfd470-9c35-404f-b967-14a9103adeea-workload-socket\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2\" (UID: \"57dfd470-9c35-404f-b967-14a9103adeea\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2"
Oct 30 21:41:26.554387 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:26.310908    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/57dfd470-9c35-404f-b967-14a9103adeea-istiod-ca-cert\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2\" (UID: \"57dfd470-9c35-404f-b967-14a9103adeea\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2"
Oct 30 21:41:26.554387 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:25.872905    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj" podUID="d0ff5023-fd3f-4734-94c6-76b666010adf" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": context deadline exceeded"
Oct 30 21:41:26.554387 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:26.391960    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b99e2de-e8a0-4566-b14c-dd627c74edf3-istiod-ca-cert\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm\" (UID: \"6b99e2de-e8a0-4566-b14c-dd627c74edf3\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm"
Oct 30 21:41:26.554387 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:25.362136    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b99e2de-e8a0-4566-b14c-dd627c74edf3-workload-socket\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm\" (UID: \"6b99e2de-e8a0-4566-b14c-dd627c74edf3\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm"
Oct 30 21:41:26.554387 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:26.516429    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b99e2de-e8a0-4566-b14c-dd627c74edf3-workload-certs\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm\" (UID: \"6b99e2de-e8a0-4566-b14c-dd627c74edf3\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm"
Oct 30 21:41:26.662667 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:26.630529    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/57dfd470-9c35-404f-b967-14a9103adeea-credential-socket\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2\" (UID: \"57dfd470-9c35-404f-b967-14a9103adeea\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2"
Oct 30 21:41:26.691556 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:26.691481    5032 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw" podUID="2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:26.741388 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:26.720821    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s"
Oct 30 21:41:26.741388 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:26.721112    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/57dfd470-9c35-404f-b967-14a9103adeea-istio-envoy\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2\" (UID: \"57dfd470-9c35-404f-b967-14a9103adeea\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2"
Oct 30 21:41:26.806930 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:26.769659    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b99e2de-e8a0-4566-b14c-dd627c74edf3-istio-envoy\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm\" (UID: \"6b99e2de-e8a0-4566-b14c-dd627c74edf3\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm"
Oct 30 21:41:26.809028 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:26.808974    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-btml4\" (UniqueName: \"kubernetes.io/projected/6b99e2de-e8a0-4566-b14c-dd627c74edf3-kube-api-access-btml4\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm\" (UID: \"6b99e2de-e8a0-4566-b14c-dd627c74edf3\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm"
Oct 30 21:41:26.878507 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:26.809059    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57dfd470-9c35-404f-b967-14a9103adeea-istio-token\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2\" (UID: \"57dfd470-9c35-404f-b967-14a9103adeea\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2"
Oct 30 21:41:26.895407 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:26.857880    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc" podUID="23b97dbb-8488-4c44-a810-882ea18ed919" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:26.991396 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:26.987929    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/57dfd470-9c35-404f-b967-14a9103adeea-istio-podinfo\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2\" (UID: \"57dfd470-9c35-404f-b967-14a9103adeea\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2"
Oct 30 21:41:26.991396 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:26.858310    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww"
Oct 30 21:41:26.991396 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:26.988732    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw" podUID="914f7544-6ef8-4496-b98d-b7ebc4123cda" containerName="main" probeResult="failure" output="Get \"https://10.132.0.86:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:27.156443 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:27.153452    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw" podUID="2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:27.537716 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:27.425744    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b99e2de-e8a0-4566-b14c-dd627c74edf3-istio-token\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm\" (UID: \"6b99e2de-e8a0-4566-b14c-dd627c74edf3\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm"
Oct 30 21:41:27.702385 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:27.688153    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg" podUID="2d0ea058-b6ca-4181-8823-2b719883e545" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": context deadline exceeded"
Oct 30 21:41:27.912099 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:27.737484    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj" podUID="d0ff5023-fd3f-4734-94c6-76b666010adf" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": context deadline exceeded"
Oct 30 21:41:27.912373 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:27.737630    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w0-000f64-6576d48f75-5rdtz" podUID="c9e3ad2b-93c6-46fd-8b0c-87e24852081e" containerName="payload-processing" probeResult="failure" output="timeout: failed to connect service \"10.132.0.68:50052\" within 1s: context deadline exceeded"
Oct 30 21:41:27.912548 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:27.737682    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-1d8034-5b96976fc5-hsr78" podUID="27aef427-6b1e-4845-b8d9-a83cba81b0e0" containerName="payload-pre-processing" probeResult="failure" output="timeout: failed to connect service \"10.132.0.64:50052\" within 1s: context deadline exceeded"
Oct 30 21:41:27.912697 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:27.737731    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w4-0fa864-5d85948857-tw7k6" podUID="76de8529-43f7-4285-864b-f8242fca2cae" containerName="payload-processing" probeResult="failure" output="timeout: failed to connect service \"10.132.0.60:50052\" within 1s: context deadline exceeded"
Oct 30 21:41:27.912842 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:27.790615    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-d9q4t" podUID="294414e8-4fc5-48ec-9c47-9660381fda80" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": context deadline exceeded"
Oct 30 21:41:27.912980 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:27.885552    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-xv7qv" podUID="4f6d23b8-1364-4b03-96c6-6649f3a0644a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.51:8000/ready\": context deadline exceeded"
Oct 30 21:41:27.932240 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:27.932197    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-jx4qg" podUID="0c945ece-1ac0-4993-aea4-e853a23a4099" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:27.950079 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:27.946482    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc" podUID="23b97dbb-8488-4c44-a810-882ea18ed919" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:27.975864 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:27.947732    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/echo-simulator-kserve-64b6f8f96c-vg752" podUID="8fa31603-e280-4566-bded-aaebc80c2179" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:27.976185 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:27.969841    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm"
Oct 30 21:41:27.994624 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:27.994565    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:41:28.017642 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:28.015930    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:41:28.019003 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:28.018963    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:41:28.044712 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:28.007471    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnvsp\" (UniqueName: \"kubernetes.io/projected/57dfd470-9c35-404f-b967-14a9103adeea-kube-api-access-mnvsp\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2\" (UID: \"57dfd470-9c35-404f-b967-14a9103adeea\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2"
Oct 30 21:41:28.044712 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:28.001274    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw" podUID="914f7544-6ef8-4496-b98d-b7ebc4123cda" containerName="main" probeResult="failure" output="Get \"https://10.132.0.86:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:28.044712 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:41:28.005494    5032 logging.go:55] [core] [Channel #774 SubChannel #775]grpc: addrConn.createTransport failed to connect to {Addr: "10.132.0.68:50052", ServerName: "10.132.0.68:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: failed to write client preface: write tcp 10.132.0.2:33194->10.132.0.68:50052: use of closed network connection"
Oct 30 21:41:28.044712 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:41:28.015820    5032 logging.go:55] [core] [Channel #772 SubChannel #778]grpc: addrConn.createTransport failed to connect to {Addr: "10.132.0.60:50052", ServerName: "10.132.0.60:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: failed to write client preface: write tcp 10.132.0.2:43430->10.132.0.60:50052: use of closed network connection"
Oct 30 21:41:28.044712 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:41:28.029975    5032 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="2.353s"
Oct 30 21:41:28.250487 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:27.970264    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx" podUID="34fdf57d-86f6-474f-a4f6-811e5063c405" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": context deadline exceeded"
Oct 30 21:41:28.308399 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:28.284001    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw" podUID="2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded"
Oct 30 21:41:28.309790 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:28.309765    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2"
Oct 30 21:41:29.078343 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:29.077786    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj" podUID="d0ff5023-fd3f-4734-94c6-76b666010adf" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:29.089929 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:29.089891    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg" podUID="2d0ea058-b6ca-4181-8823-2b719883e545" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": context deadline exceeded"
Oct 30 21:41:29.093935 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:29.093489    5032 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc" podUID="23b97dbb-8488-4c44-a810-882ea18ed919" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:29.215261 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:29.214263    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc" podUID="23b97dbb-8488-4c44-a810-882ea18ed919" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:29.215261 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:29.214585    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw" podUID="914f7544-6ef8-4496-b98d-b7ebc4123cda" containerName="main" probeResult="failure" output="Get \"https://10.132.0.86:8000/ready\": context deadline exceeded"
Oct 30 21:41:29.223504 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:29.223212    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx" podUID="34fdf57d-86f6-474f-a4f6-811e5063c405" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:29.258197 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:41:29.258157    5032 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.228s"
Oct 30 21:41:29.258386 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:29.258259    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw"
Oct 30 21:41:29.258450 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:29.258351    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc"]
Oct 30 21:41:29.258503 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:29.258491    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw"
Oct 30 21:41:29.258555 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:29.258535    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg"
Oct 30 21:41:29.258611 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:29.258570    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj"
Oct 30 21:41:29.258611 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:29.258599    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj"
Oct 30 21:41:29.258705 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:29.258631    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx"
Oct 30 21:41:29.258705 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:29.258659    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw"
Oct 30 21:41:29.272673 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:29.272486    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx"
Oct 30 21:41:29.274109 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:29.274072    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw"
Oct 30 21:41:29.287921 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:29.287741    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:41:29.301769 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:29.301731    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:41:29.302014 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:29.301988    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:41:29.302612 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:29.290027    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc" event={"ID":"3b92d898-765d-445d-9056-9f256030162e","Type":"ContainerStarted","Data":"01ca058bc33c80e0bf7c34d70704e463a3fb120936834e4dcc94b5f93d5b00c7"}
Oct 30 21:41:29.302712 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:29.302642    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg"
Oct 30 21:41:29.311405 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:29.309004    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s"]
Oct 30 21:41:29.380805 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:29.380777    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc"
Oct 30 21:41:29.482634 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:41:29.482274    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63f89cda_14de_416c_9263_00ffdb8c9ec9.slice/crio-030bed24ac41c7677b3b80d59769dae999f72bbb8567d871b500c5754906c681 WatchSource:0}: Error finding container 030bed24ac41c7677b3b80d59769dae999f72bbb8567d871b500c5754906c681: Status 404 returned error can't find the container with id 030bed24ac41c7677b3b80d59769dae999f72bbb8567d871b500c5754906c681
Oct 30 21:41:29.485458 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:29.485379    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww"]
Oct 30 21:41:29.491191 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:29.490768    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:41:29.491191 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:29.490830    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:41:29.491191 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:29.490862    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:41:29.656667 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:29.656623    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2"]
Oct 30 21:41:29.689914 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:29.688710    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:41:29.689914 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:29.688807    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:41:29.689914 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:29.688843    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:41:29.916382 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:29.916160    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm"]
Oct 30 21:41:29.929172 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:29.928919    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:41:29.929410 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:29.929379    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:41:29.929552 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:29.929532    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:41:30.388988 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:30.388846    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc" event={"ID":"3b92d898-765d-445d-9056-9f256030162e","Type":"ContainerStarted","Data":"bc19f76ff61a6b70cb04279dd2eca09b3218c48b6d2d6b731ed8e6fb6da03e66"}
Oct 30 21:41:30.397194 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:30.397162    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s" event={"ID":"222c96b4-9c61-49c0-a3ed-450ea7d17b67","Type":"ContainerStarted","Data":"a0cc01dd8a6e2215c301be962219d059ff1634bf72bac7e9f43fab59f4f5abc4"}
Oct 30 21:41:30.398744 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:30.397421    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s" event={"ID":"222c96b4-9c61-49c0-a3ed-450ea7d17b67","Type":"ContainerStarted","Data":"ff8c47fbec44be4bdd66bfe02c1b6e7640300faf5da51e3d303845f08fec717b"}
Oct 30 21:41:30.440904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:30.434540    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2" event={"ID":"57dfd470-9c35-404f-b967-14a9103adeea","Type":"ContainerStarted","Data":"573e81d49ed938098ecb7f82d952e67c858bdee65faa9ad51332ddd424dcdbcd"}
Oct 30 21:41:30.440904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:30.434607    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2" event={"ID":"57dfd470-9c35-404f-b967-14a9103adeea","Type":"ContainerStarted","Data":"0b0069e866b89018531851b1bcb7d9e9a190c17843389d89e4f289d18020ba7f"}
Oct 30 21:41:30.463030 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:30.462990    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm" event={"ID":"6b99e2de-e8a0-4566-b14c-dd627c74edf3","Type":"ContainerStarted","Data":"215dd4edb5877e9e26232807f565e61d0266b5ee2227ae963f89fae8a84effb6"}
Oct 30 21:41:30.463320 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:30.463299    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm" event={"ID":"6b99e2de-e8a0-4566-b14c-dd627c74edf3","Type":"ContainerStarted","Data":"5b29de64163757ed971d4b730c3d7a1314a1448f3d6fa8e96171954836cbe22c"}
Oct 30 21:41:30.470277 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:30.470218    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc" podStartSLOduration=7.470199521 podStartE2EDuration="7.470199521s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:41:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:41:30.465139658 +0000 UTC m=+803.266952122" watchObservedRunningTime="2026-10-30 21:41:30.470199521 +0000 UTC m=+803.272011987"
Oct 30 21:41:30.471833 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:30.471793    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww" event={"ID":"63f89cda-14de-416c-9263-00ffdb8c9ec9","Type":"ContainerStarted","Data":"329f325c035c1901134d37ebff509d6feb27eecda94f08290199a7b843bbb21a"}
Oct 30 21:41:30.472338 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:30.472303    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww" event={"ID":"63f89cda-14de-416c-9263-00ffdb8c9ec9","Type":"ContainerStarted","Data":"030bed24ac41c7677b3b80d59769dae999f72bbb8567d871b500c5754906c681"}
Oct 30 21:41:30.496487 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:30.496422    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm" podStartSLOduration=6.4964036929999995 podStartE2EDuration="6.496403693s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:41:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:41:30.48946873 +0000 UTC m=+803.291281194" watchObservedRunningTime="2026-10-30 21:41:30.496403693 +0000 UTC m=+803.298216160"
Oct 30 21:41:30.699445 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:30.696220    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s" podStartSLOduration=7.640017348 podStartE2EDuration="7.640017348s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:41:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:41:30.520728326 +0000 UTC m=+803.322540790" watchObservedRunningTime="2026-10-30 21:41:30.640017348 +0000 UTC m=+803.441829813"
Oct 30 21:41:30.758386 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:30.757715    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2" podStartSLOduration=6.757692466 podStartE2EDuration="6.757692466s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:41:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:41:30.563010701 +0000 UTC m=+803.364823166" watchObservedRunningTime="2026-10-30 21:41:30.757692466 +0000 UTC m=+803.559504933"
Oct 30 21:41:30.778385 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:30.777887    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s"
Oct 30 21:41:30.779065 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:30.778684    5032 patch_prober.go:29] interesting pod/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Oct 30 21:41:30.779065 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:30.778735    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s" podUID="222c96b4-9c61-49c0-a3ed-450ea7d17b67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Oct 30 21:41:30.820344 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:30.801763    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc"
Oct 30 21:41:30.820344 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:30.813069    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww" podStartSLOduration=7.81304704 podStartE2EDuration="7.81304704s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:41:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:41:30.632885096 +0000 UTC m=+803.434697561" watchObservedRunningTime="2026-10-30 21:41:30.81304704 +0000 UTC m=+803.614859506"
Oct 30 21:41:30.851472 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:30.841044    5032 patch_prober.go:29] interesting pod/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" start-of-body=
Oct 30 21:41:30.851472 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:30.841110    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc" podUID="3b92d898-765d-445d-9056-9f256030162e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused"
Oct 30 21:41:30.882401 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:30.874779    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww"
Oct 30 21:41:30.921757 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:30.903230    5032 patch_prober.go:29] interesting pod/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Oct 30 21:41:30.921757 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:30.903289    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww" podUID="63f89cda-14de-416c-9263-00ffdb8c9ec9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Oct 30 21:41:30.998294 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:30.997454    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm"
Oct 30 21:41:31.327755 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:31.327645    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2"
Oct 30 21:41:31.329809 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:31.329752    5032 patch_prober.go:29] interesting pod/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Oct 30 21:41:31.330078 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:31.330028    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2" podUID="57dfd470-9c35-404f-b967-14a9103adeea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Oct 30 21:41:31.575288 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:31.574868    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw" podUID="914f7544-6ef8-4496-b98d-b7ebc4123cda" containerName="main" probeResult="failure" output="Get \"https://10.132.0.86:8000/ready\": context deadline exceeded"
Oct 30 21:41:31.655902 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:31.655660    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw" podUID="2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded"
Oct 30 21:41:31.722641 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:31.709034    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj" podUID="d0ff5023-fd3f-4734-94c6-76b666010adf" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:31.796386 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:31.787489    5032 patch_prober.go:29] interesting pod/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Oct 30 21:41:31.796386 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:31.787556    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s" podUID="222c96b4-9c61-49c0-a3ed-450ea7d17b67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Oct 30 21:41:31.914465 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:31.895467    5032 patch_prober.go:29] interesting pod/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Oct 30 21:41:31.914465 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:31.895529    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww" podUID="63f89cda-14de-416c-9263-00ffdb8c9ec9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Oct 30 21:41:31.914465 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:31.895962    5032 patch_prober.go:29] interesting pod/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" start-of-body=
Oct 30 21:41:31.914465 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:31.896000    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc" podUID="3b92d898-765d-445d-9056-9f256030162e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused"
Oct 30 21:41:32.000382 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:31.999499    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc" podUID="23b97dbb-8488-4c44-a810-882ea18ed919" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": context deadline exceeded"
Oct 30 21:41:32.028520 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:32.028482    5032 patch_prober.go:29] interesting pod/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Oct 30 21:41:32.028764 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:32.028738    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm" podUID="6b99e2de-e8a0-4566-b14c-dd627c74edf3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Oct 30 21:41:32.136603 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:32.115685    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx" podUID="34fdf57d-86f6-474f-a4f6-811e5063c405" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:32.310383 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:32.301097    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg" podUID="2d0ea058-b6ca-4181-8823-2b719883e545" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:32.437876 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:32.437841    5032 patch_prober.go:29] interesting pod/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Oct 30 21:41:32.438199 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:32.438153    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2" podUID="57dfd470-9c35-404f-b967-14a9103adeea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Oct 30 21:41:32.631289 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:32.631171    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw" podUID="914f7544-6ef8-4496-b98d-b7ebc4123cda" containerName="main" probeResult="failure" output="Get \"https://10.132.0.86:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:32.721273 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:32.705128    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw" podUID="2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded"
Oct 30 21:41:32.723437 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:32.723401    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj" podUID="d0ff5023-fd3f-4734-94c6-76b666010adf" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:32.729818 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:32.729789    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw"
Oct 30 21:41:32.760941 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:32.758935    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw"
Oct 30 21:41:32.760941 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:32.758991    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj"
Oct 30 21:41:32.856764 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:32.856731    5032 patch_prober.go:29] interesting pod/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Oct 30 21:41:33.021130 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:33.021084    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s" podUID="222c96b4-9c61-49c0-a3ed-450ea7d17b67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Oct 30 21:41:33.119578 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:32.885934    5032 patch_prober.go:29] interesting pod/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Oct 30 21:41:33.120107 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:33.120065    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww" podUID="63f89cda-14de-416c-9263-00ffdb8c9ec9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Oct 30 21:41:33.142435 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:32.887534    5032 patch_prober.go:29] interesting pod/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" start-of-body=
Oct 30 21:41:33.142435 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:33.140024    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc" podUID="3b92d898-765d-445d-9056-9f256030162e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused"
Oct 30 21:41:33.142435 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:33.058031    5032 patch_prober.go:29] interesting pod/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Oct 30 21:41:33.142435 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:33.140112    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm" podUID="6b99e2de-e8a0-4566-b14c-dd627c74edf3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Oct 30 21:41:33.270348 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:33.270305    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx" podUID="34fdf57d-86f6-474f-a4f6-811e5063c405" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:33.270975 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:33.270953    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx"
Oct 30 21:41:33.333449 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:33.332640    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg" podUID="2d0ea058-b6ca-4181-8823-2b719883e545" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:33.339389 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:33.334067    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg"
Oct 30 21:41:33.797387 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:33.793754    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw" podUID="914f7544-6ef8-4496-b98d-b7ebc4123cda" containerName="main" probeResult="failure" output="Get \"https://10.132.0.86:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:34.049453 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:33.801598    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj" podUID="d0ff5023-fd3f-4734-94c6-76b666010adf" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": context deadline exceeded"
Oct 30 21:41:34.076493 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:33.976140    5032 patch_prober.go:29] interesting pod/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Oct 30 21:41:34.076795 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:34.076764    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2" podUID="57dfd470-9c35-404f-b967-14a9103adeea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Oct 30 21:41:34.130486 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:34.130434    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc" podUID="23b97dbb-8488-4c44-a810-882ea18ed919" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:34.192457 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:34.192407    5032 patch_prober.go:29] interesting pod/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Oct 30 21:41:34.192790 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:34.192725    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm" podUID="6b99e2de-e8a0-4566-b14c-dd627c74edf3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Oct 30 21:41:34.230462 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:34.230402    5032 patch_prober.go:29] interesting pod/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Oct 30 21:41:34.305273 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:34.305180    5032 patch_prober.go:29] interesting pod/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" start-of-body=
Oct 30 21:41:34.306109 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:34.305582    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc" podUID="3b92d898-765d-445d-9056-9f256030162e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused"
Oct 30 21:41:34.349143 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:34.347125    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx" podUID="34fdf57d-86f6-474f-a4f6-811e5063c405" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": context deadline exceeded"
Oct 30 21:41:34.371382 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:34.366286    5032 patch_prober.go:29] interesting pod/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Oct 30 21:41:34.371382 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:34.366383    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww" podUID="63f89cda-14de-416c-9263-00ffdb8c9ec9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Oct 30 21:41:34.378385 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:41:34.373669    5032 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.034s"
Oct 30 21:41:34.460451 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:34.410026    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw" podUID="2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded"
Oct 30 21:41:34.471982 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:34.461799    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s" podUID="222c96b4-9c61-49c0-a3ed-450ea7d17b67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Oct 30 21:41:34.471982 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:34.458555    5032 patch_prober.go:29] interesting pod/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Oct 30 21:41:34.471982 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:34.461896    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2" podUID="57dfd470-9c35-404f-b967-14a9103adeea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Oct 30 21:41:34.471982 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:34.410096    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg" podUID="2d0ea058-b6ca-4181-8823-2b719883e545" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:34.790131 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:34.790093    5032 patch_prober.go:29] interesting pod/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Oct 30 21:41:34.791370 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:34.791320    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s" podUID="222c96b4-9c61-49c0-a3ed-450ea7d17b67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Oct 30 21:41:34.817068 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:34.817034    5032 patch_prober.go:29] interesting pod/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" start-of-body=
Oct 30 21:41:34.817623 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:34.817592    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc" podUID="3b92d898-765d-445d-9056-9f256030162e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused"
Oct 30 21:41:34.892141 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:34.892106    5032 patch_prober.go:29] interesting pod/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Oct 30 21:41:34.960654 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:34.960609    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww" podUID="63f89cda-14de-416c-9263-00ffdb8c9ec9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Oct 30 21:41:35.010393 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:34.999184    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw" podUID="914f7544-6ef8-4496-b98d-b7ebc4123cda" containerName="main" probeResult="failure" output="Get \"https://10.132.0.86:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:35.055441 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:35.048520    5032 patch_prober.go:29] interesting pod/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Oct 30 21:41:35.055441 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:35.048586    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm" podUID="6b99e2de-e8a0-4566-b14c-dd627c74edf3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Oct 30 21:41:35.123387 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:35.085945    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj" podUID="d0ff5023-fd3f-4734-94c6-76b666010adf" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:35.244287 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:35.138462    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc" podUID="23b97dbb-8488-4c44-a810-882ea18ed919" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:35.254413 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:35.204754    5032 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj" podUID="d0ff5023-fd3f-4734-94c6-76b666010adf" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:35.254413 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:35.252813    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc"
Oct 30 21:41:35.348886 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:35.348788    5032 patch_prober.go:29] interesting pod/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Oct 30 21:41:35.360726 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:35.352450    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2" podUID="57dfd470-9c35-404f-b967-14a9103adeea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Oct 30 21:41:35.460435 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:35.439122    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx" podUID="34fdf57d-86f6-474f-a4f6-811e5063c405" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": context deadline exceeded"
Oct 30 21:41:35.462013 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:35.461964    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw" podUID="2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:35.466649 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:35.466614    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg" podUID="2d0ea058-b6ca-4181-8823-2b719883e545" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:35.823204 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:35.823173    5032 patch_prober.go:29] interesting pod/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Oct 30 21:41:35.865234 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:35.823787    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s" podUID="222c96b4-9c61-49c0-a3ed-450ea7d17b67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Oct 30 21:41:35.932728 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:35.890506    5032 patch_prober.go:29] interesting pod/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" start-of-body=
Oct 30 21:41:35.932728 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:35.890566    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc" podUID="3b92d898-765d-445d-9056-9f256030162e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused"
Oct 30 21:41:36.000908 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:36.000859    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw" podUID="914f7544-6ef8-4496-b98d-b7ebc4123cda" containerName="main" probeResult="failure" output="Get \"https://10.132.0.86:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:36.207508 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:36.206154    5032 patch_prober.go:29] interesting pod/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Oct 30 21:41:36.207508 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:36.206223    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww" podUID="63f89cda-14de-416c-9263-00ffdb8c9ec9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Oct 30 21:41:36.207508 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:36.206635    5032 patch_prober.go:29] interesting pod/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Oct 30 21:41:36.207508 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:36.206676    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm" podUID="6b99e2de-e8a0-4566-b14c-dd627c74edf3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Oct 30 21:41:36.245495 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:36.245442    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc" podUID="23b97dbb-8488-4c44-a810-882ea18ed919" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:36.246478 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:36.246444    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj" podUID="d0ff5023-fd3f-4734-94c6-76b666010adf" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": context deadline exceeded"
Oct 30 21:41:36.385460 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:36.385416    5032 patch_prober.go:29] interesting pod/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Oct 30 21:41:36.386057 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:36.386021    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2" podUID="57dfd470-9c35-404f-b967-14a9103adeea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Oct 30 21:41:36.441501 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:36.441459    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx" podUID="34fdf57d-86f6-474f-a4f6-811e5063c405" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": context deadline exceeded"
Oct 30 21:41:36.486473 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:36.471290    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg" podUID="2d0ea058-b6ca-4181-8823-2b719883e545" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:36.486473 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:36.474585    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw" podUID="2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded"
Oct 30 21:41:36.580385 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:36.579780    5032 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw" podUID="2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:36.808448 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:36.808138    5032 patch_prober.go:29] interesting pod/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" start-of-body=
Oct 30 21:41:36.808448 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:36.808205    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc" podUID="3b92d898-765d-445d-9056-9f256030162e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused"
Oct 30 21:41:36.849089 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:36.848338    5032 patch_prober.go:29] interesting pod/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Oct 30 21:41:36.875442 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:36.875401    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s" podUID="222c96b4-9c61-49c0-a3ed-450ea7d17b67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Oct 30 21:41:36.893387 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:36.885654    5032 patch_prober.go:29] interesting pod/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Oct 30 21:41:36.937522 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:36.937476    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww" podUID="63f89cda-14de-416c-9263-00ffdb8c9ec9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Oct 30 21:41:37.041561 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:37.022803    5032 patch_prober.go:29] interesting pod/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Oct 30 21:41:37.042860 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:37.042823    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm" podUID="6b99e2de-e8a0-4566-b14c-dd627c74edf3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Oct 30 21:41:37.066414 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:37.066293    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw" podUID="914f7544-6ef8-4496-b98d-b7ebc4123cda" containerName="main" probeResult="failure" output="Get \"https://10.132.0.86:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:37.283389 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:37.274415    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj" podUID="d0ff5023-fd3f-4734-94c6-76b666010adf" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:37.283389 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:37.274809    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc" podUID="23b97dbb-8488-4c44-a810-882ea18ed919" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:37.333681 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:37.333049    5032 patch_prober.go:29] interesting pod/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Oct 30 21:41:37.333681 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:37.333131    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2" podUID="57dfd470-9c35-404f-b967-14a9103adeea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Oct 30 21:41:37.559610 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:37.448623    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx" podUID="34fdf57d-86f6-474f-a4f6-811e5063c405" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:37.600902 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:37.510402    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg" podUID="2d0ea058-b6ca-4181-8823-2b719883e545" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": context deadline exceeded"
Oct 30 21:41:37.602091 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:37.510448    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw" podUID="2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:37.775112 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:37.775076    5032 patch_prober.go:29] interesting pod/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Oct 30 21:41:37.775350 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:37.775326    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s" podUID="222c96b4-9c61-49c0-a3ed-450ea7d17b67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Oct 30 21:41:37.858437 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:37.841960    5032 patch_prober.go:29] interesting pod/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" start-of-body=
Oct 30 21:41:37.858437 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:37.842020    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc" podUID="3b92d898-765d-445d-9056-9f256030162e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused"
Oct 30 21:41:37.937387 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:37.899059    5032 patch_prober.go:29] interesting pod/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Oct 30 21:41:37.937387 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:37.899116    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww" podUID="63f89cda-14de-416c-9263-00ffdb8c9ec9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Oct 30 21:41:38.020781 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:38.020746    5032 patch_prober.go:29] interesting pod/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Oct 30 21:41:38.025676 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:38.025637    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm" podUID="6b99e2de-e8a0-4566-b14c-dd627c74edf3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Oct 30 21:41:38.077122 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:38.076882    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw" podUID="914f7544-6ef8-4496-b98d-b7ebc4123cda" containerName="main" probeResult="failure" output="Get \"https://10.132.0.86:8000/ready\": context deadline exceeded"
Oct 30 21:41:38.420388 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:38.275619    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc" podUID="23b97dbb-8488-4c44-a810-882ea18ed919" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:38.420388 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:38.285490    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj" podUID="d0ff5023-fd3f-4734-94c6-76b666010adf" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:38.420388 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:38.376159    5032 patch_prober.go:29] interesting pod/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Oct 30 21:41:38.420388 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:38.410980    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2" podUID="57dfd470-9c35-404f-b967-14a9103adeea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Oct 30 21:41:38.555389 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:38.553873    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg"
Oct 30 21:41:38.604805 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:38.604758    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw" podUID="2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:38.606550 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:38.606513    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx" podUID="34fdf57d-86f6-474f-a4f6-811e5063c405" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:38.866945 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:38.866801    5032 patch_prober.go:29] interesting pod/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" start-of-body=
Oct 30 21:41:38.895680 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:38.895636    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc" podUID="3b92d898-765d-445d-9056-9f256030162e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused"
Oct 30 21:41:38.932452 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:38.923342    5032 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc" podUID="23b97dbb-8488-4c44-a810-882ea18ed919" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/health\": context deadline exceeded"
Oct 30 21:41:39.213025 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:38.952909    5032 patch_prober.go:29] interesting pod/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Oct 30 21:41:39.213406 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:39.213350    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s" podUID="222c96b4-9c61-49c0-a3ed-450ea7d17b67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Oct 30 21:41:39.213639 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:39.080030    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw" podUID="914f7544-6ef8-4496-b98d-b7ebc4123cda" containerName="main" probeResult="failure" output="Get \"https://10.132.0.86:8000/ready\": context deadline exceeded"
Oct 30 21:41:39.243733 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:38.952993    5032 patch_prober.go:29] interesting pod/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Oct 30 21:41:39.252537 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:39.252488    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww" podUID="63f89cda-14de-416c-9263-00ffdb8c9ec9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Oct 30 21:41:39.252817 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:39.045995    5032 patch_prober.go:29] interesting pod/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Oct 30 21:41:39.252967 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:39.252946    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm" podUID="6b99e2de-e8a0-4566-b14c-dd627c74edf3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Oct 30 21:41:39.435719 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:39.435668    5032 patch_prober.go:29] interesting pod/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Oct 30 21:41:39.462346 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:39.462302    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2" podUID="57dfd470-9c35-404f-b967-14a9103adeea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Oct 30 21:41:39.606438 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:39.470988    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj" podUID="d0ff5023-fd3f-4734-94c6-76b666010adf" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:39.657586 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:39.472063    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc" podUID="23b97dbb-8488-4c44-a810-882ea18ed919" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": context deadline exceeded"
Oct 30 21:41:39.680501 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:39.574540    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg" podUID="2d0ea058-b6ca-4181-8823-2b719883e545" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": context deadline exceeded"
Oct 30 21:41:39.680992 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:39.589356    5032 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx" podUID="34fdf57d-86f6-474f-a4f6-811e5063c405" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/health\": context deadline exceeded"
Oct 30 21:41:39.681125 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:39.630115    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx" podUID="34fdf57d-86f6-474f-a4f6-811e5063c405" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:39.695378 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:39.630179    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw" podUID="2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:41:39.697974 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:39.630224    5032 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg" podUID="2d0ea058-b6ca-4181-8823-2b719883e545" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/health\": context deadline exceeded"
Oct 30 21:41:39.798778 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:39.798740    5032 patch_prober.go:29] interesting pod/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" start-of-body=
Oct 30 21:41:39.799055 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:39.799024    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc" podUID="3b92d898-765d-445d-9056-9f256030162e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused"
Oct 30 21:41:39.802517 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:39.802479    5032 patch_prober.go:29] interesting pod/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Oct 30 21:41:39.802722 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:39.802687    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s" podUID="222c96b4-9c61-49c0-a3ed-450ea7d17b67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Oct 30 21:41:39.872593 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:39.872493    5032 patch_prober.go:29] interesting pod/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Oct 30 21:41:39.874634 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:39.874594    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww" podUID="63f89cda-14de-416c-9263-00ffdb8c9ec9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Oct 30 21:41:39.880408 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:39.880340    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg"]
Oct 30 21:41:39.893835 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:39.893807    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc"
Oct 30 21:41:39.897578 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:39.894421    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg"
Oct 30 21:41:39.899178 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:39.897965    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-xv7qv" podUID="4f6d23b8-1364-4b03-96c6-6649f3a0644a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.51:8000/ready\": context deadline exceeded"
Oct 30 21:41:39.903379 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:39.903333    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg"]
Oct 30 21:41:39.947973 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:39.947930    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/516c1abb-b9c5-4164-959b-4a565999db86-workload-socket\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg\" (UID: \"516c1abb-b9c5-4164-959b-4a565999db86\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg"
Oct 30 21:41:39.948187 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:39.948166    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/516c1abb-b9c5-4164-959b-4a565999db86-istio-envoy\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg\" (UID: \"516c1abb-b9c5-4164-959b-4a565999db86\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg"
Oct 30 21:41:39.948460 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:39.948428    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/516c1abb-b9c5-4164-959b-4a565999db86-istio-token\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg\" (UID: \"516c1abb-b9c5-4164-959b-4a565999db86\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg"
Oct 30 21:41:39.948691 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:39.948672    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/516c1abb-b9c5-4164-959b-4a565999db86-workload-certs\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg\" (UID: \"516c1abb-b9c5-4164-959b-4a565999db86\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg"
Oct 30 21:41:39.948848 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:39.948831    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/516c1abb-b9c5-4164-959b-4a565999db86-istiod-ca-cert\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg\" (UID: \"516c1abb-b9c5-4164-959b-4a565999db86\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg"
Oct 30 21:41:39.949012 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:39.948998    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/516c1abb-b9c5-4164-959b-4a565999db86-istio-data\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg\" (UID: \"516c1abb-b9c5-4164-959b-4a565999db86\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg"
Oct 30 21:41:39.949129 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:39.949116    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/516c1abb-b9c5-4164-959b-4a565999db86-istio-podinfo\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg\" (UID: \"516c1abb-b9c5-4164-959b-4a565999db86\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg"
Oct 30 21:41:39.949436 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:39.949420    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mrkp\" (UniqueName: \"kubernetes.io/projected/516c1abb-b9c5-4164-959b-4a565999db86-kube-api-access-4mrkp\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg\" (UID: \"516c1abb-b9c5-4164-959b-4a565999db86\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg"
Oct 30 21:41:39.952389 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:39.949637    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/516c1abb-b9c5-4164-959b-4a565999db86-credential-socket\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg\" (UID: \"516c1abb-b9c5-4164-959b-4a565999db86\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg"
Oct 30 21:41:40.054157 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:40.054117    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj"
Oct 30 21:41:40.057755 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:40.055799    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw"
Oct 30 21:41:40.057755 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:40.055869    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw"
Oct 30 21:41:40.093703 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:40.090412    5032 patch_prober.go:29] interesting pod/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Oct 30 21:41:40.093703 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:40.090465    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm" podUID="6b99e2de-e8a0-4566-b14c-dd627c74edf3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Oct 30 21:41:40.099717 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:40.095789    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx"
Oct 30 21:41:40.099717 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:40.096719    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc"
Oct 30 21:41:40.100052 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:40.100022    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/516c1abb-b9c5-4164-959b-4a565999db86-istio-data\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg\" (UID: \"516c1abb-b9c5-4164-959b-4a565999db86\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg"
Oct 30 21:41:40.100257 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:40.100233    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/516c1abb-b9c5-4164-959b-4a565999db86-istio-podinfo\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg\" (UID: \"516c1abb-b9c5-4164-959b-4a565999db86\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg"
Oct 30 21:41:40.100528 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:40.100505    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mrkp\" (UniqueName: \"kubernetes.io/projected/516c1abb-b9c5-4164-959b-4a565999db86-kube-api-access-4mrkp\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg\" (UID: \"516c1abb-b9c5-4164-959b-4a565999db86\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg"
Oct 30 21:41:40.100714 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:40.100697    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/516c1abb-b9c5-4164-959b-4a565999db86-credential-socket\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg\" (UID: \"516c1abb-b9c5-4164-959b-4a565999db86\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg"
Oct 30 21:41:40.100833 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:40.100806    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/516c1abb-b9c5-4164-959b-4a565999db86-istio-data\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg\" (UID: \"516c1abb-b9c5-4164-959b-4a565999db86\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg"
Oct 30 21:41:40.101117 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:40.100991    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/516c1abb-b9c5-4164-959b-4a565999db86-workload-socket\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg\" (UID: \"516c1abb-b9c5-4164-959b-4a565999db86\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg"
Oct 30 21:41:40.103426 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:40.103402    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/516c1abb-b9c5-4164-959b-4a565999db86-istio-envoy\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg\" (UID: \"516c1abb-b9c5-4164-959b-4a565999db86\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg"
Oct 30 21:41:40.103642 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:40.103620    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/516c1abb-b9c5-4164-959b-4a565999db86-istio-token\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg\" (UID: \"516c1abb-b9c5-4164-959b-4a565999db86\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg"
Oct 30 21:41:40.103866 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:40.103846    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/516c1abb-b9c5-4164-959b-4a565999db86-workload-certs\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg\" (UID: \"516c1abb-b9c5-4164-959b-4a565999db86\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg"
Oct 30 21:41:40.104070 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:40.104040    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/516c1abb-b9c5-4164-959b-4a565999db86-istiod-ca-cert\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg\" (UID: \"516c1abb-b9c5-4164-959b-4a565999db86\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg"
Oct 30 21:41:40.104410 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:40.103854    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/516c1abb-b9c5-4164-959b-4a565999db86-credential-socket\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg\" (UID: \"516c1abb-b9c5-4164-959b-4a565999db86\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg"
Oct 30 21:41:40.104903 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:40.104883    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/516c1abb-b9c5-4164-959b-4a565999db86-workload-socket\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg\" (UID: \"516c1abb-b9c5-4164-959b-4a565999db86\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg"
Oct 30 21:41:40.105201 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:40.105098    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/516c1abb-b9c5-4164-959b-4a565999db86-istiod-ca-cert\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg\" (UID: \"516c1abb-b9c5-4164-959b-4a565999db86\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg"
Oct 30 21:41:40.108688 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:40.105711    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/516c1abb-b9c5-4164-959b-4a565999db86-workload-certs\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg\" (UID: \"516c1abb-b9c5-4164-959b-4a565999db86\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg"
Oct 30 21:41:40.198785 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:40.196426    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/516c1abb-b9c5-4164-959b-4a565999db86-istio-podinfo\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg\" (UID: \"516c1abb-b9c5-4164-959b-4a565999db86\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg"
Oct 30 21:41:40.202017 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:40.200294    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/516c1abb-b9c5-4164-959b-4a565999db86-istio-envoy\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg\" (UID: \"516c1abb-b9c5-4164-959b-4a565999db86\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg"
Oct 30 21:41:40.202770 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:40.202747    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/516c1abb-b9c5-4164-959b-4a565999db86-istio-token\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg\" (UID: \"516c1abb-b9c5-4164-959b-4a565999db86\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg"
Oct 30 21:41:40.212661 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:40.212634    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mrkp\" (UniqueName: \"kubernetes.io/projected/516c1abb-b9c5-4164-959b-4a565999db86-kube-api-access-4mrkp\") pod \"e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg\" (UID: \"516c1abb-b9c5-4164-959b-4a565999db86\") " pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg"
Oct 30 21:41:40.276607 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:40.275755    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg"
Oct 30 21:41:40.347056 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:40.347023    5032 patch_prober.go:29] interesting pod/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Oct 30 21:41:40.347321 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:40.347296    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2" podUID="57dfd470-9c35-404f-b967-14a9103adeea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Oct 30 21:41:40.773599 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:40.772292    5032 patch_prober.go:29] interesting pod/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Oct 30 21:41:40.773599 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:40.772350    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s" podUID="222c96b4-9c61-49c0-a3ed-450ea7d17b67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Oct 30 21:41:40.999216 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:40.999173    5032 patch_prober.go:29] interesting pod/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Oct 30 21:41:40.999730 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:40.999239    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm" podUID="6b99e2de-e8a0-4566-b14c-dd627c74edf3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Oct 30 21:41:41.105385 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:41.100232    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:41:41.105695 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:41.105658    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:41:41.105864 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:41.105843    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:41:41.163112 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:41.163073    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg"]
Oct 30 21:41:41.537438 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:41.525638    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc"
Oct 30 21:41:41.537438 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:41.533243    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc"
Oct 30 21:41:41.570894 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:41.563163    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg" event={"ID":"516c1abb-b9c5-4164-959b-4a565999db86","Type":"ContainerStarted","Data":"60d872f16bcd12192aa1cb51eba571d228308ec92e497b818f88a27523d1eed9"}
Oct 30 21:41:41.853083 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:41.840646    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww"
Oct 30 21:41:41.853083 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:41.842268    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww"
Oct 30 21:41:41.856038 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:41.855963    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww"
Oct 30 21:41:41.990479 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:41.990446    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s"
Oct 30 21:41:42.049486 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:42.049443    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm"
Oct 30 21:41:42.091655 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:42.091181    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm"
Oct 30 21:41:42.116430 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:42.111669    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm"
Oct 30 21:41:42.123621 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:42.123281    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s"
Oct 30 21:41:42.294340 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:42.294311    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2"
Oct 30 21:41:42.294801 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:42.294764    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2"
Oct 30 21:41:42.295704 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:42.295686    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2"
Oct 30 21:41:42.572302 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:42.572267    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg" event={"ID":"516c1abb-b9c5-4164-959b-4a565999db86","Type":"ContainerStarted","Data":"ac9bedb91419f3bd1c2cd8f9d1e899dbba2d139f4cc35930d99881328eb48191"}
Oct 30 21:41:42.573708 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:42.573686    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s"
Oct 30 21:41:42.596533 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:42.596489    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg" podStartSLOduration=3.596476252 podStartE2EDuration="3.596476252s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:41:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:41:42.595711549 +0000 UTC m=+815.397524018" watchObservedRunningTime="2026-10-30 21:41:42.596476252 +0000 UTC m=+815.398288715"
Oct 30 21:41:43.283839 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:43.283789    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg"
Oct 30 21:41:43.285236 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:43.285212    5032 patch_prober.go:29] interesting pod/e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused" start-of-body=
Oct 30 21:41:43.285340 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:43.285264    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg" podUID="516c1abb-b9c5-4164-959b-4a565999db86" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused"
Oct 30 21:41:44.283602 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:44.283518    5032 patch_prober.go:29] interesting pod/e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused" start-of-body=
Oct 30 21:41:44.283804 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:44.283631    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg" podUID="516c1abb-b9c5-4164-959b-4a565999db86" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused"
Oct 30 21:41:45.288934 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:45.285820    5032 patch_prober.go:29] interesting pod/e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused" start-of-body=
Oct 30 21:41:45.288934 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:45.285902    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg" podUID="516c1abb-b9c5-4164-959b-4a565999db86" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused"
Oct 30 21:41:46.283243 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:46.283206    5032 patch_prober.go:29] interesting pod/e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused" start-of-body=
Oct 30 21:41:46.283452 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:46.283263    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg" podUID="516c1abb-b9c5-4164-959b-4a565999db86" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused"
Oct 30 21:41:47.283947 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:47.283907    5032 patch_prober.go:29] interesting pod/e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused" start-of-body=
Oct 30 21:41:47.284433 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:47.283976    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg" podUID="516c1abb-b9c5-4164-959b-4a565999db86" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused"
Oct 30 21:41:48.284140 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:48.284103    5032 patch_prober.go:29] interesting pod/e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused" start-of-body=
Oct 30 21:41:48.284627 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:48.284171    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg" podUID="516c1abb-b9c5-4164-959b-4a565999db86" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused"
Oct 30 21:41:48.369591 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:41:48.369549    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 21:41:49.285722 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:49.285605    5032 patch_prober.go:29] interesting pod/e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused" start-of-body=
Oct 30 21:41:49.285722 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:49.285678    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg" podUID="516c1abb-b9c5-4164-959b-4a565999db86" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused"
Oct 30 21:41:50.288845 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:50.288814    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg"
Oct 30 21:41:50.289424 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:50.289279    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg"
Oct 30 21:41:50.290005 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:50.289986    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg"
Oct 30 21:41:54.025777 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:54.025740    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n"]
Oct 30 21:41:54.031412 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:54.031384    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n"
Oct 30 21:41:54.049131 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:54.049096    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n"]
Oct 30 21:41:54.117915 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:54.117872    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74856e06-9b6f-425d-90f8-e973d0649f90-workload-certs\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n\" (UID: \"74856e06-9b6f-425d-90f8-e973d0649f90\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n"
Oct 30 21:41:54.118134 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:54.118012    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2n82w\" (UniqueName: \"kubernetes.io/projected/74856e06-9b6f-425d-90f8-e973d0649f90-kube-api-access-2n82w\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n\" (UID: \"74856e06-9b6f-425d-90f8-e973d0649f90\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n"
Oct 30 21:41:54.118213 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:54.118177    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74856e06-9b6f-425d-90f8-e973d0649f90-workload-socket\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n\" (UID: \"74856e06-9b6f-425d-90f8-e973d0649f90\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n"
Oct 30 21:41:54.118280 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:54.118252    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74856e06-9b6f-425d-90f8-e973d0649f90-istio-envoy\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n\" (UID: \"74856e06-9b6f-425d-90f8-e973d0649f90\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n"
Oct 30 21:41:54.118445 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:54.118415    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74856e06-9b6f-425d-90f8-e973d0649f90-istio-podinfo\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n\" (UID: \"74856e06-9b6f-425d-90f8-e973d0649f90\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n"
Oct 30 21:41:54.118529 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:54.118465    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74856e06-9b6f-425d-90f8-e973d0649f90-credential-socket\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n\" (UID: \"74856e06-9b6f-425d-90f8-e973d0649f90\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n"
Oct 30 21:41:54.118640 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:54.118615    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74856e06-9b6f-425d-90f8-e973d0649f90-istio-token\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n\" (UID: \"74856e06-9b6f-425d-90f8-e973d0649f90\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n"
Oct 30 21:41:54.118709 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:54.118696    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74856e06-9b6f-425d-90f8-e973d0649f90-istiod-ca-cert\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n\" (UID: \"74856e06-9b6f-425d-90f8-e973d0649f90\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n"
Oct 30 21:41:54.118768 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:54.118722    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74856e06-9b6f-425d-90f8-e973d0649f90-istio-data\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n\" (UID: \"74856e06-9b6f-425d-90f8-e973d0649f90\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n"
Oct 30 21:41:54.220132 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:54.220099    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2n82w\" (UniqueName: \"kubernetes.io/projected/74856e06-9b6f-425d-90f8-e973d0649f90-kube-api-access-2n82w\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n\" (UID: \"74856e06-9b6f-425d-90f8-e973d0649f90\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n"
Oct 30 21:41:54.220341 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:54.220275    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74856e06-9b6f-425d-90f8-e973d0649f90-workload-socket\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n\" (UID: \"74856e06-9b6f-425d-90f8-e973d0649f90\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n"
Oct 30 21:41:54.220482 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:54.220409    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74856e06-9b6f-425d-90f8-e973d0649f90-istio-envoy\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n\" (UID: \"74856e06-9b6f-425d-90f8-e973d0649f90\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n"
Oct 30 21:41:54.220482 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:54.220471    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74856e06-9b6f-425d-90f8-e973d0649f90-istio-podinfo\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n\" (UID: \"74856e06-9b6f-425d-90f8-e973d0649f90\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n"
Oct 30 21:41:54.220580 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:54.220499    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74856e06-9b6f-425d-90f8-e973d0649f90-credential-socket\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n\" (UID: \"74856e06-9b6f-425d-90f8-e973d0649f90\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n"
Oct 30 21:41:54.220643 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:54.220621    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74856e06-9b6f-425d-90f8-e973d0649f90-istio-token\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n\" (UID: \"74856e06-9b6f-425d-90f8-e973d0649f90\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n"
Oct 30 21:41:54.220709 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:54.220691    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74856e06-9b6f-425d-90f8-e973d0649f90-istiod-ca-cert\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n\" (UID: \"74856e06-9b6f-425d-90f8-e973d0649f90\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n"
Oct 30 21:41:54.220769 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:54.220722    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74856e06-9b6f-425d-90f8-e973d0649f90-workload-socket\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n\" (UID: \"74856e06-9b6f-425d-90f8-e973d0649f90\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n"
Oct 30 21:41:54.220769 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:54.220734    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74856e06-9b6f-425d-90f8-e973d0649f90-istio-data\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n\" (UID: \"74856e06-9b6f-425d-90f8-e973d0649f90\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n"
Oct 30 21:41:54.220895 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:54.220880    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74856e06-9b6f-425d-90f8-e973d0649f90-workload-certs\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n\" (UID: \"74856e06-9b6f-425d-90f8-e973d0649f90\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n"
Oct 30 21:41:54.221349 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:54.221289    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74856e06-9b6f-425d-90f8-e973d0649f90-credential-socket\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n\" (UID: \"74856e06-9b6f-425d-90f8-e973d0649f90\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n"
Oct 30 21:41:54.221349 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:54.221315    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74856e06-9b6f-425d-90f8-e973d0649f90-workload-certs\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n\" (UID: \"74856e06-9b6f-425d-90f8-e973d0649f90\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n"
Oct 30 21:41:54.221511 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:54.221465    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74856e06-9b6f-425d-90f8-e973d0649f90-istiod-ca-cert\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n\" (UID: \"74856e06-9b6f-425d-90f8-e973d0649f90\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n"
Oct 30 21:41:54.221723 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:54.221702    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74856e06-9b6f-425d-90f8-e973d0649f90-istio-data\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n\" (UID: \"74856e06-9b6f-425d-90f8-e973d0649f90\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n"
Oct 30 21:41:54.222929 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:54.222907    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74856e06-9b6f-425d-90f8-e973d0649f90-istio-envoy\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n\" (UID: \"74856e06-9b6f-425d-90f8-e973d0649f90\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n"
Oct 30 21:41:54.223863 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:54.223843    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74856e06-9b6f-425d-90f8-e973d0649f90-istio-podinfo\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n\" (UID: \"74856e06-9b6f-425d-90f8-e973d0649f90\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n"
Oct 30 21:41:54.232936 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:54.232909    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74856e06-9b6f-425d-90f8-e973d0649f90-istio-token\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n\" (UID: \"74856e06-9b6f-425d-90f8-e973d0649f90\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n"
Oct 30 21:41:54.233054 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:54.233039    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2n82w\" (UniqueName: \"kubernetes.io/projected/74856e06-9b6f-425d-90f8-e973d0649f90-kube-api-access-2n82w\") pod \"e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n\" (UID: \"74856e06-9b6f-425d-90f8-e973d0649f90\") " pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n"
Oct 30 21:41:54.352841 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:54.352753    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n"
Oct 30 21:41:54.361985 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:54.361953    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp"]
Oct 30 21:41:54.370860 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:54.370641    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp"
Oct 30 21:41:54.431751 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:54.428982    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnjfc\" (UniqueName: \"kubernetes.io/projected/86937e06-85cc-48d5-9935-9ae922c3518c-kube-api-access-xnjfc\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp\" (UID: \"86937e06-85cc-48d5-9935-9ae922c3518c\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp"
Oct 30 21:41:54.431751 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:54.429052    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86937e06-85cc-48d5-9935-9ae922c3518c-credential-socket\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp\" (UID: \"86937e06-85cc-48d5-9935-9ae922c3518c\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp"
Oct 30 21:41:54.440445 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:54.435076    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp"]
Oct 30 21:41:54.440445 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:54.435218    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86937e06-85cc-48d5-9935-9ae922c3518c-istio-envoy\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp\" (UID: \"86937e06-85cc-48d5-9935-9ae922c3518c\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp"
Oct 30 21:41:54.440445 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:54.435258    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86937e06-85cc-48d5-9935-9ae922c3518c-istio-data\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp\" (UID: \"86937e06-85cc-48d5-9935-9ae922c3518c\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp"
Oct 30 21:41:54.440445 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:54.435292    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86937e06-85cc-48d5-9935-9ae922c3518c-workload-socket\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp\" (UID: \"86937e06-85cc-48d5-9935-9ae922c3518c\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp"
Oct 30 21:41:54.440445 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:54.435323    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86937e06-85cc-48d5-9935-9ae922c3518c-workload-certs\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp\" (UID: \"86937e06-85cc-48d5-9935-9ae922c3518c\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp"
Oct 30 21:41:54.440445 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:54.435454    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86937e06-85cc-48d5-9935-9ae922c3518c-istio-token\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp\" (UID: \"86937e06-85cc-48d5-9935-9ae922c3518c\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp"
Oct 30 21:41:54.440445 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:54.435634    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86937e06-85cc-48d5-9935-9ae922c3518c-istiod-ca-cert\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp\" (UID: \"86937e06-85cc-48d5-9935-9ae922c3518c\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp"
Oct 30 21:41:54.440445 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:54.435731    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86937e06-85cc-48d5-9935-9ae922c3518c-istio-podinfo\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp\" (UID: \"86937e06-85cc-48d5-9935-9ae922c3518c\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp"
Oct 30 21:41:54.537186 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:54.537152    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86937e06-85cc-48d5-9935-9ae922c3518c-istio-envoy\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp\" (UID: \"86937e06-85cc-48d5-9935-9ae922c3518c\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp"
Oct 30 21:41:54.537356 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:54.537200    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86937e06-85cc-48d5-9935-9ae922c3518c-istio-data\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp\" (UID: \"86937e06-85cc-48d5-9935-9ae922c3518c\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp"
Oct 30 21:41:54.537356 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:54.537234    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86937e06-85cc-48d5-9935-9ae922c3518c-workload-socket\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp\" (UID: \"86937e06-85cc-48d5-9935-9ae922c3518c\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp"
Oct 30 21:41:54.537356 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:54.537268    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86937e06-85cc-48d5-9935-9ae922c3518c-workload-certs\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp\" (UID: \"86937e06-85cc-48d5-9935-9ae922c3518c\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp"
Oct 30 21:41:54.537550 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:54.537377    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86937e06-85cc-48d5-9935-9ae922c3518c-istio-token\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp\" (UID: \"86937e06-85cc-48d5-9935-9ae922c3518c\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp"
Oct 30 21:41:54.537550 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:54.537496    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86937e06-85cc-48d5-9935-9ae922c3518c-istiod-ca-cert\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp\" (UID: \"86937e06-85cc-48d5-9935-9ae922c3518c\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp"
Oct 30 21:41:54.537659 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:54.537579    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86937e06-85cc-48d5-9935-9ae922c3518c-istio-podinfo\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp\" (UID: \"86937e06-85cc-48d5-9935-9ae922c3518c\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp"
Oct 30 21:41:54.537715 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:54.537684    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnjfc\" (UniqueName: \"kubernetes.io/projected/86937e06-85cc-48d5-9935-9ae922c3518c-kube-api-access-xnjfc\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp\" (UID: \"86937e06-85cc-48d5-9935-9ae922c3518c\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp"
Oct 30 21:41:54.537772 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:54.537736    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86937e06-85cc-48d5-9935-9ae922c3518c-credential-socket\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp\" (UID: \"86937e06-85cc-48d5-9935-9ae922c3518c\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp"
Oct 30 21:41:54.538429 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:54.538222    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86937e06-85cc-48d5-9935-9ae922c3518c-credential-socket\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp\" (UID: \"86937e06-85cc-48d5-9935-9ae922c3518c\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp"
Oct 30 21:41:54.548702 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:54.546992    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86937e06-85cc-48d5-9935-9ae922c3518c-workload-certs\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp\" (UID: \"86937e06-85cc-48d5-9935-9ae922c3518c\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp"
Oct 30 21:41:54.548702 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:54.547993    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86937e06-85cc-48d5-9935-9ae922c3518c-workload-socket\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp\" (UID: \"86937e06-85cc-48d5-9935-9ae922c3518c\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp"
Oct 30 21:41:54.548702 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:54.548099    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86937e06-85cc-48d5-9935-9ae922c3518c-istio-data\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp\" (UID: \"86937e06-85cc-48d5-9935-9ae922c3518c\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp"
Oct 30 21:41:54.548702 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:54.548655    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86937e06-85cc-48d5-9935-9ae922c3518c-istiod-ca-cert\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp\" (UID: \"86937e06-85cc-48d5-9935-9ae922c3518c\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp"
Oct 30 21:41:54.556328 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:54.554513    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnjfc\" (UniqueName: \"kubernetes.io/projected/86937e06-85cc-48d5-9935-9ae922c3518c-kube-api-access-xnjfc\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp\" (UID: \"86937e06-85cc-48d5-9935-9ae922c3518c\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp"
Oct 30 21:41:54.559834 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:54.557063    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86937e06-85cc-48d5-9935-9ae922c3518c-istio-envoy\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp\" (UID: \"86937e06-85cc-48d5-9935-9ae922c3518c\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp"
Oct 30 21:41:54.559834 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:54.558743    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86937e06-85cc-48d5-9935-9ae922c3518c-istio-token\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp\" (UID: \"86937e06-85cc-48d5-9935-9ae922c3518c\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp"
Oct 30 21:41:54.568215 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:54.562509    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86937e06-85cc-48d5-9935-9ae922c3518c-istio-podinfo\") pod \"e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp\" (UID: \"86937e06-85cc-48d5-9935-9ae922c3518c\") " pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp"
Oct 30 21:41:54.759377 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:41:54.759336    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74856e06_9b6f_425d_90f8_e973d0649f90.slice/crio-e14161b3b44b755cfa03f984976e4fb752d989d61df4dec8db5af67806ee9f27 WatchSource:0}: Error finding container e14161b3b44b755cfa03f984976e4fb752d989d61df4dec8db5af67806ee9f27: Status 404 returned error can't find the container with id e14161b3b44b755cfa03f984976e4fb752d989d61df4dec8db5af67806ee9f27
Oct 30 21:41:54.759497 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:54.759441    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n"]
Oct 30 21:41:54.762482 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:54.762445    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:41:54.762583 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:54.762521    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:41:54.762583 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:54.762556    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:41:54.772353 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:54.772327    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp"
Oct 30 21:41:55.073706 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:55.073673    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp"]
Oct 30 21:41:55.075299 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:41:55.075267    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86937e06_85cc_48d5_9935_9ae922c3518c.slice/crio-34c23ade960d9c6231d94ca2cd5bab37d0be190bf53b0a5da0b282bf7fb66590 WatchSource:0}: Error finding container 34c23ade960d9c6231d94ca2cd5bab37d0be190bf53b0a5da0b282bf7fb66590: Status 404 returned error can't find the container with id 34c23ade960d9c6231d94ca2cd5bab37d0be190bf53b0a5da0b282bf7fb66590
Oct 30 21:41:55.078744 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:55.078699    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:41:55.078854 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:55.078797    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:41:55.078854 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:55.078840    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:41:55.759080 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:55.759032    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n" event={"ID":"74856e06-9b6f-425d-90f8-e973d0649f90","Type":"ContainerStarted","Data":"0b6751f9d63c1c06407f8c38e60aa49c0ccd14d070648e6e8d7e55147b6c1aac"}
Oct 30 21:41:55.759264 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:55.759180    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n" event={"ID":"74856e06-9b6f-425d-90f8-e973d0649f90","Type":"ContainerStarted","Data":"e14161b3b44b755cfa03f984976e4fb752d989d61df4dec8db5af67806ee9f27"}
Oct 30 21:41:55.763028 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:55.762991    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp" event={"ID":"86937e06-85cc-48d5-9935-9ae922c3518c","Type":"ContainerStarted","Data":"bbd3e0f1b0c6a756b516f0adcb34e4c1de51ead6a715f5678a60ef4ded05fe40"}
Oct 30 21:41:55.763224 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:55.763058    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp" event={"ID":"86937e06-85cc-48d5-9935-9ae922c3518c","Type":"ContainerStarted","Data":"34c23ade960d9c6231d94ca2cd5bab37d0be190bf53b0a5da0b282bf7fb66590"}
Oct 30 21:41:55.773178 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:55.773141    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp"
Oct 30 21:41:55.793657 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:55.793604    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n" podStartSLOduration=2.79359008 podStartE2EDuration="2.79359008s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:41:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:41:55.78790533 +0000 UTC m=+828.589717796" watchObservedRunningTime="2026-10-30 21:41:55.79359008 +0000 UTC m=+828.595402541"
Oct 30 21:41:55.835384 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:55.834526    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp" podStartSLOduration=1.834500204 podStartE2EDuration="1.834500204s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:41:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:41:55.823157574 +0000 UTC m=+828.624970049" watchObservedRunningTime="2026-10-30 21:41:55.834500204 +0000 UTC m=+828.636312670"
Oct 30 21:41:56.358833 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:56.358786    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n"
Oct 30 21:41:56.363555 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:56.363520    5032 patch_prober.go:29] interesting pod/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused" start-of-body=
Oct 30 21:41:56.363846 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:56.363768    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n" podUID="74856e06-9b6f-425d-90f8-e973d0649f90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused"
Oct 30 21:41:56.774486 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:56.774438    5032 patch_prober.go:29] interesting pod/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused" start-of-body=
Oct 30 21:41:56.774823 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:56.774774    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp" podUID="86937e06-85cc-48d5-9935-9ae922c3518c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused"
Oct 30 21:41:57.374147 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:57.374104    5032 patch_prober.go:29] interesting pod/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused" start-of-body=
Oct 30 21:41:57.374700 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:57.374165    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n" podUID="74856e06-9b6f-425d-90f8-e973d0649f90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused"
Oct 30 21:41:57.773134 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:57.773098    5032 patch_prober.go:29] interesting pod/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused" start-of-body=
Oct 30 21:41:57.773456 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:57.773416    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp" podUID="86937e06-85cc-48d5-9935-9ae922c3518c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused"
Oct 30 21:41:58.356385 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:58.354656    5032 patch_prober.go:29] interesting pod/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused" start-of-body=
Oct 30 21:41:58.356385 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:58.354725    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n" podUID="74856e06-9b6f-425d-90f8-e973d0649f90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused"
Oct 30 21:41:58.773310 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:58.773278    5032 patch_prober.go:29] interesting pod/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused" start-of-body=
Oct 30 21:41:58.773806 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:58.773777    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp" podUID="86937e06-85cc-48d5-9935-9ae922c3518c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused"
Oct 30 21:41:59.281610 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:59.279171    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 21:41:59.373626 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:59.373587    5032 patch_prober.go:29] interesting pod/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused" start-of-body=
Oct 30 21:41:59.375702 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:59.375665    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n" podUID="74856e06-9b6f-425d-90f8-e973d0649f90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused"
Oct 30 21:41:59.772788 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:59.772751    5032 patch_prober.go:29] interesting pod/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused" start-of-body=
Oct 30 21:41:59.773061 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:41:59.773029    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp" podUID="86937e06-85cc-48d5-9935-9ae922c3518c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused"
Oct 30 21:42:00.415387 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:00.395492    5032 patch_prober.go:29] interesting pod/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused" start-of-body=
Oct 30 21:42:00.415387 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:00.395606    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n" podUID="74856e06-9b6f-425d-90f8-e973d0649f90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused"
Oct 30 21:42:00.779444 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:00.774602    5032 patch_prober.go:29] interesting pod/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused" start-of-body=
Oct 30 21:42:00.779444 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:00.774684    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp" podUID="86937e06-85cc-48d5-9935-9ae922c3518c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused"
Oct 30 21:42:01.358390 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:01.357468    5032 patch_prober.go:29] interesting pod/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused" start-of-body=
Oct 30 21:42:01.358390 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:01.357531    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n" podUID="74856e06-9b6f-425d-90f8-e973d0649f90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused"
Oct 30 21:42:01.482386 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:01.477447    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg" podUID="2d0ea058-b6ca-4181-8823-2b719883e545" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": context deadline exceeded"
Oct 30 21:42:01.515859 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:01.507173    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw" podUID="2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:02.038706 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:02.038662    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw" podUID="914f7544-6ef8-4496-b98d-b7ebc4123cda" containerName="main" probeResult="failure" output="Get \"https://10.132.0.86:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:02.072409 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:02.067686    5032 patch_prober.go:29] interesting pod/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused" start-of-body=
Oct 30 21:42:02.072409 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:02.067764    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp" podUID="86937e06-85cc-48d5-9935-9ae922c3518c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused"
Oct 30 21:42:02.093424 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:02.079466    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj" podUID="d0ff5023-fd3f-4734-94c6-76b666010adf" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:02.265674 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:02.265626    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc" podUID="23b97dbb-8488-4c44-a810-882ea18ed919" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:02.358456 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:02.358341    5032 patch_prober.go:29] interesting pod/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused" start-of-body=
Oct 30 21:42:02.359220 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:02.359190    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n" podUID="74856e06-9b6f-425d-90f8-e973d0649f90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused"
Oct 30 21:42:02.465428 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:02.459425    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx" podUID="34fdf57d-86f6-474f-a4f6-811e5063c405" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:02.706288 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:02.672556    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw" podUID="2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:02.706990 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:02.706945    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw"
Oct 30 21:42:02.764648 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:02.764587    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg" podUID="2d0ea058-b6ca-4181-8823-2b719883e545" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": context deadline exceeded"
Oct 30 21:42:02.793407 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:02.765909    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg"
Oct 30 21:42:02.822409 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:02.796819    5032 patch_prober.go:29] interesting pod/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused" start-of-body=
Oct 30 21:42:02.822409 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:02.796877    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp" podUID="86937e06-85cc-48d5-9935-9ae922c3518c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused"
Oct 30 21:42:03.051174 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:03.050979    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw" podUID="914f7544-6ef8-4496-b98d-b7ebc4123cda" containerName="main" probeResult="failure" output="Get \"https://10.132.0.86:8000/ready\": context deadline exceeded"
Oct 30 21:42:03.124414 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:03.085428    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw"
Oct 30 21:42:03.170403 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:03.147045    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj" podUID="d0ff5023-fd3f-4734-94c6-76b666010adf" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:03.170403 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:03.153416    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj"
Oct 30 21:42:03.353446 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:03.283961    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc" podUID="23b97dbb-8488-4c44-a810-882ea18ed919" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:03.357815 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:03.357784    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc"
Oct 30 21:42:03.496395 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:03.472143    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx" podUID="34fdf57d-86f6-474f-a4f6-811e5063c405" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": context deadline exceeded"
Oct 30 21:42:03.537626 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:03.537591    5032 patch_prober.go:29] interesting pod/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused" start-of-body=
Oct 30 21:42:03.538269 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:03.538234    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n" podUID="74856e06-9b6f-425d-90f8-e973d0649f90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused"
Oct 30 21:42:03.681527 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:03.679792    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw" podUID="2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:03.826404 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:03.818303    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg" podUID="2d0ea058-b6ca-4181-8823-2b719883e545" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:03.826404 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:03.822346    5032 patch_prober.go:29] interesting pod/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused" start-of-body=
Oct 30 21:42:03.826404 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:03.822419    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp" podUID="86937e06-85cc-48d5-9935-9ae922c3518c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused"
Oct 30 21:42:04.096501 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:04.096449    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw" podUID="914f7544-6ef8-4496-b98d-b7ebc4123cda" containerName="main" probeResult="failure" output="Get \"https://10.132.0.86:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:04.124388 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:04.099668    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx"
Oct 30 21:42:04.412554 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:04.195064    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj" podUID="d0ff5023-fd3f-4734-94c6-76b666010adf" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:04.415636 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:04.363225    5032 patch_prober.go:29] interesting pod/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused" start-of-body=
Oct 30 21:42:04.415636 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:04.413579    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n" podUID="74856e06-9b6f-425d-90f8-e973d0649f90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused"
Oct 30 21:42:04.415636 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:04.363273    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc" podUID="23b97dbb-8488-4c44-a810-882ea18ed919" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:04.415636 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:04.384651    5032 patch_prober.go:29] interesting pod/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:42:04.415636 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:04.414215    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2" podUID="62000861-c573-4dba-8bca-c6f962057f5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:04.553416 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:04.553323    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx" podUID="34fdf57d-86f6-474f-a4f6-811e5063c405" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": context deadline exceeded"
Oct 30 21:42:04.687798 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:04.687681    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw" podUID="2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:04.841254 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:04.841203    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg" podUID="2d0ea058-b6ca-4181-8823-2b719883e545" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": context deadline exceeded"
Oct 30 21:42:05.129711 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:04.893422    5032 patch_prober.go:29] interesting pod/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused" start-of-body=
Oct 30 21:42:05.138386 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:05.131038    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp" podUID="86937e06-85cc-48d5-9935-9ae922c3518c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused"
Oct 30 21:42:05.139401 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:05.139346    5032 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj" podUID="d0ff5023-fd3f-4734-94c6-76b666010adf" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:05.147385 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:05.143983    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw" podUID="914f7544-6ef8-4496-b98d-b7ebc4123cda" containerName="main" probeResult="failure" output="Get \"https://10.132.0.86:8000/ready\": context deadline exceeded"
Oct 30 21:42:05.368164 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:05.366471    5032 patch_prober.go:29] interesting pod/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused" start-of-body=
Oct 30 21:42:05.368164 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:05.366531    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n" podUID="74856e06-9b6f-425d-90f8-e973d0649f90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused"
Oct 30 21:42:05.465860 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:05.465777    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj" podUID="d0ff5023-fd3f-4734-94c6-76b666010adf" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": context deadline exceeded"
Oct 30 21:42:05.554984 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:05.554930    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc" podUID="23b97dbb-8488-4c44-a810-882ea18ed919" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": context deadline exceeded"
Oct 30 21:42:05.660740 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:05.660684    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx" podUID="34fdf57d-86f6-474f-a4f6-811e5063c405" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": context deadline exceeded"
Oct 30 21:42:05.774447 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:05.692506    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw" podUID="2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded"
Oct 30 21:42:06.140446 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:06.115663    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg" podUID="2d0ea058-b6ca-4181-8823-2b719883e545" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:06.237391 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:06.170492    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw" podUID="914f7544-6ef8-4496-b98d-b7ebc4123cda" containerName="main" probeResult="failure" output="Get \"https://10.132.0.86:8000/ready\": context deadline exceeded"
Oct 30 21:42:06.390400 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:06.382897    5032 patch_prober.go:29] interesting pod/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused" start-of-body=
Oct 30 21:42:06.390400 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:06.382960    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n" podUID="74856e06-9b6f-425d-90f8-e973d0649f90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused"
Oct 30 21:42:06.411485 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:06.399351    5032 patch_prober.go:29] interesting pod/e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.98:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:42:06.411485 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:06.399422    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg" podUID="516c1abb-b9c5-4164-959b-4a565999db86" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.98:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:06.548394 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:06.548337    5032 patch_prober.go:29] interesting pod/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused" start-of-body=
Oct 30 21:42:06.577902 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:06.577854    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp" podUID="86937e06-85cc-48d5-9935-9ae922c3518c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused"
Oct 30 21:42:06.640397 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:06.578152    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc" podUID="23b97dbb-8488-4c44-a810-882ea18ed919" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:06.650386 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:06.518433    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj" podUID="d0ff5023-fd3f-4734-94c6-76b666010adf" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:06.666570 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:06.656509    5032 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw" podUID="2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:06.747777 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:06.747495    5032 patch_prober.go:29] interesting pod/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:42:06.748274 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:06.748243    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv" podUID="30ea822a-85f1-4db5-a3a2-b8edd78d1200" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:06.808710 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:06.808670    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx" podUID="34fdf57d-86f6-474f-a4f6-811e5063c405" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": context deadline exceeded"
Oct 30 21:42:06.809293 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:06.809259    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw" podUID="2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded"
Oct 30 21:42:06.813622 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:06.813590    5032 patch_prober.go:29] interesting pod/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused" start-of-body=
Oct 30 21:42:06.906417 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:06.890130    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp" podUID="86937e06-85cc-48d5-9935-9ae922c3518c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused"
Oct 30 21:42:07.034490 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:07.034447    5032 patch_prober.go:29] interesting pod/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:42:07.056323 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:07.056279    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8" podUID="e7679391-8575-43d7-b7f1-5b1ab0bcee5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:07.204594 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:07.204540    5032 patch_prober.go:29] interesting pod/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:42:07.214527 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:07.214477    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj" podUID="4577b3de-b9e4-47a7-87e0-bcba13b11bbe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:07.274868 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:07.262464    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw" podUID="914f7544-6ef8-4496-b98d-b7ebc4123cda" containerName="main" probeResult="failure" output="Get \"https://10.132.0.86:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:07.274868 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:07.262953    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg" podUID="2d0ea058-b6ca-4181-8823-2b719883e545" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": context deadline exceeded"
Oct 30 21:42:07.649387 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:07.648830    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc" podUID="23b97dbb-8488-4c44-a810-882ea18ed919" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:07.688388 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:07.686048    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj" podUID="d0ff5023-fd3f-4734-94c6-76b666010adf" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:07.780417 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:07.778144    5032 patch_prober.go:29] interesting pod/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused" start-of-body=
Oct 30 21:42:07.780417 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:07.778204    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp" podUID="86937e06-85cc-48d5-9935-9ae922c3518c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused"
Oct 30 21:42:07.810385 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:07.809282    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx" podUID="34fdf57d-86f6-474f-a4f6-811e5063c405" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:07.810385 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:07.809630    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw" podUID="2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded"
Oct 30 21:42:07.875706 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:07.875665    5032 patch_prober.go:29] interesting pod/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:42:07.876012 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:07.875979    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz" podUID="ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:07.904444 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:07.888254    5032 patch_prober.go:29] interesting pod/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:42:07.904444 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:07.902571    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv" podUID="aed33297-3e27-4d4a-a767-4191e8bc5127" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:08.301419 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:08.266562    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg" podUID="2d0ea058-b6ca-4181-8823-2b719883e545" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:08.301419 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:08.266845    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw" podUID="914f7544-6ef8-4496-b98d-b7ebc4123cda" containerName="main" probeResult="failure" output="Get \"https://10.132.0.86:8000/ready\": context deadline exceeded"
Oct 30 21:42:08.459498 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:08.459456    5032 patch_prober.go:29] interesting pod/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:42:08.474389 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:08.459764    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n" podUID="74856e06-9b6f-425d-90f8-e973d0649f90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:08.583081 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:42:08.571631    5032 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4577b3de_b9e4_47a7_87e0_bcba13b11bbe.slice/crio-57c7c539da9fc4d0ce4e272f8504d17cfc38051e78794d02ced2ca7bc8643e96\": RecentStats: unable to find data in memory cache]"
Oct 30 21:42:08.800420 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:08.796535    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj" podUID="d0ff5023-fd3f-4734-94c6-76b666010adf" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:08.800420 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:08.797420    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc" podUID="23b97dbb-8488-4c44-a810-882ea18ed919" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": context deadline exceeded"
Oct 30 21:42:08.820462 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:08.814458    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw" podUID="2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:08.820462 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:08.819466    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx" podUID="34fdf57d-86f6-474f-a4f6-811e5063c405" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": context deadline exceeded"
Oct 30 21:42:08.893581 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:08.893224    5032 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc" podUID="23b97dbb-8488-4c44-a810-882ea18ed919" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/health\": context deadline exceeded"
Oct 30 21:42:09.292402 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:09.288505    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw" podUID="914f7544-6ef8-4496-b98d-b7ebc4123cda" containerName="main" probeResult="failure" output="Get \"https://10.132.0.86:8000/ready\": context deadline exceeded"
Oct 30 21:42:09.292402 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:09.288897    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg" podUID="2d0ea058-b6ca-4181-8823-2b719883e545" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": context deadline exceeded"
Oct 30 21:42:09.376481 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:09.374652    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n"
Oct 30 21:42:09.557185 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:09.557079    5032 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx" podUID="34fdf57d-86f6-474f-a4f6-811e5063c405" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:09.748386 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:09.691030    5032 patch_prober.go:29] interesting pod/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:42:09.748386 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:09.748176    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n" podUID="74856e06-9b6f-425d-90f8-e973d0649f90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:09.821161 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:09.821053    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx" podUID="34fdf57d-86f6-474f-a4f6-811e5063c405" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:09.825022 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:09.824981    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp"
Oct 30 21:42:09.867385 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:09.862922    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw" podUID="2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded"
Oct 30 21:42:09.937925 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:09.926052    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj" podUID="d0ff5023-fd3f-4734-94c6-76b666010adf" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": context deadline exceeded"
Oct 30 21:42:09.937925 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:09.927085    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc" podUID="23b97dbb-8488-4c44-a810-882ea18ed919" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": context deadline exceeded"
Oct 30 21:42:09.994881 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:09.980629    5032 patch_prober.go:29] interesting pod/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.100:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:42:09.994881 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:09.980697    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp" podUID="86937e06-85cc-48d5-9935-9ae922c3518c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.100:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:10.142957 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:10.140312    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n"
Oct 30 21:42:10.191393 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:10.189263    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg"
Oct 30 21:42:10.218337 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:10.218295    5032 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg" podUID="2d0ea058-b6ca-4181-8823-2b719883e545" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:10.234608 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:10.234578    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx"
Oct 30 21:42:10.259745 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:10.259714    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc"
Oct 30 21:42:10.261932 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:10.260058    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw"
Oct 30 21:42:10.267403 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:10.262895    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw"
Oct 30 21:42:10.267403 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:10.262948    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj"
Oct 30 21:42:10.293776 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:10.293424    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n"
Oct 30 21:42:10.329427 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:10.324937    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp"
Oct 30 21:42:10.329427 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:10.327598    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp"
Oct 30 21:42:18.349860 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:42:18.349780    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 21:42:37.272114 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:37.272060    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw" podUID="914f7544-6ef8-4496-b98d-b7ebc4123cda" containerName="main" probeResult="failure" output="Get \"https://10.132.0.86:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:37.435385 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:37.434514    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj" podUID="d0ff5023-fd3f-4734-94c6-76b666010adf" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:37.714384 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:37.712466    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw" podUID="2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded"
Oct 30 21:42:37.828985 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:37.828932    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg" podUID="2d0ea058-b6ca-4181-8823-2b719883e545" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": context deadline exceeded"
Oct 30 21:42:38.349386 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:38.312430    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw" podUID="914f7544-6ef8-4496-b98d-b7ebc4123cda" containerName="main" probeResult="failure" output="Get \"https://10.132.0.86:8000/ready\": context deadline exceeded"
Oct 30 21:42:38.349386 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:38.313684    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw"
Oct 30 21:42:38.449393 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:38.436999    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj" podUID="d0ff5023-fd3f-4734-94c6-76b666010adf" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:38.575388 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:38.533990    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc" podUID="23b97dbb-8488-4c44-a810-882ea18ed919" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": context deadline exceeded"
Oct 30 21:42:38.575388 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:38.536077    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj"
Oct 30 21:42:38.575388 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:38.565992    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx" podUID="34fdf57d-86f6-474f-a4f6-811e5063c405" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:38.733687 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:38.733644    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw" podUID="2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded"
Oct 30 21:42:38.759392 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:38.744185    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw"
Oct 30 21:42:39.042447 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:39.025737    5032 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc" podUID="23b97dbb-8488-4c44-a810-882ea18ed919" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/health\": context deadline exceeded"
Oct 30 21:42:39.177388 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:39.115909    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg" podUID="2d0ea058-b6ca-4181-8823-2b719883e545" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:39.351129 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:39.315506    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw" podUID="914f7544-6ef8-4496-b98d-b7ebc4123cda" containerName="main" probeResult="failure" output="Get \"https://10.132.0.86:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:39.377396 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:39.352344    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg"
Oct 30 21:42:39.839402 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:39.632155    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj" podUID="d0ff5023-fd3f-4734-94c6-76b666010adf" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:39.912444 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:39.738580    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw" podUID="2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded"
Oct 30 21:42:39.929400 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:39.739335    5032 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg" podUID="2d0ea058-b6ca-4181-8823-2b719883e545" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/health\": context deadline exceeded"
Oct 30 21:42:39.941452 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:39.759639    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc" podUID="23b97dbb-8488-4c44-a810-882ea18ed919" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": context deadline exceeded"
Oct 30 21:42:39.956808 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:39.956756    5032 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx" podUID="34fdf57d-86f6-474f-a4f6-811e5063c405" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/health\": context deadline exceeded"
Oct 30 21:42:39.968387 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:39.957641    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx" podUID="34fdf57d-86f6-474f-a4f6-811e5063c405" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": context deadline exceeded"
Oct 30 21:42:40.095125 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:40.095033    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc"
Oct 30 21:42:40.368640 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:40.368499    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw" podUID="914f7544-6ef8-4496-b98d-b7ebc4123cda" containerName="main" probeResult="failure" output="Get \"https://10.132.0.86:8000/ready\": context deadline exceeded"
Oct 30 21:42:40.395388 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:40.369269    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg" podUID="2d0ea058-b6ca-4181-8823-2b719883e545" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": context deadline exceeded"
Oct 30 21:42:40.852959 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:40.852915    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj" podUID="d0ff5023-fd3f-4734-94c6-76b666010adf" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:40.868700 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:40.868654    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx"
Oct 30 21:42:40.920010 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:40.919943    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-jx4qg" podUID="0c945ece-1ac0-4993-aea4-e853a23a4099" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": context deadline exceeded"
Oct 30 21:42:40.946696 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:40.922981    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw" podUID="2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded"
Oct 30 21:42:41.020807 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:41.020767    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc" podUID="23b97dbb-8488-4c44-a810-882ea18ed919" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": context deadline exceeded"
Oct 30 21:42:41.034515 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:41.034465    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-xv7qv" podUID="4f6d23b8-1364-4b03-96c6-6649f3a0644a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:41.228880 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:41.037591    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="opendatahub/model-serving-api-75bd89fdfc-wrl7s" podUID="9ef57dfb-240c-42e2-bc9d-c68c63f0a11c" containerName="server" probeResult="failure" output="Get \"https://10.132.0.45:8443/readyz\": context deadline exceeded"
Oct 30 21:42:41.269673 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:41.039652    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx" podUID="34fdf57d-86f6-474f-a4f6-811e5063c405" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:41.582574 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:41.582459    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg" podUID="2d0ea058-b6ca-4181-8823-2b719883e545" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:41.667435 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:41.599113    5032 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw" podUID="914f7544-6ef8-4496-b98d-b7ebc4123cda" containerName="main" probeResult="failure" output="Get \"https://10.132.0.86:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:41.667792 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:41.667751    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw" podUID="914f7544-6ef8-4496-b98d-b7ebc4123cda" containerName="main" probeResult="failure" output="Get \"https://10.132.0.86:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:41.893446 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:41.868641    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj" podUID="d0ff5023-fd3f-4734-94c6-76b666010adf" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:41.893446 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:41.873504    5032 patch_prober.go:29] interesting pod/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.100:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:42:41.893446 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:41.873572    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp" podUID="86937e06-85cc-48d5-9935-9ae922c3518c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.100:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:41.912438 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:41.906198    5032 patch_prober.go:29] interesting pod/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:42:41.912438 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:41.906278    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n" podUID="74856e06-9b6f-425d-90f8-e973d0649f90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:41.976387 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:41.961488    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw" podUID="2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded"
Oct 30 21:42:42.025536 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:42.025486    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc" podUID="23b97dbb-8488-4c44-a810-882ea18ed919" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:42.291937 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:42.291889    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx" podUID="34fdf57d-86f6-474f-a4f6-811e5063c405" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": context deadline exceeded"
Oct 30 21:42:42.601608 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:42.581461    5032 patch_prober.go:29] interesting pod/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:42:42.601608 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:42.581535    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc" podUID="3b92d898-765d-445d-9056-9f256030162e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:42.641309 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:42.610532    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg" podUID="2d0ea058-b6ca-4181-8823-2b719883e545" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:42.671252 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:42.671205    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw" podUID="914f7544-6ef8-4496-b98d-b7ebc4123cda" containerName="main" probeResult="failure" output="Get \"https://10.132.0.86:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:42.932448 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:42.907659    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj" podUID="d0ff5023-fd3f-4734-94c6-76b666010adf" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:43.056388 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:43.011493    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw" podUID="2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:43.129403 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:43.061433    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc" podUID="23b97dbb-8488-4c44-a810-882ea18ed919" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:43.473388 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:43.469063    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx" podUID="34fdf57d-86f6-474f-a4f6-811e5063c405" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": context deadline exceeded"
Oct 30 21:42:43.517918 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:43.517882    5032 patch_prober.go:29] interesting pod/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:42:43.518228 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:43.518184    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2" podUID="57dfd470-9c35-404f-b967-14a9103adeea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:43.579157 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:43.579109    5032 patch_prober.go:29] interesting pod/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:42:43.579481 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:43.579450    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s" podUID="222c96b4-9c61-49c0-a3ed-450ea7d17b67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:43.709433 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:43.664071    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg" podUID="2d0ea058-b6ca-4181-8823-2b719883e545" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:43.735571 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:43.675495    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw" podUID="914f7544-6ef8-4496-b98d-b7ebc4123cda" containerName="main" probeResult="failure" output="Get \"https://10.132.0.86:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:43.942387 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:43.938477    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj" podUID="d0ff5023-fd3f-4734-94c6-76b666010adf" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:44.042441 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:44.018891    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw" podUID="2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:44.427940 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:44.409899    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc" podUID="23b97dbb-8488-4c44-a810-882ea18ed919" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:44.436159 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:44.436127    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw"
Oct 30 21:42:44.481389 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:44.475924    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx" podUID="34fdf57d-86f6-474f-a4f6-811e5063c405" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": context deadline exceeded"
Oct 30 21:42:44.537862 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:44.537457    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj"
Oct 30 21:42:44.590038 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:44.590005    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc"
Oct 30 21:42:44.595612 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:44.593747    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw"
Oct 30 21:42:44.606605 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:44.606578    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg"
Oct 30 21:42:44.616408 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:44.615763    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx"
Oct 30 21:42:45.698798 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:45.698764    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e19227f8-6dc7579b49-vczvc"]
Oct 30 21:42:45.704882 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:45.704853    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e19227f8-6dc7579b49-vczvc"
Oct 30 21:42:45.726415 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:45.724717    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-e19227f8-serving-cert\""
Oct 30 21:42:45.732306 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:45.732224    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e19227f8-6dc7579b49-vczvc"]
Oct 30 21:42:45.853449 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:45.853415    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxhkt\" (UniqueName: \"kubernetes.io/projected/0e0a98e9-be96-4c34-95eb-b74c3f411721-kube-api-access-hxhkt\") pod \"maas-api-e2e-ait-e19227f8-6dc7579b49-vczvc\" (UID: \"0e0a98e9-be96-4c34-95eb-b74c3f411721\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e19227f8-6dc7579b49-vczvc"
Oct 30 21:42:45.853635 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:45.853556    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e0a98e9-be96-4c34-95eb-b74c3f411721-maas-api-tls\") pod \"maas-api-e2e-ait-e19227f8-6dc7579b49-vczvc\" (UID: \"0e0a98e9-be96-4c34-95eb-b74c3f411721\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e19227f8-6dc7579b49-vczvc"
Oct 30 21:42:45.853635 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:45.853596    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e0a98e9-be96-4c34-95eb-b74c3f411721-metrics-certs\") pod \"maas-api-e2e-ait-e19227f8-6dc7579b49-vczvc\" (UID: \"0e0a98e9-be96-4c34-95eb-b74c3f411721\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e19227f8-6dc7579b49-vczvc"
Oct 30 21:42:45.853776 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:45.853745    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e0a98e9-be96-4c34-95eb-b74c3f411721-openshift-service-ca\") pod \"maas-api-e2e-ait-e19227f8-6dc7579b49-vczvc\" (UID: \"0e0a98e9-be96-4c34-95eb-b74c3f411721\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e19227f8-6dc7579b49-vczvc"
Oct 30 21:42:45.954883 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:45.954851    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxhkt\" (UniqueName: \"kubernetes.io/projected/0e0a98e9-be96-4c34-95eb-b74c3f411721-kube-api-access-hxhkt\") pod \"maas-api-e2e-ait-e19227f8-6dc7579b49-vczvc\" (UID: \"0e0a98e9-be96-4c34-95eb-b74c3f411721\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e19227f8-6dc7579b49-vczvc"
Oct 30 21:42:45.955072 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:45.954938    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e0a98e9-be96-4c34-95eb-b74c3f411721-maas-api-tls\") pod \"maas-api-e2e-ait-e19227f8-6dc7579b49-vczvc\" (UID: \"0e0a98e9-be96-4c34-95eb-b74c3f411721\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e19227f8-6dc7579b49-vczvc"
Oct 30 21:42:45.955072 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:45.955013    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e0a98e9-be96-4c34-95eb-b74c3f411721-metrics-certs\") pod \"maas-api-e2e-ait-e19227f8-6dc7579b49-vczvc\" (UID: \"0e0a98e9-be96-4c34-95eb-b74c3f411721\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e19227f8-6dc7579b49-vczvc"
Oct 30 21:42:45.955195 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:45.955123    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e0a98e9-be96-4c34-95eb-b74c3f411721-openshift-service-ca\") pod \"maas-api-e2e-ait-e19227f8-6dc7579b49-vczvc\" (UID: \"0e0a98e9-be96-4c34-95eb-b74c3f411721\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e19227f8-6dc7579b49-vczvc"
Oct 30 21:42:45.955844 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:45.955811    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e0a98e9-be96-4c34-95eb-b74c3f411721-openshift-service-ca\") pod \"maas-api-e2e-ait-e19227f8-6dc7579b49-vczvc\" (UID: \"0e0a98e9-be96-4c34-95eb-b74c3f411721\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e19227f8-6dc7579b49-vczvc"
Oct 30 21:42:45.957508 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:45.957485    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e0a98e9-be96-4c34-95eb-b74c3f411721-metrics-certs\") pod \"maas-api-e2e-ait-e19227f8-6dc7579b49-vczvc\" (UID: \"0e0a98e9-be96-4c34-95eb-b74c3f411721\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e19227f8-6dc7579b49-vczvc"
Oct 30 21:42:45.957714 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:45.957696    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e0a98e9-be96-4c34-95eb-b74c3f411721-maas-api-tls\") pod \"maas-api-e2e-ait-e19227f8-6dc7579b49-vczvc\" (UID: \"0e0a98e9-be96-4c34-95eb-b74c3f411721\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e19227f8-6dc7579b49-vczvc"
Oct 30 21:42:45.967851 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:45.967818    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxhkt\" (UniqueName: \"kubernetes.io/projected/0e0a98e9-be96-4c34-95eb-b74c3f411721-kube-api-access-hxhkt\") pod \"maas-api-e2e-ait-e19227f8-6dc7579b49-vczvc\" (UID: \"0e0a98e9-be96-4c34-95eb-b74c3f411721\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e19227f8-6dc7579b49-vczvc"
Oct 30 21:42:46.041752 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:46.041658    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e19227f8-6dc7579b49-vczvc"
Oct 30 21:42:46.478780 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:46.478715    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e19227f8-6dc7579b49-vczvc"]
Oct 30 21:42:46.657727 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:46.657689    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e19227f8-6dc7579b49-vczvc" event={"ID":"0e0a98e9-be96-4c34-95eb-b74c3f411721","Type":"ContainerStarted","Data":"0842861241cab700af688841634ceda2b75cdb230d46f95ab79047336b243bc9"}
Oct 30 21:42:47.673495 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:47.671142    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e19227f8-6dc7579b49-vczvc" event={"ID":"0e0a98e9-be96-4c34-95eb-b74c3f411721","Type":"ContainerStarted","Data":"b0225b8e0aa10ac01c042249c65173d031a4f3a666cf7e00d3febea51220b802"}
Oct 30 21:42:47.673495 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:47.671230    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e19227f8-6dc7579b49-vczvc"
Oct 30 21:42:47.696462 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:47.696385    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e19227f8-6dc7579b49-vczvc" podStartSLOduration=1.992919224 podStartE2EDuration="2.696351708s" totalImagesPullingTime="703.432484ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:42:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:42:47.692813612 +0000 UTC m=+880.494626076" watchObservedRunningTime="2026-10-30 21:42:47.696351708 +0000 UTC m=+880.498164195"
Oct 30 21:42:48.359730 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:42:48.359682    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 21:42:53.689700 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:42:53.689669    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e19227f8-6dc7579b49-vczvc"
Oct 30 21:43:00.396780 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:00.395681    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 21:43:03.420552 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:03.419094    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg" podUID="2d0ea058-b6ca-4181-8823-2b719883e545" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": context deadline exceeded"
Oct 30 21:43:03.421060 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:03.420936    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx" podUID="34fdf57d-86f6-474f-a4f6-811e5063c405" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:43:03.753413 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:03.746989    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw" podUID="914f7544-6ef8-4496-b98d-b7ebc4123cda" containerName="main" probeResult="failure" output="Get \"https://10.132.0.86:8000/ready\": context deadline exceeded"
Oct 30 21:43:03.932383 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:03.921199    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw" podUID="2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:43:04.042495 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:04.029696    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj" podUID="d0ff5023-fd3f-4734-94c6-76b666010adf" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": context deadline exceeded"
Oct 30 21:43:04.042495 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:04.030689    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc" podUID="23b97dbb-8488-4c44-a810-882ea18ed919" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:43:04.082187 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:04.081487    5032 patch_prober.go:29] interesting pod/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:43:04.082187 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:04.081557    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf" podUID="2b720f88-2fd8-4c4d-a300-941b1a9db6e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:43:04.430524 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:04.422473    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx" podUID="34fdf57d-86f6-474f-a4f6-811e5063c405" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": context deadline exceeded"
Oct 30 21:43:04.754385 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:04.448264    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg" podUID="2d0ea058-b6ca-4181-8823-2b719883e545" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:43:04.754385 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:04.730270    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx"
Oct 30 21:43:04.754385 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:04.730334    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg"
Oct 30 21:43:04.754385 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:04.621516    5032 patch_prober.go:29] interesting pod/e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:43:04.754385 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:04.730733    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h" podUID="fb5d58b6-89f9-4735-ba24-e4165dcb4c81" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:43:04.790423 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:04.781321    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw" podUID="914f7544-6ef8-4496-b98d-b7ebc4123cda" containerName="main" probeResult="failure" output="Get \"https://10.132.0.86:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:43:04.790423 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:04.781604    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw"
Oct 30 21:43:04.967405 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:04.964081    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw" podUID="2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded"
Oct 30 21:43:04.990199 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:04.990162    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw"
Oct 30 21:43:05.067444 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:05.056260    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj" podUID="d0ff5023-fd3f-4734-94c6-76b666010adf" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:43:05.071133 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:05.071104    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj"
Oct 30 21:43:05.098913 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:05.071094    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc" podUID="23b97dbb-8488-4c44-a810-882ea18ed919" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:43:05.098913 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:05.098075    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc"
Oct 30 21:43:05.153297 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:05.153243    5032 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj" podUID="d0ff5023-fd3f-4734-94c6-76b666010adf" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:43:05.745051 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:05.744994    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg" podUID="2d0ea058-b6ca-4181-8823-2b719883e545" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:43:05.746036 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:05.745992    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx" podUID="34fdf57d-86f6-474f-a4f6-811e5063c405" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": context deadline exceeded"
Oct 30 21:43:05.811398 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:05.809511    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw" podUID="914f7544-6ef8-4496-b98d-b7ebc4123cda" containerName="main" probeResult="failure" output="Get \"https://10.132.0.86:8000/ready\": context deadline exceeded"
Oct 30 21:43:05.968967 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:05.968927    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw" podUID="2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded"
Oct 30 21:43:06.080962 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:06.080839    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj" podUID="d0ff5023-fd3f-4734-94c6-76b666010adf" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": context deadline exceeded"
Oct 30 21:43:06.110578 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:06.110519    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc" podUID="23b97dbb-8488-4c44-a810-882ea18ed919" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:43:06.613724 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:06.611462    5032 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw" podUID="2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:43:06.755276 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:06.746524    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx" podUID="34fdf57d-86f6-474f-a4f6-811e5063c405" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:43:06.789387 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:06.771692    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg" podUID="2d0ea058-b6ca-4181-8823-2b719883e545" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:43:06.828538 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:06.828475    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw" podUID="914f7544-6ef8-4496-b98d-b7ebc4123cda" containerName="main" probeResult="failure" output="Get \"https://10.132.0.86:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:43:06.973709 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:06.973662    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw" podUID="2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:43:07.049421 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:07.046878    5032 patch_prober.go:29] interesting pod/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:43:07.049421 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:07.047040    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8" podUID="e7679391-8575-43d7-b7f1-5b1ab0bcee5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:43:07.088929 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:07.088056    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj" podUID="d0ff5023-fd3f-4734-94c6-76b666010adf" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:43:07.129468 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:07.126862    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc" podUID="23b97dbb-8488-4c44-a810-882ea18ed919" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": context deadline exceeded"
Oct 30 21:43:07.850886 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:07.850834    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw" podUID="914f7544-6ef8-4496-b98d-b7ebc4123cda" containerName="main" probeResult="failure" output="Get \"https://10.132.0.86:8000/ready\": context deadline exceeded"
Oct 30 21:43:07.949437 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:07.933817    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx" podUID="34fdf57d-86f6-474f-a4f6-811e5063c405" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": context deadline exceeded"
Oct 30 21:43:08.032408 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:08.001971    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw" podUID="2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:43:08.032408 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:08.014285    5032 patch_prober.go:29] interesting pod/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:43:08.032408 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:08.014354    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz" podUID="ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:43:08.032408 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:08.015079    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg" podUID="2d0ea058-b6ca-4181-8823-2b719883e545" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": context deadline exceeded"
Oct 30 21:43:08.157446 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:08.135149    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj" podUID="d0ff5023-fd3f-4734-94c6-76b666010adf" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": context deadline exceeded"
Oct 30 21:43:08.176385 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:08.162531    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc" podUID="23b97dbb-8488-4c44-a810-882ea18ed919" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:43:09.219195 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:09.219165    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-sklxd_67292bdc-7fe1-449d-af75-2baa90c16ea9/console-operator/1.log"
Oct 30 21:43:09.228409 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:09.224448    5032 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc" podUID="23b97dbb-8488-4c44-a810-882ea18ed919" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:43:09.228409 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:09.224677    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw" podUID="914f7544-6ef8-4496-b98d-b7ebc4123cda" containerName="main" probeResult="failure" output="Get \"https://10.132.0.86:8000/ready\": context deadline exceeded"
Oct 30 21:43:09.246514 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:09.246472    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx" podUID="34fdf57d-86f6-474f-a4f6-811e5063c405" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": context deadline exceeded"
Oct 30 21:43:09.288720 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:09.288677    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc" podUID="23b97dbb-8488-4c44-a810-882ea18ed919" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:43:09.314394 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:09.288964    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg" podUID="2d0ea058-b6ca-4181-8823-2b719883e545" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:43:09.315928 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:09.289500    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw" podUID="2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded"
Oct 30 21:43:09.316589 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:09.289536    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj" podUID="d0ff5023-fd3f-4734-94c6-76b666010adf" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:43:09.531437 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:09.526474    5032 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx" podUID="34fdf57d-86f6-474f-a4f6-811e5063c405" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/health\": context deadline exceeded"
Oct 30 21:43:09.620962 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:09.620912    5032 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg" podUID="2d0ea058-b6ca-4181-8823-2b719883e545" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:43:09.870391 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:09.866685    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-sklxd_67292bdc-7fe1-449d-af75-2baa90c16ea9/console-operator/1.log"
Oct 30 21:43:09.927076 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:09.927047    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw"
Oct 30 21:43:10.280384 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:10.265699    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx" podUID="34fdf57d-86f6-474f-a4f6-811e5063c405" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": context deadline exceeded"
Oct 30 21:43:10.353464 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:10.353403    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc" podUID="23b97dbb-8488-4c44-a810-882ea18ed919" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:43:10.666836 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:10.360976    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg" podUID="2d0ea058-b6ca-4181-8823-2b719883e545" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:43:10.666836 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:10.362794    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj" podUID="d0ff5023-fd3f-4734-94c6-76b666010adf" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:43:10.667089 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:10.362997    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw" podUID="2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded"
Oct 30 21:43:10.748860 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:10.748830    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg"
Oct 30 21:43:11.039523 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:11.039489    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw"
Oct 30 21:43:11.073206 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:11.073169    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc"
Oct 30 21:43:11.081757 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:11.081729    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj"
Oct 30 21:43:11.127022 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:11.126994    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx"
Oct 30 21:43:18.349300 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:43:18.349262    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 21:43:24.326722 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:24.326685    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a5af9561-9884bf4bb-w7gn4"]
Oct 30 21:43:24.332772 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:24.332747    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a5af9561-9884bf4bb-w7gn4"
Oct 30 21:43:24.342132 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:24.342104    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-a5af9561-serving-cert\""
Oct 30 21:43:24.343738 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:24.343702    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a5af9561-9884bf4bb-w7gn4"]
Oct 30 21:43:24.482559 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:24.482521    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dkjm\" (UniqueName: \"kubernetes.io/projected/2cc56813-ea74-445b-a09c-dae498e1eaf2-kube-api-access-4dkjm\") pod \"maas-api-e2e-derive-a5af9561-9884bf4bb-w7gn4\" (UID: \"2cc56813-ea74-445b-a09c-dae498e1eaf2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a5af9561-9884bf4bb-w7gn4"
Oct 30 21:43:24.482745 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:24.482591    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2cc56813-ea74-445b-a09c-dae498e1eaf2-metrics-certs\") pod \"maas-api-e2e-derive-a5af9561-9884bf4bb-w7gn4\" (UID: \"2cc56813-ea74-445b-a09c-dae498e1eaf2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a5af9561-9884bf4bb-w7gn4"
Oct 30 21:43:24.482745 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:24.482710    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2cc56813-ea74-445b-a09c-dae498e1eaf2-openshift-service-ca\") pod \"maas-api-e2e-derive-a5af9561-9884bf4bb-w7gn4\" (UID: \"2cc56813-ea74-445b-a09c-dae498e1eaf2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a5af9561-9884bf4bb-w7gn4"
Oct 30 21:43:24.482850 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:24.482775    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2cc56813-ea74-445b-a09c-dae498e1eaf2-maas-api-tls\") pod \"maas-api-e2e-derive-a5af9561-9884bf4bb-w7gn4\" (UID: \"2cc56813-ea74-445b-a09c-dae498e1eaf2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a5af9561-9884bf4bb-w7gn4"
Oct 30 21:43:24.584089 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:24.584000    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2cc56813-ea74-445b-a09c-dae498e1eaf2-openshift-service-ca\") pod \"maas-api-e2e-derive-a5af9561-9884bf4bb-w7gn4\" (UID: \"2cc56813-ea74-445b-a09c-dae498e1eaf2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a5af9561-9884bf4bb-w7gn4"
Oct 30 21:43:24.584089 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:24.584085    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2cc56813-ea74-445b-a09c-dae498e1eaf2-maas-api-tls\") pod \"maas-api-e2e-derive-a5af9561-9884bf4bb-w7gn4\" (UID: \"2cc56813-ea74-445b-a09c-dae498e1eaf2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a5af9561-9884bf4bb-w7gn4"
Oct 30 21:43:24.584444 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:24.584375    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dkjm\" (UniqueName: \"kubernetes.io/projected/2cc56813-ea74-445b-a09c-dae498e1eaf2-kube-api-access-4dkjm\") pod \"maas-api-e2e-derive-a5af9561-9884bf4bb-w7gn4\" (UID: \"2cc56813-ea74-445b-a09c-dae498e1eaf2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a5af9561-9884bf4bb-w7gn4"
Oct 30 21:43:24.584655 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:24.584624    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2cc56813-ea74-445b-a09c-dae498e1eaf2-metrics-certs\") pod \"maas-api-e2e-derive-a5af9561-9884bf4bb-w7gn4\" (UID: \"2cc56813-ea74-445b-a09c-dae498e1eaf2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a5af9561-9884bf4bb-w7gn4"
Oct 30 21:43:24.584957 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:24.584923    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2cc56813-ea74-445b-a09c-dae498e1eaf2-openshift-service-ca\") pod \"maas-api-e2e-derive-a5af9561-9884bf4bb-w7gn4\" (UID: \"2cc56813-ea74-445b-a09c-dae498e1eaf2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a5af9561-9884bf4bb-w7gn4"
Oct 30 21:43:24.587820 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:24.587781    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2cc56813-ea74-445b-a09c-dae498e1eaf2-maas-api-tls\") pod \"maas-api-e2e-derive-a5af9561-9884bf4bb-w7gn4\" (UID: \"2cc56813-ea74-445b-a09c-dae498e1eaf2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a5af9561-9884bf4bb-w7gn4"
Oct 30 21:43:24.590745 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:24.590720    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2cc56813-ea74-445b-a09c-dae498e1eaf2-metrics-certs\") pod \"maas-api-e2e-derive-a5af9561-9884bf4bb-w7gn4\" (UID: \"2cc56813-ea74-445b-a09c-dae498e1eaf2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a5af9561-9884bf4bb-w7gn4"
Oct 30 21:43:24.599313 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:24.599279    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dkjm\" (UniqueName: \"kubernetes.io/projected/2cc56813-ea74-445b-a09c-dae498e1eaf2-kube-api-access-4dkjm\") pod \"maas-api-e2e-derive-a5af9561-9884bf4bb-w7gn4\" (UID: \"2cc56813-ea74-445b-a09c-dae498e1eaf2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a5af9561-9884bf4bb-w7gn4"
Oct 30 21:43:24.656079 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:24.656035    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a5af9561-9884bf4bb-w7gn4"
Oct 30 21:43:24.962493 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:24.962464    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a5af9561-9884bf4bb-w7gn4"]
Oct 30 21:43:24.963610 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:43:24.963582    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cc56813_ea74_445b_a09c_dae498e1eaf2.slice/crio-ace4593c63be3275a39605b5a740f2c8a962406b0005c151902cfa661acdf124 WatchSource:0}: Error finding container ace4593c63be3275a39605b5a740f2c8a962406b0005c151902cfa661acdf124: Status 404 returned error can't find the container with id ace4593c63be3275a39605b5a740f2c8a962406b0005c151902cfa661acdf124
Oct 30 21:43:25.115383 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:25.114033    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a5af9561-9884bf4bb-w7gn4" event={"ID":"2cc56813-ea74-445b-a09c-dae498e1eaf2","Type":"ContainerStarted","Data":"ace4593c63be3275a39605b5a740f2c8a962406b0005c151902cfa661acdf124"}
Oct 30 21:43:26.124988 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:26.124955    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a5af9561-9884bf4bb-w7gn4" event={"ID":"2cc56813-ea74-445b-a09c-dae498e1eaf2","Type":"ContainerStarted","Data":"fed2c1d2d874251533e089f15e4d696b027b63d9a32c35b12a20835280fd9062"}
Oct 30 21:43:26.125392 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:26.125152    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a5af9561-9884bf4bb-w7gn4"
Oct 30 21:43:26.155071 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:26.155014    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a5af9561-9884bf4bb-w7gn4" podStartSLOduration=1.747881915 podStartE2EDuration="2.154997579s" totalImagesPullingTime="407.115664ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:43:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:43:26.149103421 +0000 UTC m=+918.950915897" watchObservedRunningTime="2026-10-30 21:43:26.154997579 +0000 UTC m=+918.956810054"
Oct 30 21:43:29.599143 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:29.599105    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6d718596-64457bc59f-vf7p9"]
Oct 30 21:43:29.604528 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:29.604501    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6d718596-64457bc59f-vf7p9"
Oct 30 21:43:29.618632 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:29.618602    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-6d718596-serving-cert\""
Oct 30 21:43:29.631968 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:29.631942    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6d718596-64457bc59f-vf7p9"]
Oct 30 21:43:29.774729 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:29.774693    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4dd760d4-d256-430f-83c0-226c0688f585-metrics-certs\") pod \"maas-api-e2e-ait-6d718596-64457bc59f-vf7p9\" (UID: \"4dd760d4-d256-430f-83c0-226c0688f585\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6d718596-64457bc59f-vf7p9"
Oct 30 21:43:29.775526 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:29.775498    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4dd760d4-d256-430f-83c0-226c0688f585-maas-api-tls\") pod \"maas-api-e2e-ait-6d718596-64457bc59f-vf7p9\" (UID: \"4dd760d4-d256-430f-83c0-226c0688f585\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6d718596-64457bc59f-vf7p9"
Oct 30 21:43:29.775734 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:29.775710    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fthxw\" (UniqueName: \"kubernetes.io/projected/4dd760d4-d256-430f-83c0-226c0688f585-kube-api-access-fthxw\") pod \"maas-api-e2e-ait-6d718596-64457bc59f-vf7p9\" (UID: \"4dd760d4-d256-430f-83c0-226c0688f585\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6d718596-64457bc59f-vf7p9"
Oct 30 21:43:29.775868 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:29.775800    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4dd760d4-d256-430f-83c0-226c0688f585-openshift-service-ca\") pod \"maas-api-e2e-ait-6d718596-64457bc59f-vf7p9\" (UID: \"4dd760d4-d256-430f-83c0-226c0688f585\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6d718596-64457bc59f-vf7p9"
Oct 30 21:43:29.877289 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:29.877197    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4dd760d4-d256-430f-83c0-226c0688f585-metrics-certs\") pod \"maas-api-e2e-ait-6d718596-64457bc59f-vf7p9\" (UID: \"4dd760d4-d256-430f-83c0-226c0688f585\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6d718596-64457bc59f-vf7p9"
Oct 30 21:43:29.877469 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:29.877321    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4dd760d4-d256-430f-83c0-226c0688f585-maas-api-tls\") pod \"maas-api-e2e-ait-6d718596-64457bc59f-vf7p9\" (UID: \"4dd760d4-d256-430f-83c0-226c0688f585\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6d718596-64457bc59f-vf7p9"
Oct 30 21:43:29.877469 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:29.877345    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fthxw\" (UniqueName: \"kubernetes.io/projected/4dd760d4-d256-430f-83c0-226c0688f585-kube-api-access-fthxw\") pod \"maas-api-e2e-ait-6d718596-64457bc59f-vf7p9\" (UID: \"4dd760d4-d256-430f-83c0-226c0688f585\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6d718596-64457bc59f-vf7p9"
Oct 30 21:43:29.877560 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:29.877484    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4dd760d4-d256-430f-83c0-226c0688f585-openshift-service-ca\") pod \"maas-api-e2e-ait-6d718596-64457bc59f-vf7p9\" (UID: \"4dd760d4-d256-430f-83c0-226c0688f585\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6d718596-64457bc59f-vf7p9"
Oct 30 21:43:29.878335 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:29.878316    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4dd760d4-d256-430f-83c0-226c0688f585-openshift-service-ca\") pod \"maas-api-e2e-ait-6d718596-64457bc59f-vf7p9\" (UID: \"4dd760d4-d256-430f-83c0-226c0688f585\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6d718596-64457bc59f-vf7p9"
Oct 30 21:43:29.878733 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:43:29.878715    5032 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-6d718596-serving-cert: secret "maas-api-e2e-ait-6d718596-serving-cert" not found
Oct 30 21:43:29.878849 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:43:29.878807    5032 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4dd760d4-d256-430f-83c0-226c0688f585-maas-api-tls podName:4dd760d4-d256-430f-83c0-226c0688f585 nodeName:}" failed. No retries permitted until 2026-10-30 21:43:30.378786744 +0000 UTC m=+923.180599186 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4dd760d4-d256-430f-83c0-226c0688f585-maas-api-tls") pod "maas-api-e2e-ait-6d718596-64457bc59f-vf7p9" (UID: "4dd760d4-d256-430f-83c0-226c0688f585") : secret "maas-api-e2e-ait-6d718596-serving-cert" not found
Oct 30 21:43:29.880870 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:29.880840    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4dd760d4-d256-430f-83c0-226c0688f585-metrics-certs\") pod \"maas-api-e2e-ait-6d718596-64457bc59f-vf7p9\" (UID: \"4dd760d4-d256-430f-83c0-226c0688f585\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6d718596-64457bc59f-vf7p9"
Oct 30 21:43:29.894069 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:29.894031    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fthxw\" (UniqueName: \"kubernetes.io/projected/4dd760d4-d256-430f-83c0-226c0688f585-kube-api-access-fthxw\") pod \"maas-api-e2e-ait-6d718596-64457bc59f-vf7p9\" (UID: \"4dd760d4-d256-430f-83c0-226c0688f585\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6d718596-64457bc59f-vf7p9"
Oct 30 21:43:30.294160 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:30.294123    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e19227f8-6dc7579b49-vczvc"]
Oct 30 21:43:30.294431 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:30.294405    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e19227f8-6dc7579b49-vczvc" podUID="0e0a98e9-be96-4c34-95eb-b74c3f411721" containerName="maas-api" containerID="cri-o://b0225b8e0aa10ac01c042249c65173d031a4f3a666cf7e00d3febea51220b802" gracePeriod=30
Oct 30 21:43:30.385762 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:30.385733    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4dd760d4-d256-430f-83c0-226c0688f585-maas-api-tls\") pod \"maas-api-e2e-ait-6d718596-64457bc59f-vf7p9\" (UID: \"4dd760d4-d256-430f-83c0-226c0688f585\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6d718596-64457bc59f-vf7p9"
Oct 30 21:43:30.388225 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:30.388197    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4dd760d4-d256-430f-83c0-226c0688f585-maas-api-tls\") pod \"maas-api-e2e-ait-6d718596-64457bc59f-vf7p9\" (UID: \"4dd760d4-d256-430f-83c0-226c0688f585\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6d718596-64457bc59f-vf7p9"
Oct 30 21:43:30.719030 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:30.718993    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6d718596-64457bc59f-vf7p9"
Oct 30 21:43:30.847401 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:30.845019    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e19227f8-6dc7579b49-vczvc"
Oct 30 21:43:30.924382 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:30.924113    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0e0a98e9-be96-4c34-95eb-b74c3f411721-kube-api-access-hxhkt\" (UniqueName: \"kubernetes.io/projected/0e0a98e9-be96-4c34-95eb-b74c3f411721-kube-api-access-hxhkt\") pod \"0e0a98e9-be96-4c34-95eb-b74c3f411721\" (UID: \"0e0a98e9-be96-4c34-95eb-b74c3f411721\") "
Oct 30 21:43:30.924382 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:30.924254    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0e0a98e9-be96-4c34-95eb-b74c3f411721-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e0a98e9-be96-4c34-95eb-b74c3f411721-maas-api-tls\") pod \"0e0a98e9-be96-4c34-95eb-b74c3f411721\" (UID: \"0e0a98e9-be96-4c34-95eb-b74c3f411721\") "
Oct 30 21:43:30.924573 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:30.924470    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0e0a98e9-be96-4c34-95eb-b74c3f411721-metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e0a98e9-be96-4c34-95eb-b74c3f411721-metrics-certs\") pod \"0e0a98e9-be96-4c34-95eb-b74c3f411721\" (UID: \"0e0a98e9-be96-4c34-95eb-b74c3f411721\") "
Oct 30 21:43:30.924573 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:30.924503    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0e0a98e9-be96-4c34-95eb-b74c3f411721-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e0a98e9-be96-4c34-95eb-b74c3f411721-openshift-service-ca\") pod \"0e0a98e9-be96-4c34-95eb-b74c3f411721\" (UID: \"0e0a98e9-be96-4c34-95eb-b74c3f411721\") "
Oct 30 21:43:30.927840 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:30.925124    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e0a98e9-be96-4c34-95eb-b74c3f411721-openshift-service-ca" pod "0e0a98e9-be96-4c34-95eb-b74c3f411721" (UID: "0e0a98e9-be96-4c34-95eb-b74c3f411721"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:43:30.927840 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:30.925535    5032 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e0a98e9-be96-4c34-95eb-b74c3f411721-openshift-service-ca\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:43:30.930566 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:30.928743    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e0a98e9-be96-4c34-95eb-b74c3f411721-maas-api-tls" pod "0e0a98e9-be96-4c34-95eb-b74c3f411721" (UID: "0e0a98e9-be96-4c34-95eb-b74c3f411721"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:43:30.933400 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:30.933073    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e0a98e9-be96-4c34-95eb-b74c3f411721-metrics-certs" pod "0e0a98e9-be96-4c34-95eb-b74c3f411721" (UID: "0e0a98e9-be96-4c34-95eb-b74c3f411721"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:43:30.933825 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:30.933575    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e0a98e9-be96-4c34-95eb-b74c3f411721-kube-api-access-hxhkt" pod "0e0a98e9-be96-4c34-95eb-b74c3f411721" (UID: "0e0a98e9-be96-4c34-95eb-b74c3f411721"). InnerVolumeSpecName "kube-api-access-hxhkt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:43:31.027509 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:31.027429    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hxhkt\" (UniqueName: \"kubernetes.io/projected/0e0a98e9-be96-4c34-95eb-b74c3f411721-kube-api-access-hxhkt\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:43:31.027509 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:31.027459    5032 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e0a98e9-be96-4c34-95eb-b74c3f411721-maas-api-tls\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:43:31.027509 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:31.027469    5032 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e0a98e9-be96-4c34-95eb-b74c3f411721-metrics-certs\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:43:31.104681 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:31.103805    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6d718596-64457bc59f-vf7p9"]
Oct 30 21:43:31.108419 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:43:31.106908    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4dd760d4_d256_430f_83c0_226c0688f585.slice/crio-935338fb2e9fe9db1663a57cb51dcc85bc649d9342806102eaf7e714fda8a04d WatchSource:0}: Error finding container 935338fb2e9fe9db1663a57cb51dcc85bc649d9342806102eaf7e714fda8a04d: Status 404 returned error can't find the container with id 935338fb2e9fe9db1663a57cb51dcc85bc649d9342806102eaf7e714fda8a04d
Oct 30 21:43:31.195923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:31.195883    5032 generic.go:356] "Generic (PLEG): container finished" podID="0e0a98e9-be96-4c34-95eb-b74c3f411721" containerID="b0225b8e0aa10ac01c042249c65173d031a4f3a666cf7e00d3febea51220b802" exitCode=0
Oct 30 21:43:31.196129 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:31.195967    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e19227f8-6dc7579b49-vczvc" event={"ID":"0e0a98e9-be96-4c34-95eb-b74c3f411721","Type":"ContainerDied","Data":"b0225b8e0aa10ac01c042249c65173d031a4f3a666cf7e00d3febea51220b802"}
Oct 30 21:43:31.196129 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:31.196016    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e19227f8-6dc7579b49-vczvc" event={"ID":"0e0a98e9-be96-4c34-95eb-b74c3f411721","Type":"ContainerDied","Data":"0842861241cab700af688841634ceda2b75cdb230d46f95ab79047336b243bc9"}
Oct 30 21:43:31.196129 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:31.196046    5032 scope.go:122] "RemoveContainer" containerID="b0225b8e0aa10ac01c042249c65173d031a4f3a666cf7e00d3febea51220b802"
Oct 30 21:43:31.196296 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:31.196212    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e19227f8-6dc7579b49-vczvc"
Oct 30 21:43:31.224863 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:31.219997    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6d718596-64457bc59f-vf7p9" event={"ID":"4dd760d4-d256-430f-83c0-226c0688f585","Type":"ContainerStarted","Data":"935338fb2e9fe9db1663a57cb51dcc85bc649d9342806102eaf7e714fda8a04d"}
Oct 30 21:43:31.232409 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:31.232352    5032 scope.go:122] "RemoveContainer" containerID="b0225b8e0aa10ac01c042249c65173d031a4f3a666cf7e00d3febea51220b802"
Oct 30 21:43:31.235743 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:43:31.235711    5032 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0225b8e0aa10ac01c042249c65173d031a4f3a666cf7e00d3febea51220b802\": container with ID starting with b0225b8e0aa10ac01c042249c65173d031a4f3a666cf7e00d3febea51220b802 not found: ID does not exist" containerID="b0225b8e0aa10ac01c042249c65173d031a4f3a666cf7e00d3febea51220b802"
Oct 30 21:43:31.235867 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:31.235751    5032 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0225b8e0aa10ac01c042249c65173d031a4f3a666cf7e00d3febea51220b802"} err="failed to get container status \"b0225b8e0aa10ac01c042249c65173d031a4f3a666cf7e00d3febea51220b802\": rpc error: code = NotFound desc = could not find container \"b0225b8e0aa10ac01c042249c65173d031a4f3a666cf7e00d3febea51220b802\": container with ID starting with b0225b8e0aa10ac01c042249c65173d031a4f3a666cf7e00d3febea51220b802 not found: ID does not exist"
Oct 30 21:43:31.252390 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:31.252272    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e19227f8-6dc7579b49-vczvc"]
Oct 30 21:43:31.291450 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:31.283634    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e19227f8-6dc7579b49-vczvc"]
Oct 30 21:43:31.351416 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:31.351295    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0e0a98e9-be96-4c34-95eb-b74c3f411721" path="/var/lib/kubelet/pods/0e0a98e9-be96-4c34-95eb-b74c3f411721/volumes"
Oct 30 21:43:32.138588 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:32.138555    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a5af9561-9884bf4bb-w7gn4"
Oct 30 21:43:32.236379 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:32.235963    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6d718596-64457bc59f-vf7p9" event={"ID":"4dd760d4-d256-430f-83c0-226c0688f585","Type":"ContainerStarted","Data":"5595444ea397d8322d36b79b23d5159ab9cffaabe3938b83ca341cc961eb634f"}
Oct 30 21:43:32.236379 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:32.236033    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6d718596-64457bc59f-vf7p9"
Oct 30 21:43:32.261450 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:32.261395    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6d718596-64457bc59f-vf7p9" podStartSLOduration=2.792189246 podStartE2EDuration="3.261378442s" totalImagesPullingTime="469.189196ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:43:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:43:32.260169796 +0000 UTC m=+925.061982260" watchObservedRunningTime="2026-10-30 21:43:32.261378442 +0000 UTC m=+925.063190908"
Oct 30 21:43:38.253154 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:38.253120    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6d718596-64457bc59f-vf7p9"
Oct 30 21:43:44.117711 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:44.117671    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-484f7514-6876b65d9d-7vsp2"]
Oct 30 21:43:44.118834 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:44.118817    5032 cpu_manager.go:414] "containerMap: removing container" podUID="0e0a98e9-be96-4c34-95eb-b74c3f411721" containerName="maas-api"
Oct 30 21:43:44.118891 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:44.118837    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0e0a98e9-be96-4c34-95eb-b74c3f411721" containerName="maas-api"
Oct 30 21:43:44.119089 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:44.119077    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="0e0a98e9-be96-4c34-95eb-b74c3f411721" containerName="maas-api"
Oct 30 21:43:44.123492 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:44.123470    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-484f7514-6876b65d9d-7vsp2"
Oct 30 21:43:44.127822 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:44.127795    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-484f7514-serving-cert\""
Oct 30 21:43:44.135152 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:44.135119    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-484f7514-6876b65d9d-7vsp2"]
Oct 30 21:43:44.207224 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:44.207178    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3786e884-a0b0-4487-ba9f-972b7ce795c2-metrics-certs\") pod \"maas-api-e2e-ait-484f7514-6876b65d9d-7vsp2\" (UID: \"3786e884-a0b0-4487-ba9f-972b7ce795c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-484f7514-6876b65d9d-7vsp2"
Oct 30 21:43:44.207224 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:44.207220    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3786e884-a0b0-4487-ba9f-972b7ce795c2-openshift-service-ca\") pod \"maas-api-e2e-ait-484f7514-6876b65d9d-7vsp2\" (UID: \"3786e884-a0b0-4487-ba9f-972b7ce795c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-484f7514-6876b65d9d-7vsp2"
Oct 30 21:43:44.207586 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:44.207571    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86z9c\" (UniqueName: \"kubernetes.io/projected/3786e884-a0b0-4487-ba9f-972b7ce795c2-kube-api-access-86z9c\") pod \"maas-api-e2e-ait-484f7514-6876b65d9d-7vsp2\" (UID: \"3786e884-a0b0-4487-ba9f-972b7ce795c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-484f7514-6876b65d9d-7vsp2"
Oct 30 21:43:44.207648 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:44.207617    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3786e884-a0b0-4487-ba9f-972b7ce795c2-maas-api-tls\") pod \"maas-api-e2e-ait-484f7514-6876b65d9d-7vsp2\" (UID: \"3786e884-a0b0-4487-ba9f-972b7ce795c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-484f7514-6876b65d9d-7vsp2"
Oct 30 21:43:44.315382 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:44.309480    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3786e884-a0b0-4487-ba9f-972b7ce795c2-openshift-service-ca\") pod \"maas-api-e2e-ait-484f7514-6876b65d9d-7vsp2\" (UID: \"3786e884-a0b0-4487-ba9f-972b7ce795c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-484f7514-6876b65d9d-7vsp2"
Oct 30 21:43:44.315382 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:44.309747    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86z9c\" (UniqueName: \"kubernetes.io/projected/3786e884-a0b0-4487-ba9f-972b7ce795c2-kube-api-access-86z9c\") pod \"maas-api-e2e-ait-484f7514-6876b65d9d-7vsp2\" (UID: \"3786e884-a0b0-4487-ba9f-972b7ce795c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-484f7514-6876b65d9d-7vsp2"
Oct 30 21:43:44.315382 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:44.309795    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3786e884-a0b0-4487-ba9f-972b7ce795c2-maas-api-tls\") pod \"maas-api-e2e-ait-484f7514-6876b65d9d-7vsp2\" (UID: \"3786e884-a0b0-4487-ba9f-972b7ce795c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-484f7514-6876b65d9d-7vsp2"
Oct 30 21:43:44.315382 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:44.309883    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3786e884-a0b0-4487-ba9f-972b7ce795c2-metrics-certs\") pod \"maas-api-e2e-ait-484f7514-6876b65d9d-7vsp2\" (UID: \"3786e884-a0b0-4487-ba9f-972b7ce795c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-484f7514-6876b65d9d-7vsp2"
Oct 30 21:43:44.315382 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:44.313291    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3786e884-a0b0-4487-ba9f-972b7ce795c2-openshift-service-ca\") pod \"maas-api-e2e-ait-484f7514-6876b65d9d-7vsp2\" (UID: \"3786e884-a0b0-4487-ba9f-972b7ce795c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-484f7514-6876b65d9d-7vsp2"
Oct 30 21:43:44.325159 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:44.325090    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3786e884-a0b0-4487-ba9f-972b7ce795c2-metrics-certs\") pod \"maas-api-e2e-ait-484f7514-6876b65d9d-7vsp2\" (UID: \"3786e884-a0b0-4487-ba9f-972b7ce795c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-484f7514-6876b65d9d-7vsp2"
Oct 30 21:43:44.330551 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:44.330519    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3786e884-a0b0-4487-ba9f-972b7ce795c2-maas-api-tls\") pod \"maas-api-e2e-ait-484f7514-6876b65d9d-7vsp2\" (UID: \"3786e884-a0b0-4487-ba9f-972b7ce795c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-484f7514-6876b65d9d-7vsp2"
Oct 30 21:43:44.336263 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:44.336194    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-86z9c\" (UniqueName: \"kubernetes.io/projected/3786e884-a0b0-4487-ba9f-972b7ce795c2-kube-api-access-86z9c\") pod \"maas-api-e2e-ait-484f7514-6876b65d9d-7vsp2\" (UID: \"3786e884-a0b0-4487-ba9f-972b7ce795c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-484f7514-6876b65d9d-7vsp2"
Oct 30 21:43:44.454176 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:44.454136    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-484f7514-6876b65d9d-7vsp2"
Oct 30 21:43:44.743570 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:44.743539    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-484f7514-6876b65d9d-7vsp2"]
Oct 30 21:43:44.743683 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:43:44.743572    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3786e884_a0b0_4487_ba9f_972b7ce795c2.slice/crio-025d052832500ddabd32c67c387817ce0264148f75a9bb199510ea24436bbeb6 WatchSource:0}: Error finding container 025d052832500ddabd32c67c387817ce0264148f75a9bb199510ea24436bbeb6: Status 404 returned error can't find the container with id 025d052832500ddabd32c67c387817ce0264148f75a9bb199510ea24436bbeb6
Oct 30 21:43:45.387663 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:45.387623    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-484f7514-6876b65d9d-7vsp2" event={"ID":"3786e884-a0b0-4487-ba9f-972b7ce795c2","Type":"ContainerStarted","Data":"32094cec41bf262f5d2363dbef1fe1dabf72a04f95636f2782674d733423c5f6"}
Oct 30 21:43:45.388185 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:45.387680    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-484f7514-6876b65d9d-7vsp2" event={"ID":"3786e884-a0b0-4487-ba9f-972b7ce795c2","Type":"ContainerStarted","Data":"025d052832500ddabd32c67c387817ce0264148f75a9bb199510ea24436bbeb6"}
Oct 30 21:43:45.388185 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:45.387806    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-484f7514-6876b65d9d-7vsp2"
Oct 30 21:43:45.422121 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:45.422068    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-484f7514-6876b65d9d-7vsp2" podStartSLOduration=0.968288657 podStartE2EDuration="1.422055078s" totalImagesPullingTime="453.766421ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:43:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:43:45.418436108 +0000 UTC m=+938.220248572" watchObservedRunningTime="2026-10-30 21:43:45.422055078 +0000 UTC m=+938.223867542"
Oct 30 21:43:48.348888 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:43:48.348799    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 21:43:48.370381 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:48.370337    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 21:43:51.464386 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:51.463551    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-484f7514-6876b65d9d-7vsp2"
Oct 30 21:43:51.769447 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:51.768526    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg" podUID="2d0ea058-b6ca-4181-8823-2b719883e545" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:43:51.847386 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:51.846597    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw" podUID="914f7544-6ef8-4496-b98d-b7ebc4123cda" containerName="main" probeResult="failure" output="Get \"https://10.132.0.86:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:43:51.998388 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:51.990740    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw" podUID="2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:43:52.093918 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:52.093810    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj" podUID="d0ff5023-fd3f-4734-94c6-76b666010adf" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:43:52.116400 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:52.112755    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc" podUID="23b97dbb-8488-4c44-a810-882ea18ed919" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:43:52.827398 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:52.773460    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg" podUID="2d0ea058-b6ca-4181-8823-2b719883e545" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:43:52.829326 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:52.778445    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx" podUID="34fdf57d-86f6-474f-a4f6-811e5063c405" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:43:52.830753 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:52.830727    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg"
Oct 30 21:43:52.849791 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:52.849752    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw" podUID="914f7544-6ef8-4496-b98d-b7ebc4123cda" containerName="main" probeResult="failure" output="Get \"https://10.132.0.86:8000/ready\": context deadline exceeded"
Oct 30 21:43:52.874316 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:52.851847    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw"
Oct 30 21:43:53.076393 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:53.024402    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg"
Oct 30 21:43:53.076393 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:53.061890    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw" podUID="2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded"
Oct 30 21:43:53.097455 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:53.088456    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw"
Oct 30 21:43:53.136386 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:53.104565    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj" podUID="d0ff5023-fd3f-4734-94c6-76b666010adf" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:43:53.273442 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:53.115925    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc" podUID="23b97dbb-8488-4c44-a810-882ea18ed919" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:43:53.274154 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:53.274128    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj"
Oct 30 21:43:53.292436 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:53.292397    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc"
Oct 30 21:43:53.957385 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:53.928588    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw"
Oct 30 21:43:53.991908 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:53.991881    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj"
Oct 30 21:43:54.008054 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:54.008010    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc"
Oct 30 21:43:54.102763 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:54.102726    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw" podUID="2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:43:54.403193 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:54.403104    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw"
Oct 30 21:43:56.629640 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:56.629608    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a5af9561-9884bf4bb-w7gn4"]
Oct 30 21:43:56.630056 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:56.629813    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a5af9561-9884bf4bb-w7gn4" podUID="2cc56813-ea74-445b-a09c-dae498e1eaf2" containerName="maas-api" containerID="cri-o://fed2c1d2d874251533e089f15e4d696b027b63d9a32c35b12a20835280fd9062" gracePeriod=30
Oct 30 21:43:56.969511 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:56.969485    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a5af9561-9884bf4bb-w7gn4"
Oct 30 21:43:57.048546 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:57.048506    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2cc56813-ea74-445b-a09c-dae498e1eaf2-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2cc56813-ea74-445b-a09c-dae498e1eaf2-maas-api-tls\") pod \"2cc56813-ea74-445b-a09c-dae498e1eaf2\" (UID: \"2cc56813-ea74-445b-a09c-dae498e1eaf2\") "
Oct 30 21:43:57.048728 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:57.048568    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2cc56813-ea74-445b-a09c-dae498e1eaf2-kube-api-access-4dkjm\" (UniqueName: \"kubernetes.io/projected/2cc56813-ea74-445b-a09c-dae498e1eaf2-kube-api-access-4dkjm\") pod \"2cc56813-ea74-445b-a09c-dae498e1eaf2\" (UID: \"2cc56813-ea74-445b-a09c-dae498e1eaf2\") "
Oct 30 21:43:57.048879 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:57.048845    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2cc56813-ea74-445b-a09c-dae498e1eaf2-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2cc56813-ea74-445b-a09c-dae498e1eaf2-openshift-service-ca\") pod \"2cc56813-ea74-445b-a09c-dae498e1eaf2\" (UID: \"2cc56813-ea74-445b-a09c-dae498e1eaf2\") "
Oct 30 21:43:57.049012 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:57.048991    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2cc56813-ea74-445b-a09c-dae498e1eaf2-metrics-certs\" (UniqueName: \"kubernetes.io/secret/2cc56813-ea74-445b-a09c-dae498e1eaf2-metrics-certs\") pod \"2cc56813-ea74-445b-a09c-dae498e1eaf2\" (UID: \"2cc56813-ea74-445b-a09c-dae498e1eaf2\") "
Oct 30 21:43:57.051139 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:57.050796    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2cc56813-ea74-445b-a09c-dae498e1eaf2-openshift-service-ca" pod "2cc56813-ea74-445b-a09c-dae498e1eaf2" (UID: "2cc56813-ea74-445b-a09c-dae498e1eaf2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:43:57.057508 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:57.056517    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2cc56813-ea74-445b-a09c-dae498e1eaf2-maas-api-tls" pod "2cc56813-ea74-445b-a09c-dae498e1eaf2" (UID: "2cc56813-ea74-445b-a09c-dae498e1eaf2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:43:57.057508 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:57.056538    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2cc56813-ea74-445b-a09c-dae498e1eaf2-metrics-certs" pod "2cc56813-ea74-445b-a09c-dae498e1eaf2" (UID: "2cc56813-ea74-445b-a09c-dae498e1eaf2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:43:57.057508 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:57.056542    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2cc56813-ea74-445b-a09c-dae498e1eaf2-kube-api-access-4dkjm" pod "2cc56813-ea74-445b-a09c-dae498e1eaf2" (UID: "2cc56813-ea74-445b-a09c-dae498e1eaf2"). InnerVolumeSpecName "kube-api-access-4dkjm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:43:57.151766 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:57.151735    5032 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2cc56813-ea74-445b-a09c-dae498e1eaf2-maas-api-tls\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:43:57.151766 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:57.151770    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4dkjm\" (UniqueName: \"kubernetes.io/projected/2cc56813-ea74-445b-a09c-dae498e1eaf2-kube-api-access-4dkjm\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:43:57.151985 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:57.151781    5032 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2cc56813-ea74-445b-a09c-dae498e1eaf2-openshift-service-ca\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:43:57.151985 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:57.151791    5032 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2cc56813-ea74-445b-a09c-dae498e1eaf2-metrics-certs\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:43:57.515667 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:57.515621    5032 generic.go:356] "Generic (PLEG): container finished" podID="2cc56813-ea74-445b-a09c-dae498e1eaf2" containerID="fed2c1d2d874251533e089f15e4d696b027b63d9a32c35b12a20835280fd9062" exitCode=0
Oct 30 21:43:57.515864 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:57.515820    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a5af9561-9884bf4bb-w7gn4" event={"ID":"2cc56813-ea74-445b-a09c-dae498e1eaf2","Type":"ContainerDied","Data":"fed2c1d2d874251533e089f15e4d696b027b63d9a32c35b12a20835280fd9062"}
Oct 30 21:43:57.515936 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:57.515916    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a5af9561-9884bf4bb-w7gn4" event={"ID":"2cc56813-ea74-445b-a09c-dae498e1eaf2","Type":"ContainerDied","Data":"ace4593c63be3275a39605b5a740f2c8a962406b0005c151902cfa661acdf124"}
Oct 30 21:43:57.515980 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:57.515958    5032 scope.go:122] "RemoveContainer" containerID="fed2c1d2d874251533e089f15e4d696b027b63d9a32c35b12a20835280fd9062"
Oct 30 21:43:57.516201 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:57.516175    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a5af9561-9884bf4bb-w7gn4"
Oct 30 21:43:57.531577 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:57.531551    5032 scope.go:122] "RemoveContainer" containerID="fed2c1d2d874251533e089f15e4d696b027b63d9a32c35b12a20835280fd9062"
Oct 30 21:43:57.531943 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:43:57.531915    5032 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fed2c1d2d874251533e089f15e4d696b027b63d9a32c35b12a20835280fd9062\": container with ID starting with fed2c1d2d874251533e089f15e4d696b027b63d9a32c35b12a20835280fd9062 not found: ID does not exist" containerID="fed2c1d2d874251533e089f15e4d696b027b63d9a32c35b12a20835280fd9062"
Oct 30 21:43:57.532036 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:57.531957    5032 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fed2c1d2d874251533e089f15e4d696b027b63d9a32c35b12a20835280fd9062"} err="failed to get container status \"fed2c1d2d874251533e089f15e4d696b027b63d9a32c35b12a20835280fd9062\": rpc error: code = NotFound desc = could not find container \"fed2c1d2d874251533e089f15e4d696b027b63d9a32c35b12a20835280fd9062\": container with ID starting with fed2c1d2d874251533e089f15e4d696b027b63d9a32c35b12a20835280fd9062 not found: ID does not exist"
Oct 30 21:43:57.553498 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:57.553455    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a5af9561-9884bf4bb-w7gn4"]
Oct 30 21:43:57.560427 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:57.560391    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a5af9561-9884bf4bb-w7gn4"]
Oct 30 21:43:59.350371 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:43:59.350325    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2cc56813-ea74-445b-a09c-dae498e1eaf2" path="/var/lib/kubelet/pods/2cc56813-ea74-445b-a09c-dae498e1eaf2/volumes"
Oct 30 21:44:02.786970 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:02.786932    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6d718596-64457bc59f-vf7p9"]
Oct 30 21:44:02.787601 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:02.787200    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6d718596-64457bc59f-vf7p9" podUID="4dd760d4-d256-430f-83c0-226c0688f585" containerName="maas-api" containerID="cri-o://5595444ea397d8322d36b79b23d5159ab9cffaabe3938b83ca341cc961eb634f" gracePeriod=30
Oct 30 21:44:03.206388 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:03.206281    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6d718596-64457bc59f-vf7p9"
Oct 30 21:44:03.319248 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:03.319159    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4dd760d4-d256-430f-83c0-226c0688f585-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4dd760d4-d256-430f-83c0-226c0688f585-maas-api-tls\") pod \"4dd760d4-d256-430f-83c0-226c0688f585\" (UID: \"4dd760d4-d256-430f-83c0-226c0688f585\") "
Oct 30 21:44:03.319980 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:03.319960    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4dd760d4-d256-430f-83c0-226c0688f585-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4dd760d4-d256-430f-83c0-226c0688f585-openshift-service-ca\") pod \"4dd760d4-d256-430f-83c0-226c0688f585\" (UID: \"4dd760d4-d256-430f-83c0-226c0688f585\") "
Oct 30 21:44:03.320692 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:03.320671    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4dd760d4-d256-430f-83c0-226c0688f585-metrics-certs\" (UniqueName: \"kubernetes.io/secret/4dd760d4-d256-430f-83c0-226c0688f585-metrics-certs\") pod \"4dd760d4-d256-430f-83c0-226c0688f585\" (UID: \"4dd760d4-d256-430f-83c0-226c0688f585\") "
Oct 30 21:44:03.321175 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:03.321156    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4dd760d4-d256-430f-83c0-226c0688f585-kube-api-access-fthxw\" (UniqueName: \"kubernetes.io/projected/4dd760d4-d256-430f-83c0-226c0688f585-kube-api-access-fthxw\") pod \"4dd760d4-d256-430f-83c0-226c0688f585\" (UID: \"4dd760d4-d256-430f-83c0-226c0688f585\") "
Oct 30 21:44:03.322087 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:03.320460    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4dd760d4-d256-430f-83c0-226c0688f585-openshift-service-ca" pod "4dd760d4-d256-430f-83c0-226c0688f585" (UID: "4dd760d4-d256-430f-83c0-226c0688f585"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:44:03.326857 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:03.326787    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4dd760d4-d256-430f-83c0-226c0688f585-maas-api-tls" pod "4dd760d4-d256-430f-83c0-226c0688f585" (UID: "4dd760d4-d256-430f-83c0-226c0688f585"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:44:03.327058 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:03.327040    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4dd760d4-d256-430f-83c0-226c0688f585-metrics-certs" pod "4dd760d4-d256-430f-83c0-226c0688f585" (UID: "4dd760d4-d256-430f-83c0-226c0688f585"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:44:03.333590 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:03.333551    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4dd760d4-d256-430f-83c0-226c0688f585-kube-api-access-fthxw" pod "4dd760d4-d256-430f-83c0-226c0688f585" (UID: "4dd760d4-d256-430f-83c0-226c0688f585"). InnerVolumeSpecName "kube-api-access-fthxw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:44:03.424630 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:03.424595    5032 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4dd760d4-d256-430f-83c0-226c0688f585-metrics-certs\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:44:03.424630 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:03.424630    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fthxw\" (UniqueName: \"kubernetes.io/projected/4dd760d4-d256-430f-83c0-226c0688f585-kube-api-access-fthxw\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:44:03.424858 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:03.424647    5032 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4dd760d4-d256-430f-83c0-226c0688f585-maas-api-tls\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:44:03.424858 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:03.424663    5032 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4dd760d4-d256-430f-83c0-226c0688f585-openshift-service-ca\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:44:03.586289 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:03.586181    5032 generic.go:356] "Generic (PLEG): container finished" podID="4dd760d4-d256-430f-83c0-226c0688f585" containerID="5595444ea397d8322d36b79b23d5159ab9cffaabe3938b83ca341cc961eb634f" exitCode=0
Oct 30 21:44:03.586289 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:03.586235    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6d718596-64457bc59f-vf7p9"
Oct 30 21:44:03.586565 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:03.586472    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6d718596-64457bc59f-vf7p9" event={"ID":"4dd760d4-d256-430f-83c0-226c0688f585","Type":"ContainerDied","Data":"5595444ea397d8322d36b79b23d5159ab9cffaabe3938b83ca341cc961eb634f"}
Oct 30 21:44:03.586565 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:03.586517    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6d718596-64457bc59f-vf7p9" event={"ID":"4dd760d4-d256-430f-83c0-226c0688f585","Type":"ContainerDied","Data":"935338fb2e9fe9db1663a57cb51dcc85bc649d9342806102eaf7e714fda8a04d"}
Oct 30 21:44:03.586565 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:03.586544    5032 scope.go:122] "RemoveContainer" containerID="5595444ea397d8322d36b79b23d5159ab9cffaabe3938b83ca341cc961eb634f"
Oct 30 21:44:03.599822 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:03.599790    5032 scope.go:122] "RemoveContainer" containerID="5595444ea397d8322d36b79b23d5159ab9cffaabe3938b83ca341cc961eb634f"
Oct 30 21:44:03.600186 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:44:03.600161    5032 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5595444ea397d8322d36b79b23d5159ab9cffaabe3938b83ca341cc961eb634f\": container with ID starting with 5595444ea397d8322d36b79b23d5159ab9cffaabe3938b83ca341cc961eb634f not found: ID does not exist" containerID="5595444ea397d8322d36b79b23d5159ab9cffaabe3938b83ca341cc961eb634f"
Oct 30 21:44:03.600255 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:03.600198    5032 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5595444ea397d8322d36b79b23d5159ab9cffaabe3938b83ca341cc961eb634f"} err="failed to get container status \"5595444ea397d8322d36b79b23d5159ab9cffaabe3938b83ca341cc961eb634f\": rpc error: code = NotFound desc = could not find container \"5595444ea397d8322d36b79b23d5159ab9cffaabe3938b83ca341cc961eb634f\": container with ID starting with 5595444ea397d8322d36b79b23d5159ab9cffaabe3938b83ca341cc961eb634f not found: ID does not exist"
Oct 30 21:44:03.612415 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:03.608606    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6d718596-64457bc59f-vf7p9"]
Oct 30 21:44:03.614100 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:03.614069    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6d718596-64457bc59f-vf7p9"]
Oct 30 21:44:05.348510 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:05.348476    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4dd760d4-d256-430f-83c0-226c0688f585" path="/var/lib/kubelet/pods/4dd760d4-d256-430f-83c0-226c0688f585/volumes"
Oct 30 21:44:14.128253 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:14.128221    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-2b36c1-799756d5b8-2flb4"]
Oct 30 21:44:14.129064 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:14.129045    5032 cpu_manager.go:414] "containerMap: removing container" podUID="4dd760d4-d256-430f-83c0-226c0688f585" containerName="maas-api"
Oct 30 21:44:14.129126 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:14.129065    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4dd760d4-d256-430f-83c0-226c0688f585" containerName="maas-api"
Oct 30 21:44:14.129126 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:14.129082    5032 cpu_manager.go:414] "containerMap: removing container" podUID="2cc56813-ea74-445b-a09c-dae498e1eaf2" containerName="maas-api"
Oct 30 21:44:14.129126 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:14.129089    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2cc56813-ea74-445b-a09c-dae498e1eaf2" containerName="maas-api"
Oct 30 21:44:14.129246 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:14.129236    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="4dd760d4-d256-430f-83c0-226c0688f585" containerName="maas-api"
Oct 30 21:44:14.129281 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:14.129249    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="2cc56813-ea74-445b-a09c-dae498e1eaf2" containerName="maas-api"
Oct 30 21:44:14.135895 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:14.134912    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-2b36c1-799756d5b8-2flb4"
Oct 30 21:44:14.139022 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:14.138995    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-2b36c1-799756d5b8-2flb4"]
Oct 30 21:44:14.141535 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:14.141514    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w1-2b36c1-serving-cert\""
Oct 30 21:44:14.262190 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:14.262157    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3eecfc0e-8c6b-4d24-b14b-242e6e52829d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w1-2b36c1-799756d5b8-2flb4\" (UID: \"3eecfc0e-8c6b-4d24-b14b-242e6e52829d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-2b36c1-799756d5b8-2flb4"
Oct 30 21:44:14.262390 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:14.262227    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3eecfc0e-8c6b-4d24-b14b-242e6e52829d-metrics-certs\") pod \"maas-api-e2e-shared-a-w1-2b36c1-799756d5b8-2flb4\" (UID: \"3eecfc0e-8c6b-4d24-b14b-242e6e52829d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-2b36c1-799756d5b8-2flb4"
Oct 30 21:44:14.262453 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:14.262391    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3eecfc0e-8c6b-4d24-b14b-242e6e52829d-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-2b36c1-799756d5b8-2flb4\" (UID: \"3eecfc0e-8c6b-4d24-b14b-242e6e52829d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-2b36c1-799756d5b8-2flb4"
Oct 30 21:44:14.262626 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:14.262610    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hk9gt\" (UniqueName: \"kubernetes.io/projected/3eecfc0e-8c6b-4d24-b14b-242e6e52829d-kube-api-access-hk9gt\") pod \"maas-api-e2e-shared-a-w1-2b36c1-799756d5b8-2flb4\" (UID: \"3eecfc0e-8c6b-4d24-b14b-242e6e52829d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-2b36c1-799756d5b8-2flb4"
Oct 30 21:44:14.364705 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:14.364667    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3eecfc0e-8c6b-4d24-b14b-242e6e52829d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w1-2b36c1-799756d5b8-2flb4\" (UID: \"3eecfc0e-8c6b-4d24-b14b-242e6e52829d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-2b36c1-799756d5b8-2flb4"
Oct 30 21:44:14.364904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:14.364751    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3eecfc0e-8c6b-4d24-b14b-242e6e52829d-metrics-certs\") pod \"maas-api-e2e-shared-a-w1-2b36c1-799756d5b8-2flb4\" (UID: \"3eecfc0e-8c6b-4d24-b14b-242e6e52829d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-2b36c1-799756d5b8-2flb4"
Oct 30 21:44:14.364904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:14.364881    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3eecfc0e-8c6b-4d24-b14b-242e6e52829d-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-2b36c1-799756d5b8-2flb4\" (UID: \"3eecfc0e-8c6b-4d24-b14b-242e6e52829d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-2b36c1-799756d5b8-2flb4"
Oct 30 21:44:14.365159 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:14.365137    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hk9gt\" (UniqueName: \"kubernetes.io/projected/3eecfc0e-8c6b-4d24-b14b-242e6e52829d-kube-api-access-hk9gt\") pod \"maas-api-e2e-shared-a-w1-2b36c1-799756d5b8-2flb4\" (UID: \"3eecfc0e-8c6b-4d24-b14b-242e6e52829d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-2b36c1-799756d5b8-2flb4"
Oct 30 21:44:14.365559 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:14.365539    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3eecfc0e-8c6b-4d24-b14b-242e6e52829d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w1-2b36c1-799756d5b8-2flb4\" (UID: \"3eecfc0e-8c6b-4d24-b14b-242e6e52829d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-2b36c1-799756d5b8-2flb4"
Oct 30 21:44:14.367614 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:14.367589    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3eecfc0e-8c6b-4d24-b14b-242e6e52829d-metrics-certs\") pod \"maas-api-e2e-shared-a-w1-2b36c1-799756d5b8-2flb4\" (UID: \"3eecfc0e-8c6b-4d24-b14b-242e6e52829d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-2b36c1-799756d5b8-2flb4"
Oct 30 21:44:14.367717 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:14.367698    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3eecfc0e-8c6b-4d24-b14b-242e6e52829d-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-2b36c1-799756d5b8-2flb4\" (UID: \"3eecfc0e-8c6b-4d24-b14b-242e6e52829d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-2b36c1-799756d5b8-2flb4"
Oct 30 21:44:14.380477 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:14.380399    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hk9gt\" (UniqueName: \"kubernetes.io/projected/3eecfc0e-8c6b-4d24-b14b-242e6e52829d-kube-api-access-hk9gt\") pod \"maas-api-e2e-shared-a-w1-2b36c1-799756d5b8-2flb4\" (UID: \"3eecfc0e-8c6b-4d24-b14b-242e6e52829d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-2b36c1-799756d5b8-2flb4"
Oct 30 21:44:14.466426 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:14.466387    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-2b36c1-799756d5b8-2flb4"
Oct 30 21:44:14.759179 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:44:14.759142    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3eecfc0e_8c6b_4d24_b14b_242e6e52829d.slice/crio-3b498a23a040429da0be75bdfd226034ad85a073c0a3591d32d3a7d300d7122a WatchSource:0}: Error finding container 3b498a23a040429da0be75bdfd226034ad85a073c0a3591d32d3a7d300d7122a: Status 404 returned error can't find the container with id 3b498a23a040429da0be75bdfd226034ad85a073c0a3591d32d3a7d300d7122a
Oct 30 21:44:14.759348 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:14.759324    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-2b36c1-799756d5b8-2flb4"]
Oct 30 21:44:14.783044 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:14.783012    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-2b36c1-799756d5b8-2flb4" event={"ID":"3eecfc0e-8c6b-4d24-b14b-242e6e52829d","Type":"ContainerStarted","Data":"3b498a23a040429da0be75bdfd226034ad85a073c0a3591d32d3a7d300d7122a"}
Oct 30 21:44:15.792785 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:15.792746    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-2b36c1-799756d5b8-2flb4" event={"ID":"3eecfc0e-8c6b-4d24-b14b-242e6e52829d","Type":"ContainerStarted","Data":"3c959a66d56f631820780ffa8da16c97d6a9a69641312e653cabe92b738fb100"}
Oct 30 21:44:15.793282 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:15.792846    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-2b36c1-799756d5b8-2flb4"
Oct 30 21:44:15.821961 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:15.821908    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-2b36c1-799756d5b8-2flb4" podStartSLOduration=1.379974093 podStartE2EDuration="1.821892541s" totalImagesPullingTime="441.918448ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:44:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:44:15.817555337 +0000 UTC m=+968.619367800" watchObservedRunningTime="2026-10-30 21:44:15.821892541 +0000 UTC m=+968.623705005"
Oct 30 21:44:16.768050 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:16.768003    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-484f7514-6876b65d9d-7vsp2"]
Oct 30 21:44:16.768351 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:16.768299    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-484f7514-6876b65d9d-7vsp2" podUID="3786e884-a0b0-4487-ba9f-972b7ce795c2" containerName="maas-api" containerID="cri-o://32094cec41bf262f5d2363dbef1fe1dabf72a04f95636f2782674d733423c5f6" gracePeriod=30
Oct 30 21:44:17.085201 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:17.085168    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-484f7514-6876b65d9d-7vsp2"
Oct 30 21:44:17.121808 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:17.121773    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3786e884-a0b0-4487-ba9f-972b7ce795c2-kube-api-access-86z9c\" (UniqueName: \"kubernetes.io/projected/3786e884-a0b0-4487-ba9f-972b7ce795c2-kube-api-access-86z9c\") pod \"3786e884-a0b0-4487-ba9f-972b7ce795c2\" (UID: \"3786e884-a0b0-4487-ba9f-972b7ce795c2\") "
Oct 30 21:44:17.122009 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:17.121834    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3786e884-a0b0-4487-ba9f-972b7ce795c2-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3786e884-a0b0-4487-ba9f-972b7ce795c2-maas-api-tls\") pod \"3786e884-a0b0-4487-ba9f-972b7ce795c2\" (UID: \"3786e884-a0b0-4487-ba9f-972b7ce795c2\") "
Oct 30 21:44:17.122009 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:17.121911    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3786e884-a0b0-4487-ba9f-972b7ce795c2-metrics-certs\" (UniqueName: \"kubernetes.io/secret/3786e884-a0b0-4487-ba9f-972b7ce795c2-metrics-certs\") pod \"3786e884-a0b0-4487-ba9f-972b7ce795c2\" (UID: \"3786e884-a0b0-4487-ba9f-972b7ce795c2\") "
Oct 30 21:44:17.122009 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:17.121953    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3786e884-a0b0-4487-ba9f-972b7ce795c2-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3786e884-a0b0-4487-ba9f-972b7ce795c2-openshift-service-ca\") pod \"3786e884-a0b0-4487-ba9f-972b7ce795c2\" (UID: \"3786e884-a0b0-4487-ba9f-972b7ce795c2\") "
Oct 30 21:44:17.122747 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:17.122714    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3786e884-a0b0-4487-ba9f-972b7ce795c2-openshift-service-ca" pod "3786e884-a0b0-4487-ba9f-972b7ce795c2" (UID: "3786e884-a0b0-4487-ba9f-972b7ce795c2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:44:17.122982 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:17.122957    5032 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3786e884-a0b0-4487-ba9f-972b7ce795c2-openshift-service-ca\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:44:17.124566 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:17.124537    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3786e884-a0b0-4487-ba9f-972b7ce795c2-maas-api-tls" pod "3786e884-a0b0-4487-ba9f-972b7ce795c2" (UID: "3786e884-a0b0-4487-ba9f-972b7ce795c2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:44:17.126115 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:17.126086    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3786e884-a0b0-4487-ba9f-972b7ce795c2-metrics-certs" pod "3786e884-a0b0-4487-ba9f-972b7ce795c2" (UID: "3786e884-a0b0-4487-ba9f-972b7ce795c2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:44:17.126194 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:17.126111    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3786e884-a0b0-4487-ba9f-972b7ce795c2-kube-api-access-86z9c" pod "3786e884-a0b0-4487-ba9f-972b7ce795c2" (UID: "3786e884-a0b0-4487-ba9f-972b7ce795c2"). InnerVolumeSpecName "kube-api-access-86z9c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:44:17.225109 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:17.225070    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-86z9c\" (UniqueName: \"kubernetes.io/projected/3786e884-a0b0-4487-ba9f-972b7ce795c2-kube-api-access-86z9c\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:44:17.225109 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:17.225108    5032 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3786e884-a0b0-4487-ba9f-972b7ce795c2-maas-api-tls\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:44:17.225320 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:17.225121    5032 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3786e884-a0b0-4487-ba9f-972b7ce795c2-metrics-certs\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:44:17.824940 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:17.824903    5032 generic.go:356] "Generic (PLEG): container finished" podID="3786e884-a0b0-4487-ba9f-972b7ce795c2" containerID="32094cec41bf262f5d2363dbef1fe1dabf72a04f95636f2782674d733423c5f6" exitCode=0
Oct 30 21:44:17.825137 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:17.825019    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-484f7514-6876b65d9d-7vsp2" event={"ID":"3786e884-a0b0-4487-ba9f-972b7ce795c2","Type":"ContainerDied","Data":"32094cec41bf262f5d2363dbef1fe1dabf72a04f95636f2782674d733423c5f6"}
Oct 30 21:44:17.825137 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:17.825073    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-484f7514-6876b65d9d-7vsp2" event={"ID":"3786e884-a0b0-4487-ba9f-972b7ce795c2","Type":"ContainerDied","Data":"025d052832500ddabd32c67c387817ce0264148f75a9bb199510ea24436bbeb6"}
Oct 30 21:44:17.825137 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:17.825102    5032 scope.go:122] "RemoveContainer" containerID="32094cec41bf262f5d2363dbef1fe1dabf72a04f95636f2782674d733423c5f6"
Oct 30 21:44:17.825312 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:17.825260    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-484f7514-6876b65d9d-7vsp2"
Oct 30 21:44:17.841578 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:17.841556    5032 scope.go:122] "RemoveContainer" containerID="32094cec41bf262f5d2363dbef1fe1dabf72a04f95636f2782674d733423c5f6"
Oct 30 21:44:17.842049 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:44:17.842018    5032 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"32094cec41bf262f5d2363dbef1fe1dabf72a04f95636f2782674d733423c5f6\": container with ID starting with 32094cec41bf262f5d2363dbef1fe1dabf72a04f95636f2782674d733423c5f6 not found: ID does not exist" containerID="32094cec41bf262f5d2363dbef1fe1dabf72a04f95636f2782674d733423c5f6"
Oct 30 21:44:17.842150 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:17.842060    5032 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32094cec41bf262f5d2363dbef1fe1dabf72a04f95636f2782674d733423c5f6"} err="failed to get container status \"32094cec41bf262f5d2363dbef1fe1dabf72a04f95636f2782674d733423c5f6\": rpc error: code = NotFound desc = could not find container \"32094cec41bf262f5d2363dbef1fe1dabf72a04f95636f2782674d733423c5f6\": container with ID starting with 32094cec41bf262f5d2363dbef1fe1dabf72a04f95636f2782674d733423c5f6 not found: ID does not exist"
Oct 30 21:44:17.859680 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:17.859649    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-484f7514-6876b65d9d-7vsp2"]
Oct 30 21:44:17.864074 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:17.864046    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-484f7514-6876b65d9d-7vsp2"]
Oct 30 21:44:18.347667 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:44:18.347619    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 21:44:19.006040 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:19.005989    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-1f725d-7678fd7cb5-ntchg"]
Oct 30 21:44:19.007436 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:19.007340    5032 cpu_manager.go:414] "containerMap: removing container" podUID="3786e884-a0b0-4487-ba9f-972b7ce795c2" containerName="maas-api"
Oct 30 21:44:19.008175 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:19.008144    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3786e884-a0b0-4487-ba9f-972b7ce795c2" containerName="maas-api"
Oct 30 21:44:19.008457 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:19.008439    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="3786e884-a0b0-4487-ba9f-972b7ce795c2" containerName="maas-api"
Oct 30 21:44:19.014564 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:19.014542    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-1f725d-7678fd7cb5-ntchg"
Oct 30 21:44:19.020429 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:19.020405    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-1f725d-serving-cert\""
Oct 30 21:44:19.033913 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:19.033096    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-1f725d-7678fd7cb5-ntchg"]
Oct 30 21:44:19.082772 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:19.082118    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b68282ae-6782-4b8f-9686-73089f54ff78-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-1f725d-7678fd7cb5-ntchg\" (UID: \"b68282ae-6782-4b8f-9686-73089f54ff78\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-1f725d-7678fd7cb5-ntchg"
Oct 30 21:44:19.082772 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:19.082298    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b68282ae-6782-4b8f-9686-73089f54ff78-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-1f725d-7678fd7cb5-ntchg\" (UID: \"b68282ae-6782-4b8f-9686-73089f54ff78\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-1f725d-7678fd7cb5-ntchg"
Oct 30 21:44:19.082772 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:19.082433    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5m89f\" (UniqueName: \"kubernetes.io/projected/b68282ae-6782-4b8f-9686-73089f54ff78-kube-api-access-5m89f\") pod \"maas-api-e2e-shared-a-w2-1f725d-7678fd7cb5-ntchg\" (UID: \"b68282ae-6782-4b8f-9686-73089f54ff78\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-1f725d-7678fd7cb5-ntchg"
Oct 30 21:44:19.082772 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:19.082568    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b68282ae-6782-4b8f-9686-73089f54ff78-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-1f725d-7678fd7cb5-ntchg\" (UID: \"b68282ae-6782-4b8f-9686-73089f54ff78\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-1f725d-7678fd7cb5-ntchg"
Oct 30 21:44:19.184094 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:19.184049    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b68282ae-6782-4b8f-9686-73089f54ff78-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-1f725d-7678fd7cb5-ntchg\" (UID: \"b68282ae-6782-4b8f-9686-73089f54ff78\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-1f725d-7678fd7cb5-ntchg"
Oct 30 21:44:19.184274 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:19.184147    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5m89f\" (UniqueName: \"kubernetes.io/projected/b68282ae-6782-4b8f-9686-73089f54ff78-kube-api-access-5m89f\") pod \"maas-api-e2e-shared-a-w2-1f725d-7678fd7cb5-ntchg\" (UID: \"b68282ae-6782-4b8f-9686-73089f54ff78\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-1f725d-7678fd7cb5-ntchg"
Oct 30 21:44:19.184317 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:19.184276    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b68282ae-6782-4b8f-9686-73089f54ff78-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-1f725d-7678fd7cb5-ntchg\" (UID: \"b68282ae-6782-4b8f-9686-73089f54ff78\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-1f725d-7678fd7cb5-ntchg"
Oct 30 21:44:19.184461 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:19.184445    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b68282ae-6782-4b8f-9686-73089f54ff78-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-1f725d-7678fd7cb5-ntchg\" (UID: \"b68282ae-6782-4b8f-9686-73089f54ff78\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-1f725d-7678fd7cb5-ntchg"
Oct 30 21:44:19.185753 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:19.185725    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b68282ae-6782-4b8f-9686-73089f54ff78-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-1f725d-7678fd7cb5-ntchg\" (UID: \"b68282ae-6782-4b8f-9686-73089f54ff78\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-1f725d-7678fd7cb5-ntchg"
Oct 30 21:44:19.187330 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:19.187303    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b68282ae-6782-4b8f-9686-73089f54ff78-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-1f725d-7678fd7cb5-ntchg\" (UID: \"b68282ae-6782-4b8f-9686-73089f54ff78\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-1f725d-7678fd7cb5-ntchg"
Oct 30 21:44:19.187564 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:19.187546    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b68282ae-6782-4b8f-9686-73089f54ff78-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-1f725d-7678fd7cb5-ntchg\" (UID: \"b68282ae-6782-4b8f-9686-73089f54ff78\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-1f725d-7678fd7cb5-ntchg"
Oct 30 21:44:19.198199 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:19.198177    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5m89f\" (UniqueName: \"kubernetes.io/projected/b68282ae-6782-4b8f-9686-73089f54ff78-kube-api-access-5m89f\") pod \"maas-api-e2e-shared-a-w2-1f725d-7678fd7cb5-ntchg\" (UID: \"b68282ae-6782-4b8f-9686-73089f54ff78\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-1f725d-7678fd7cb5-ntchg"
Oct 30 21:44:19.347830 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:19.347721    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-1f725d-7678fd7cb5-ntchg"
Oct 30 21:44:19.363282 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:19.363244    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3786e884-a0b0-4487-ba9f-972b7ce795c2" path="/var/lib/kubelet/pods/3786e884-a0b0-4487-ba9f-972b7ce795c2/volumes"
Oct 30 21:44:19.677924 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:19.677888    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-1f725d-7678fd7cb5-ntchg"]
Oct 30 21:44:19.678665 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:44:19.678634    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb68282ae_6782_4b8f_9686_73089f54ff78.slice/crio-c562c534d8bb7d222fa7884dd5e91aa25c85202c7941e0506d712282b2d19699 WatchSource:0}: Error finding container c562c534d8bb7d222fa7884dd5e91aa25c85202c7941e0506d712282b2d19699: Status 404 returned error can't find the container with id c562c534d8bb7d222fa7884dd5e91aa25c85202c7941e0506d712282b2d19699
Oct 30 21:44:19.924424 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:19.924327    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-1f725d-7678fd7cb5-ntchg" event={"ID":"b68282ae-6782-4b8f-9686-73089f54ff78","Type":"ContainerStarted","Data":"c562c534d8bb7d222fa7884dd5e91aa25c85202c7941e0506d712282b2d19699"}
Oct 30 21:44:20.935452 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:20.935417    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-1f725d-7678fd7cb5-ntchg" event={"ID":"b68282ae-6782-4b8f-9686-73089f54ff78","Type":"ContainerStarted","Data":"1d830815087efc871482b672a5263c6704a207bd47f1d95dcf7c22da5edc8014"}
Oct 30 21:44:20.935910 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:20.935496    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-1f725d-7678fd7cb5-ntchg"
Oct 30 21:44:20.963633 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:20.963580    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-1f725d-7678fd7cb5-ntchg" podStartSLOduration=2.501756356 podStartE2EDuration="2.963564847s" totalImagesPullingTime="461.808491ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:44:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:44:20.957859152 +0000 UTC m=+973.759671620" watchObservedRunningTime="2026-10-30 21:44:20.963564847 +0000 UTC m=+973.765377311"
Oct 30 21:44:21.813241 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:21.813195    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-2b36c1-799756d5b8-2flb4"
Oct 30 21:44:26.951958 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:26.951920    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-1f725d-7678fd7cb5-ntchg"
Oct 30 21:44:32.154594 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:32.154547    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-54h7q"]
Oct 30 21:44:32.160598 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:32.160572    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-54h7q"
Oct 30 21:44:32.176188 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:32.176159    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-54h7q"]
Oct 30 21:44:32.198962 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:32.198927    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/56d64255-3745-4df3-b9f3-9a15db845698-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-54h7q\" (UID: \"56d64255-3745-4df3-b9f3-9a15db845698\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-54h7q"
Oct 30 21:44:32.199130 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:32.198997    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/56d64255-3745-4df3-b9f3-9a15db845698-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-54h7q\" (UID: \"56d64255-3745-4df3-b9f3-9a15db845698\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-54h7q"
Oct 30 21:44:32.199130 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:32.199109    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/56d64255-3745-4df3-b9f3-9a15db845698-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-54h7q\" (UID: \"56d64255-3745-4df3-b9f3-9a15db845698\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-54h7q"
Oct 30 21:44:32.199231 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:32.199167    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntqw5\" (UniqueName: \"kubernetes.io/projected/56d64255-3745-4df3-b9f3-9a15db845698-kube-api-access-ntqw5\") pod \"istiod-openshift-gateway-7ddd9cfdc8-54h7q\" (UID: \"56d64255-3745-4df3-b9f3-9a15db845698\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-54h7q"
Oct 30 21:44:32.199354 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:32.199330    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/56d64255-3745-4df3-b9f3-9a15db845698-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-54h7q\" (UID: \"56d64255-3745-4df3-b9f3-9a15db845698\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-54h7q"
Oct 30 21:44:32.199458 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:32.199436    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/56d64255-3745-4df3-b9f3-9a15db845698-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-54h7q\" (UID: \"56d64255-3745-4df3-b9f3-9a15db845698\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-54h7q"
Oct 30 21:44:32.199561 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:32.199545    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56d64255-3745-4df3-b9f3-9a15db845698-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-54h7q\" (UID: \"56d64255-3745-4df3-b9f3-9a15db845698\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-54h7q"
Oct 30 21:44:32.301616 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:32.301578    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/56d64255-3745-4df3-b9f3-9a15db845698-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-54h7q\" (UID: \"56d64255-3745-4df3-b9f3-9a15db845698\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-54h7q"
Oct 30 21:44:32.301820 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:32.301632    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/56d64255-3745-4df3-b9f3-9a15db845698-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-54h7q\" (UID: \"56d64255-3745-4df3-b9f3-9a15db845698\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-54h7q"
Oct 30 21:44:32.301820 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:32.301756    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/56d64255-3745-4df3-b9f3-9a15db845698-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-54h7q\" (UID: \"56d64255-3745-4df3-b9f3-9a15db845698\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-54h7q"
Oct 30 21:44:32.301820 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:32.301812    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntqw5\" (UniqueName: \"kubernetes.io/projected/56d64255-3745-4df3-b9f3-9a15db845698-kube-api-access-ntqw5\") pod \"istiod-openshift-gateway-7ddd9cfdc8-54h7q\" (UID: \"56d64255-3745-4df3-b9f3-9a15db845698\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-54h7q"
Oct 30 21:44:32.301991 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:32.301934    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/56d64255-3745-4df3-b9f3-9a15db845698-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-54h7q\" (UID: \"56d64255-3745-4df3-b9f3-9a15db845698\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-54h7q"
Oct 30 21:44:32.302045 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:32.301990    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/56d64255-3745-4df3-b9f3-9a15db845698-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-54h7q\" (UID: \"56d64255-3745-4df3-b9f3-9a15db845698\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-54h7q"
Oct 30 21:44:32.302136 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:32.302098    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56d64255-3745-4df3-b9f3-9a15db845698-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-54h7q\" (UID: \"56d64255-3745-4df3-b9f3-9a15db845698\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-54h7q"
Oct 30 21:44:32.302798 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:32.302769    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/56d64255-3745-4df3-b9f3-9a15db845698-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-54h7q\" (UID: \"56d64255-3745-4df3-b9f3-9a15db845698\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-54h7q"
Oct 30 21:44:32.304265 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:32.304242    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/56d64255-3745-4df3-b9f3-9a15db845698-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-54h7q\" (UID: \"56d64255-3745-4df3-b9f3-9a15db845698\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-54h7q"
Oct 30 21:44:32.304592 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:32.304570    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/56d64255-3745-4df3-b9f3-9a15db845698-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-54h7q\" (UID: \"56d64255-3745-4df3-b9f3-9a15db845698\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-54h7q"
Oct 30 21:44:32.304675 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:32.304608    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/56d64255-3745-4df3-b9f3-9a15db845698-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-54h7q\" (UID: \"56d64255-3745-4df3-b9f3-9a15db845698\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-54h7q"
Oct 30 21:44:32.304675 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:32.304646    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/56d64255-3745-4df3-b9f3-9a15db845698-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-54h7q\" (UID: \"56d64255-3745-4df3-b9f3-9a15db845698\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-54h7q"
Oct 30 21:44:32.318550 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:32.318528    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56d64255-3745-4df3-b9f3-9a15db845698-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-54h7q\" (UID: \"56d64255-3745-4df3-b9f3-9a15db845698\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-54h7q"
Oct 30 21:44:32.324233 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:32.324211    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntqw5\" (UniqueName: \"kubernetes.io/projected/56d64255-3745-4df3-b9f3-9a15db845698-kube-api-access-ntqw5\") pod \"istiod-openshift-gateway-7ddd9cfdc8-54h7q\" (UID: \"56d64255-3745-4df3-b9f3-9a15db845698\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-54h7q"
Oct 30 21:44:32.479503 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:32.477081    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-54h7q"
Oct 30 21:44:32.788789 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:32.788758    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-54h7q"]
Oct 30 21:44:32.789384 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:44:32.789341    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56d64255_3745_4df3_b9f3_9a15db845698.slice/crio-56ba11ead5c7fa1d91d7b73f756fa69b529233047e3ff4cda084b3ad3a63bbab WatchSource:0}: Error finding container 56ba11ead5c7fa1d91d7b73f756fa69b529233047e3ff4cda084b3ad3a63bbab: Status 404 returned error can't find the container with id 56ba11ead5c7fa1d91d7b73f756fa69b529233047e3ff4cda084b3ad3a63bbab
Oct 30 21:44:32.792134 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:32.792102    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:44:32.792200 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:32.792165    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:44:33.077438 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:33.077312    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-54h7q" event={"ID":"56d64255-3745-4df3-b9f3-9a15db845698","Type":"ContainerStarted","Data":"d4d452f3a4fcedff664a52ee2e3b2fe1fd9311fa6b4219d56cc165516f404f94"}
Oct 30 21:44:33.077438 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:33.077417    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-54h7q" event={"ID":"56d64255-3745-4df3-b9f3-9a15db845698","Type":"ContainerStarted","Data":"56ba11ead5c7fa1d91d7b73f756fa69b529233047e3ff4cda084b3ad3a63bbab"}
Oct 30 21:44:33.077737 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:33.077452    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-54h7q"
Oct 30 21:44:33.079224 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:33.079194    5032 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-54h7q container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:44:33.079415 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:33.079247    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-54h7q" podUID="56d64255-3745-4df3-b9f3-9a15db845698" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:44:33.130775 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:33.130716    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-54h7q" podStartSLOduration=1.130697478 podStartE2EDuration="1.130697478s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:44:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:44:33.126593867 +0000 UTC m=+985.928406334" watchObservedRunningTime="2026-10-30 21:44:33.130697478 +0000 UTC m=+985.932509943"
Oct 30 21:44:33.316067 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:33.316033    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 21:44:34.122386 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:34.114493    5032 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-54h7q container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:44:34.122386 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:34.114577    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-54h7q" podUID="56d64255-3745-4df3-b9f3-9a15db845698" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:44:37.101088 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:37.101051    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-54h7q"
Oct 30 21:44:48.347636 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:44:48.347594    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 21:44:50.714147 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:50.714112    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fbf428-59dbc6d9d7-tndfr"]
Oct 30 21:44:50.720445 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:50.720410    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fbf428-59dbc6d9d7-tndfr"
Oct 30 21:44:50.745318 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:50.745277    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-fbf428-serving-cert\""
Oct 30 21:44:50.748701 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:50.748669    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fbf428-59dbc6d9d7-tndfr"]
Oct 30 21:44:50.805635 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:50.805580    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c6d3630-8f1a-4298-ab10-d5619869e909-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-fbf428-59dbc6d9d7-tndfr\" (UID: \"4c6d3630-8f1a-4298-ab10-d5619869e909\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fbf428-59dbc6d9d7-tndfr"
Oct 30 21:44:50.805859 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:50.805727    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mwr9\" (UniqueName: \"kubernetes.io/projected/4c6d3630-8f1a-4298-ab10-d5619869e909-kube-api-access-7mwr9\") pod \"maas-api-e2e-shared-b-w2-fbf428-59dbc6d9d7-tndfr\" (UID: \"4c6d3630-8f1a-4298-ab10-d5619869e909\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fbf428-59dbc6d9d7-tndfr"
Oct 30 21:44:50.805940 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:50.805920    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c6d3630-8f1a-4298-ab10-d5619869e909-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-fbf428-59dbc6d9d7-tndfr\" (UID: \"4c6d3630-8f1a-4298-ab10-d5619869e909\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fbf428-59dbc6d9d7-tndfr"
Oct 30 21:44:50.806107 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:50.806090    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c6d3630-8f1a-4298-ab10-d5619869e909-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-fbf428-59dbc6d9d7-tndfr\" (UID: \"4c6d3630-8f1a-4298-ab10-d5619869e909\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fbf428-59dbc6d9d7-tndfr"
Oct 30 21:44:50.917655 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:50.917609    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c6d3630-8f1a-4298-ab10-d5619869e909-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-fbf428-59dbc6d9d7-tndfr\" (UID: \"4c6d3630-8f1a-4298-ab10-d5619869e909\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fbf428-59dbc6d9d7-tndfr"
Oct 30 21:44:50.918392 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:50.918319    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c6d3630-8f1a-4298-ab10-d5619869e909-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-fbf428-59dbc6d9d7-tndfr\" (UID: \"4c6d3630-8f1a-4298-ab10-d5619869e909\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fbf428-59dbc6d9d7-tndfr"
Oct 30 21:44:50.920291 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:50.920232    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c6d3630-8f1a-4298-ab10-d5619869e909-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-fbf428-59dbc6d9d7-tndfr\" (UID: \"4c6d3630-8f1a-4298-ab10-d5619869e909\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fbf428-59dbc6d9d7-tndfr"
Oct 30 21:44:50.921063 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:50.921040    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mwr9\" (UniqueName: \"kubernetes.io/projected/4c6d3630-8f1a-4298-ab10-d5619869e909-kube-api-access-7mwr9\") pod \"maas-api-e2e-shared-b-w2-fbf428-59dbc6d9d7-tndfr\" (UID: \"4c6d3630-8f1a-4298-ab10-d5619869e909\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fbf428-59dbc6d9d7-tndfr"
Oct 30 21:44:50.921957 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:50.919457    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c6d3630-8f1a-4298-ab10-d5619869e909-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-fbf428-59dbc6d9d7-tndfr\" (UID: \"4c6d3630-8f1a-4298-ab10-d5619869e909\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fbf428-59dbc6d9d7-tndfr"
Oct 30 21:44:50.932223 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:50.932128    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c6d3630-8f1a-4298-ab10-d5619869e909-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-fbf428-59dbc6d9d7-tndfr\" (UID: \"4c6d3630-8f1a-4298-ab10-d5619869e909\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fbf428-59dbc6d9d7-tndfr"
Oct 30 21:44:50.957567 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:50.940355    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c6d3630-8f1a-4298-ab10-d5619869e909-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-fbf428-59dbc6d9d7-tndfr\" (UID: \"4c6d3630-8f1a-4298-ab10-d5619869e909\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fbf428-59dbc6d9d7-tndfr"
Oct 30 21:44:50.961012 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:50.960941    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mwr9\" (UniqueName: \"kubernetes.io/projected/4c6d3630-8f1a-4298-ab10-d5619869e909-kube-api-access-7mwr9\") pod \"maas-api-e2e-shared-b-w2-fbf428-59dbc6d9d7-tndfr\" (UID: \"4c6d3630-8f1a-4298-ab10-d5619869e909\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fbf428-59dbc6d9d7-tndfr"
Oct 30 21:44:51.037305 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:51.037220    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fbf428-59dbc6d9d7-tndfr"
Oct 30 21:44:51.356718 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:51.356687    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fbf428-59dbc6d9d7-tndfr"]
Oct 30 21:44:51.357343 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:44:51.357314    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c6d3630_8f1a_4298_ab10_d5619869e909.slice/crio-b0b8c5a9628608e3596b6e64da0be381e9fd60c3d63802491f16fe0c24bc43cb WatchSource:0}: Error finding container b0b8c5a9628608e3596b6e64da0be381e9fd60c3d63802491f16fe0c24bc43cb: Status 404 returned error can't find the container with id b0b8c5a9628608e3596b6e64da0be381e9fd60c3d63802491f16fe0c24bc43cb
Oct 30 21:44:52.270275 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:52.270234    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fbf428-59dbc6d9d7-tndfr" event={"ID":"4c6d3630-8f1a-4298-ab10-d5619869e909","Type":"ContainerStarted","Data":"04caebcc2bdb9e2a30094e841b5c2f07d0a482c6bc093561730f4a50f4bc4dac"}
Oct 30 21:44:52.270765 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:52.270302    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fbf428-59dbc6d9d7-tndfr" event={"ID":"4c6d3630-8f1a-4298-ab10-d5619869e909","Type":"ContainerStarted","Data":"b0b8c5a9628608e3596b6e64da0be381e9fd60c3d63802491f16fe0c24bc43cb"}
Oct 30 21:44:52.270765 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:52.270346    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fbf428-59dbc6d9d7-tndfr"
Oct 30 21:44:52.304163 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:52.304102    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fbf428-59dbc6d9d7-tndfr" podStartSLOduration=1.717200639 podStartE2EDuration="2.304086118s" totalImagesPullingTime="586.885479ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:44:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:44:52.294811882 +0000 UTC m=+1005.096624388" watchObservedRunningTime="2026-10-30 21:44:52.304086118 +0000 UTC m=+1005.105898581"
Oct 30 21:44:58.323385 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:44:58.319686    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fbf428-59dbc6d9d7-tndfr"
Oct 30 21:45:07.590331 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:45:07.590290    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-0c62ab-5d54447cf4-6c8wt"]
Oct 30 21:45:07.601414 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:45:07.601381    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-0c62ab-5d54447cf4-6c8wt"
Oct 30 21:45:07.604602 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:45:07.603966    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-0c62ab-5d54447cf4-6c8wt"]
Oct 30 21:45:07.607532 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:45:07.607505    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w5-0c62ab-serving-cert\""
Oct 30 21:45:07.718848 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:45:07.718814    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ce45345-9ad2-4e54-b19b-ffa9a73db1be-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w5-0c62ab-5d54447cf4-6c8wt\" (UID: \"6ce45345-9ad2-4e54-b19b-ffa9a73db1be\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-0c62ab-5d54447cf4-6c8wt"
Oct 30 21:45:07.719046 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:45:07.718973    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ce45345-9ad2-4e54-b19b-ffa9a73db1be-maas-api-tls\") pod \"maas-api-e2e-shared-b-w5-0c62ab-5d54447cf4-6c8wt\" (UID: \"6ce45345-9ad2-4e54-b19b-ffa9a73db1be\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-0c62ab-5d54447cf4-6c8wt"
Oct 30 21:45:07.719165 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:45:07.719146    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ce45345-9ad2-4e54-b19b-ffa9a73db1be-metrics-certs\") pod \"maas-api-e2e-shared-b-w5-0c62ab-5d54447cf4-6c8wt\" (UID: \"6ce45345-9ad2-4e54-b19b-ffa9a73db1be\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-0c62ab-5d54447cf4-6c8wt"
Oct 30 21:45:07.719306 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:45:07.719292    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jpf4\" (UniqueName: \"kubernetes.io/projected/6ce45345-9ad2-4e54-b19b-ffa9a73db1be-kube-api-access-9jpf4\") pod \"maas-api-e2e-shared-b-w5-0c62ab-5d54447cf4-6c8wt\" (UID: \"6ce45345-9ad2-4e54-b19b-ffa9a73db1be\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-0c62ab-5d54447cf4-6c8wt"
Oct 30 21:45:07.826680 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:45:07.822520    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ce45345-9ad2-4e54-b19b-ffa9a73db1be-metrics-certs\") pod \"maas-api-e2e-shared-b-w5-0c62ab-5d54447cf4-6c8wt\" (UID: \"6ce45345-9ad2-4e54-b19b-ffa9a73db1be\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-0c62ab-5d54447cf4-6c8wt"
Oct 30 21:45:07.826680 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:45:07.822698    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jpf4\" (UniqueName: \"kubernetes.io/projected/6ce45345-9ad2-4e54-b19b-ffa9a73db1be-kube-api-access-9jpf4\") pod \"maas-api-e2e-shared-b-w5-0c62ab-5d54447cf4-6c8wt\" (UID: \"6ce45345-9ad2-4e54-b19b-ffa9a73db1be\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-0c62ab-5d54447cf4-6c8wt"
Oct 30 21:45:07.826680 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:45:07.822824    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ce45345-9ad2-4e54-b19b-ffa9a73db1be-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w5-0c62ab-5d54447cf4-6c8wt\" (UID: \"6ce45345-9ad2-4e54-b19b-ffa9a73db1be\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-0c62ab-5d54447cf4-6c8wt"
Oct 30 21:45:07.826680 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:45:07.822926    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ce45345-9ad2-4e54-b19b-ffa9a73db1be-maas-api-tls\") pod \"maas-api-e2e-shared-b-w5-0c62ab-5d54447cf4-6c8wt\" (UID: \"6ce45345-9ad2-4e54-b19b-ffa9a73db1be\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-0c62ab-5d54447cf4-6c8wt"
Oct 30 21:45:07.839456 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:45:07.835690    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ce45345-9ad2-4e54-b19b-ffa9a73db1be-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w5-0c62ab-5d54447cf4-6c8wt\" (UID: \"6ce45345-9ad2-4e54-b19b-ffa9a73db1be\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-0c62ab-5d54447cf4-6c8wt"
Oct 30 21:45:07.849954 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:45:07.849146    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jpf4\" (UniqueName: \"kubernetes.io/projected/6ce45345-9ad2-4e54-b19b-ffa9a73db1be-kube-api-access-9jpf4\") pod \"maas-api-e2e-shared-b-w5-0c62ab-5d54447cf4-6c8wt\" (UID: \"6ce45345-9ad2-4e54-b19b-ffa9a73db1be\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-0c62ab-5d54447cf4-6c8wt"
Oct 30 21:45:07.849954 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:45:07.849924    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ce45345-9ad2-4e54-b19b-ffa9a73db1be-maas-api-tls\") pod \"maas-api-e2e-shared-b-w5-0c62ab-5d54447cf4-6c8wt\" (UID: \"6ce45345-9ad2-4e54-b19b-ffa9a73db1be\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-0c62ab-5d54447cf4-6c8wt"
Oct 30 21:45:07.854530 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:45:07.853942    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ce45345-9ad2-4e54-b19b-ffa9a73db1be-metrics-certs\") pod \"maas-api-e2e-shared-b-w5-0c62ab-5d54447cf4-6c8wt\" (UID: \"6ce45345-9ad2-4e54-b19b-ffa9a73db1be\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-0c62ab-5d54447cf4-6c8wt"
Oct 30 21:45:07.924638 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:45:07.924603    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-0c62ab-5d54447cf4-6c8wt"
Oct 30 21:45:08.259151 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:45:08.257760    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-0c62ab-5d54447cf4-6c8wt"]
Oct 30 21:45:08.262684 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:45:08.262654    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ce45345_9ad2_4e54_b19b_ffa9a73db1be.slice/crio-4dc350615a651ec827bba2752cf4e8cc3dd74faa073d5f9d069afb45dbf8dca2 WatchSource:0}: Error finding container 4dc350615a651ec827bba2752cf4e8cc3dd74faa073d5f9d069afb45dbf8dca2: Status 404 returned error can't find the container with id 4dc350615a651ec827bba2752cf4e8cc3dd74faa073d5f9d069afb45dbf8dca2
Oct 30 21:45:08.462073 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:45:08.462035    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-0c62ab-5d54447cf4-6c8wt" event={"ID":"6ce45345-9ad2-4e54-b19b-ffa9a73db1be","Type":"ContainerStarted","Data":"4dc350615a651ec827bba2752cf4e8cc3dd74faa073d5f9d069afb45dbf8dca2"}
Oct 30 21:45:09.490830 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:45:09.490789    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-0c62ab-5d54447cf4-6c8wt" event={"ID":"6ce45345-9ad2-4e54-b19b-ffa9a73db1be","Type":"ContainerStarted","Data":"9d8fcddf56ea2414d0625b1298728e94385f773b114ac206d8144c5b05976526"}
Oct 30 21:45:09.491429 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:45:09.490889    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-0c62ab-5d54447cf4-6c8wt"
Oct 30 21:45:09.542562 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:45:09.542511    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-0c62ab-5d54447cf4-6c8wt" podStartSLOduration=2.077236747 podStartE2EDuration="2.542497201s" totalImagesPullingTime="465.260454ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:45:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:45:09.541849923 +0000 UTC m=+1022.343662398" watchObservedRunningTime="2026-10-30 21:45:09.542497201 +0000 UTC m=+1022.344309665"
Oct 30 21:45:13.146325 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:45:13.146286    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 21:45:15.553421 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:45:15.548516    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-0c62ab-5d54447cf4-6c8wt"
Oct 30 21:45:18.490899 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:45:18.490686    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 21:45:48.347621 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:45:48.347576    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 21:45:56.100718 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:45:56.100681    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 21:46:18.353990 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:46:18.353951    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 21:46:26.716806 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:46:26.716771    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7386906d-5b9645d4df-cwtsd"]
Oct 30 21:46:26.723043 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:46:26.723018    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7386906d-5b9645d4df-cwtsd"
Oct 30 21:46:26.758790 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:46:26.758760    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-7386906d-serving-cert\""
Oct 30 21:46:26.759947 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:46:26.759894    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7386906d-5b9645d4df-cwtsd"]
Oct 30 21:46:26.792905 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:46:26.792867    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a12e67c9-820d-48a8-b5e0-c9656ee8d199-metrics-certs\") pod \"maas-api-e2e-mt-7386906d-5b9645d4df-cwtsd\" (UID: \"a12e67c9-820d-48a8-b5e0-c9656ee8d199\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7386906d-5b9645d4df-cwtsd"
Oct 30 21:46:26.793120 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:46:26.793008    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a12e67c9-820d-48a8-b5e0-c9656ee8d199-openshift-service-ca\") pod \"maas-api-e2e-mt-7386906d-5b9645d4df-cwtsd\" (UID: \"a12e67c9-820d-48a8-b5e0-c9656ee8d199\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7386906d-5b9645d4df-cwtsd"
Oct 30 21:46:26.793256 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:46:26.793240    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8x9qh\" (UniqueName: \"kubernetes.io/projected/a12e67c9-820d-48a8-b5e0-c9656ee8d199-kube-api-access-8x9qh\") pod \"maas-api-e2e-mt-7386906d-5b9645d4df-cwtsd\" (UID: \"a12e67c9-820d-48a8-b5e0-c9656ee8d199\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7386906d-5b9645d4df-cwtsd"
Oct 30 21:46:26.793326 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:46:26.793270    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a12e67c9-820d-48a8-b5e0-c9656ee8d199-maas-api-tls\") pod \"maas-api-e2e-mt-7386906d-5b9645d4df-cwtsd\" (UID: \"a12e67c9-820d-48a8-b5e0-c9656ee8d199\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7386906d-5b9645d4df-cwtsd"
Oct 30 21:46:26.894924 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:46:26.894886    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8x9qh\" (UniqueName: \"kubernetes.io/projected/a12e67c9-820d-48a8-b5e0-c9656ee8d199-kube-api-access-8x9qh\") pod \"maas-api-e2e-mt-7386906d-5b9645d4df-cwtsd\" (UID: \"a12e67c9-820d-48a8-b5e0-c9656ee8d199\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7386906d-5b9645d4df-cwtsd"
Oct 30 21:46:26.895124 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:46:26.894947    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a12e67c9-820d-48a8-b5e0-c9656ee8d199-maas-api-tls\") pod \"maas-api-e2e-mt-7386906d-5b9645d4df-cwtsd\" (UID: \"a12e67c9-820d-48a8-b5e0-c9656ee8d199\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7386906d-5b9645d4df-cwtsd"
Oct 30 21:46:26.895200 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:46:26.895087    5032 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-7386906d-serving-cert: secret "maas-api-e2e-mt-7386906d-serving-cert" not found
Oct 30 21:46:26.895200 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:46:26.895167    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a12e67c9-820d-48a8-b5e0-c9656ee8d199-metrics-certs\") pod \"maas-api-e2e-mt-7386906d-5b9645d4df-cwtsd\" (UID: \"a12e67c9-820d-48a8-b5e0-c9656ee8d199\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7386906d-5b9645d4df-cwtsd"
Oct 30 21:46:26.895318 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:46:26.895224    5032 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a12e67c9-820d-48a8-b5e0-c9656ee8d199-maas-api-tls podName:a12e67c9-820d-48a8-b5e0-c9656ee8d199 nodeName:}" failed. No retries permitted until 2026-10-30 21:46:27.395203065 +0000 UTC m=+1100.197015525 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a12e67c9-820d-48a8-b5e0-c9656ee8d199-maas-api-tls") pod "maas-api-e2e-mt-7386906d-5b9645d4df-cwtsd" (UID: "a12e67c9-820d-48a8-b5e0-c9656ee8d199") : secret "maas-api-e2e-mt-7386906d-serving-cert" not found
Oct 30 21:46:26.895416 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:46:26.895355    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a12e67c9-820d-48a8-b5e0-c9656ee8d199-openshift-service-ca\") pod \"maas-api-e2e-mt-7386906d-5b9645d4df-cwtsd\" (UID: \"a12e67c9-820d-48a8-b5e0-c9656ee8d199\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7386906d-5b9645d4df-cwtsd"
Oct 30 21:46:26.896636 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:46:26.896564    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a12e67c9-820d-48a8-b5e0-c9656ee8d199-openshift-service-ca\") pod \"maas-api-e2e-mt-7386906d-5b9645d4df-cwtsd\" (UID: \"a12e67c9-820d-48a8-b5e0-c9656ee8d199\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7386906d-5b9645d4df-cwtsd"
Oct 30 21:46:26.899553 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:46:26.899462    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a12e67c9-820d-48a8-b5e0-c9656ee8d199-metrics-certs\") pod \"maas-api-e2e-mt-7386906d-5b9645d4df-cwtsd\" (UID: \"a12e67c9-820d-48a8-b5e0-c9656ee8d199\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7386906d-5b9645d4df-cwtsd"
Oct 30 21:46:26.927785 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:46:26.927746    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8x9qh\" (UniqueName: \"kubernetes.io/projected/a12e67c9-820d-48a8-b5e0-c9656ee8d199-kube-api-access-8x9qh\") pod \"maas-api-e2e-mt-7386906d-5b9645d4df-cwtsd\" (UID: \"a12e67c9-820d-48a8-b5e0-c9656ee8d199\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7386906d-5b9645d4df-cwtsd"
Oct 30 21:46:27.412384 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:46:27.406382    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a12e67c9-820d-48a8-b5e0-c9656ee8d199-maas-api-tls\") pod \"maas-api-e2e-mt-7386906d-5b9645d4df-cwtsd\" (UID: \"a12e67c9-820d-48a8-b5e0-c9656ee8d199\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7386906d-5b9645d4df-cwtsd"
Oct 30 21:46:27.446400 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:46:27.443818    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a12e67c9-820d-48a8-b5e0-c9656ee8d199-maas-api-tls\") pod \"maas-api-e2e-mt-7386906d-5b9645d4df-cwtsd\" (UID: \"a12e67c9-820d-48a8-b5e0-c9656ee8d199\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7386906d-5b9645d4df-cwtsd"
Oct 30 21:46:27.666696 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:46:27.666608    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7386906d-5b9645d4df-cwtsd"
Oct 30 21:46:27.971104 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:46:27.971069    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda12e67c9_820d_48a8_b5e0_c9656ee8d199.slice/crio-53324da251af317d8382afed8d5a0c90c6d44f01fbe86640c25794f5ce61dfc0 WatchSource:0}: Error finding container 53324da251af317d8382afed8d5a0c90c6d44f01fbe86640c25794f5ce61dfc0: Status 404 returned error can't find the container with id 53324da251af317d8382afed8d5a0c90c6d44f01fbe86640c25794f5ce61dfc0
Oct 30 21:46:27.972822 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:46:27.972798    5032 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 21:46:27.974671 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:46:27.974638    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7386906d-5b9645d4df-cwtsd"]
Oct 30 21:46:28.462329 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:46:28.462287    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7386906d-5b9645d4df-cwtsd" event={"ID":"a12e67c9-820d-48a8-b5e0-c9656ee8d199","Type":"ContainerStarted","Data":"a22043c396d0c96d7a9bec07ec2e85b4bec83dd385b1e8bef7e6381bb50fea07"}
Oct 30 21:46:28.462517 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:46:28.462356    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7386906d-5b9645d4df-cwtsd" event={"ID":"a12e67c9-820d-48a8-b5e0-c9656ee8d199","Type":"ContainerStarted","Data":"53324da251af317d8382afed8d5a0c90c6d44f01fbe86640c25794f5ce61dfc0"}
Oct 30 21:46:28.462517 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:46:28.462402    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7386906d-5b9645d4df-cwtsd"
Oct 30 21:46:28.485665 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:46:28.485609    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7386906d-5b9645d4df-cwtsd" podStartSLOduration=2.129550381 podStartE2EDuration="2.485593737s" totalImagesPullingTime="356.043356ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:46:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:46:28.482398505 +0000 UTC m=+1101.284210969" watchObservedRunningTime="2026-10-30 21:46:28.485593737 +0000 UTC m=+1101.287406241"
Oct 30 21:46:34.478460 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:46:34.478426    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7386906d-5b9645d4df-cwtsd"
Oct 30 21:46:48.369091 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:46:48.369006    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 21:47:04.035692 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:04.035662    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 21:47:04.390513 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:04.390432    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-aad4dd-679948c8f9-bwf4l"]
Oct 30 21:47:04.403821 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:04.402648    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-aad4dd-679948c8f9-bwf4l"]
Oct 30 21:47:04.403821 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:04.402804    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-aad4dd-679948c8f9-bwf4l"
Oct 30 21:47:04.412100 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:04.412073    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w4-aad4dd-serving-cert\""
Oct 30 21:47:04.585108 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:04.585069    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2d543f80-60d7-4c58-ae3d-667f2e577282-maas-api-tls\") pod \"maas-api-e2e-shared-b-w4-aad4dd-679948c8f9-bwf4l\" (UID: \"2d543f80-60d7-4c58-ae3d-667f2e577282\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-aad4dd-679948c8f9-bwf4l"
Oct 30 21:47:04.585286 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:04.585237    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d543f80-60d7-4c58-ae3d-667f2e577282-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w4-aad4dd-679948c8f9-bwf4l\" (UID: \"2d543f80-60d7-4c58-ae3d-667f2e577282\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-aad4dd-679948c8f9-bwf4l"
Oct 30 21:47:04.585397 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:04.585372    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbbjz\" (UniqueName: \"kubernetes.io/projected/2d543f80-60d7-4c58-ae3d-667f2e577282-kube-api-access-bbbjz\") pod \"maas-api-e2e-shared-b-w4-aad4dd-679948c8f9-bwf4l\" (UID: \"2d543f80-60d7-4c58-ae3d-667f2e577282\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-aad4dd-679948c8f9-bwf4l"
Oct 30 21:47:04.585575 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:04.585545    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2d543f80-60d7-4c58-ae3d-667f2e577282-metrics-certs\") pod \"maas-api-e2e-shared-b-w4-aad4dd-679948c8f9-bwf4l\" (UID: \"2d543f80-60d7-4c58-ae3d-667f2e577282\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-aad4dd-679948c8f9-bwf4l"
Oct 30 21:47:04.687584 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:04.687484    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2d543f80-60d7-4c58-ae3d-667f2e577282-maas-api-tls\") pod \"maas-api-e2e-shared-b-w4-aad4dd-679948c8f9-bwf4l\" (UID: \"2d543f80-60d7-4c58-ae3d-667f2e577282\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-aad4dd-679948c8f9-bwf4l"
Oct 30 21:47:04.687745 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:04.687621    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d543f80-60d7-4c58-ae3d-667f2e577282-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w4-aad4dd-679948c8f9-bwf4l\" (UID: \"2d543f80-60d7-4c58-ae3d-667f2e577282\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-aad4dd-679948c8f9-bwf4l"
Oct 30 21:47:04.687793 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:04.687739    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbbjz\" (UniqueName: \"kubernetes.io/projected/2d543f80-60d7-4c58-ae3d-667f2e577282-kube-api-access-bbbjz\") pod \"maas-api-e2e-shared-b-w4-aad4dd-679948c8f9-bwf4l\" (UID: \"2d543f80-60d7-4c58-ae3d-667f2e577282\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-aad4dd-679948c8f9-bwf4l"
Oct 30 21:47:04.687900 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:04.687873    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2d543f80-60d7-4c58-ae3d-667f2e577282-metrics-certs\") pod \"maas-api-e2e-shared-b-w4-aad4dd-679948c8f9-bwf4l\" (UID: \"2d543f80-60d7-4c58-ae3d-667f2e577282\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-aad4dd-679948c8f9-bwf4l"
Oct 30 21:47:04.688655 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:04.688634    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d543f80-60d7-4c58-ae3d-667f2e577282-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w4-aad4dd-679948c8f9-bwf4l\" (UID: \"2d543f80-60d7-4c58-ae3d-667f2e577282\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-aad4dd-679948c8f9-bwf4l"
Oct 30 21:47:04.690735 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:04.690714    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2d543f80-60d7-4c58-ae3d-667f2e577282-metrics-certs\") pod \"maas-api-e2e-shared-b-w4-aad4dd-679948c8f9-bwf4l\" (UID: \"2d543f80-60d7-4c58-ae3d-667f2e577282\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-aad4dd-679948c8f9-bwf4l"
Oct 30 21:47:04.690983 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:04.690963    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2d543f80-60d7-4c58-ae3d-667f2e577282-maas-api-tls\") pod \"maas-api-e2e-shared-b-w4-aad4dd-679948c8f9-bwf4l\" (UID: \"2d543f80-60d7-4c58-ae3d-667f2e577282\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-aad4dd-679948c8f9-bwf4l"
Oct 30 21:47:04.700766 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:04.700738    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbbjz\" (UniqueName: \"kubernetes.io/projected/2d543f80-60d7-4c58-ae3d-667f2e577282-kube-api-access-bbbjz\") pod \"maas-api-e2e-shared-b-w4-aad4dd-679948c8f9-bwf4l\" (UID: \"2d543f80-60d7-4c58-ae3d-667f2e577282\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-aad4dd-679948c8f9-bwf4l"
Oct 30 21:47:04.760036 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:04.759995    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-aad4dd-679948c8f9-bwf4l"
Oct 30 21:47:05.100747 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:05.100604    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-aad4dd-679948c8f9-bwf4l"]
Oct 30 21:47:05.888912 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:05.888880    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-aad4dd-679948c8f9-bwf4l" event={"ID":"2d543f80-60d7-4c58-ae3d-667f2e577282","Type":"ContainerStarted","Data":"34290ab3c027e8fe6ae94a2720c387c62af229e217cc7adad72af0748cb713f1"}
Oct 30 21:47:05.889135 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:05.888934    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-aad4dd-679948c8f9-bwf4l" event={"ID":"2d543f80-60d7-4c58-ae3d-667f2e577282","Type":"ContainerStarted","Data":"eaca0eab88d721808069f449465b07901c521f699ca62e3b26790a4220e28dd3"}
Oct 30 21:47:05.889135 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:05.889026    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-aad4dd-679948c8f9-bwf4l"
Oct 30 21:47:05.915759 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:05.915704    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-aad4dd-679948c8f9-bwf4l" podStartSLOduration=1.36662469 podStartE2EDuration="1.91568916s" totalImagesPullingTime="549.06447ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:47:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:47:05.91387937 +0000 UTC m=+1138.715691833" watchObservedRunningTime="2026-10-30 21:47:05.91568916 +0000 UTC m=+1138.717501624"
Oct 30 21:47:11.578055 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:11.578021    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl"]
Oct 30 21:47:11.583643 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:11.583620    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl"
Oct 30 21:47:11.604784 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:11.604751    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-7386906d-openshift-default-dockercfg-ffr97\""
Oct 30 21:47:11.614502 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:11.614469    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl"]
Oct 30 21:47:11.658951 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:11.658914    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f924f351-d12c-438b-91e0-71f9a316b14d-istio-data\") pod \"e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl\" (UID: \"f924f351-d12c-438b-91e0-71f9a316b14d\") " pod="openshift-ingress/e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl"
Oct 30 21:47:11.659132 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:11.658959    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f924f351-d12c-438b-91e0-71f9a316b14d-istiod-ca-cert\") pod \"e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl\" (UID: \"f924f351-d12c-438b-91e0-71f9a316b14d\") " pod="openshift-ingress/e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl"
Oct 30 21:47:11.659191 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:11.659145    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmn8w\" (UniqueName: \"kubernetes.io/projected/f924f351-d12c-438b-91e0-71f9a316b14d-kube-api-access-lmn8w\") pod \"e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl\" (UID: \"f924f351-d12c-438b-91e0-71f9a316b14d\") " pod="openshift-ingress/e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl"
Oct 30 21:47:11.659266 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:11.659248    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f924f351-d12c-438b-91e0-71f9a316b14d-istio-token\") pod \"e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl\" (UID: \"f924f351-d12c-438b-91e0-71f9a316b14d\") " pod="openshift-ingress/e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl"
Oct 30 21:47:11.659333 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:11.659289    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f924f351-d12c-438b-91e0-71f9a316b14d-credential-socket\") pod \"e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl\" (UID: \"f924f351-d12c-438b-91e0-71f9a316b14d\") " pod="openshift-ingress/e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl"
Oct 30 21:47:11.659490 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:11.659465    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f924f351-d12c-438b-91e0-71f9a316b14d-istio-podinfo\") pod \"e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl\" (UID: \"f924f351-d12c-438b-91e0-71f9a316b14d\") " pod="openshift-ingress/e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl"
Oct 30 21:47:11.659570 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:11.659555    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f924f351-d12c-438b-91e0-71f9a316b14d-workload-certs\") pod \"e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl\" (UID: \"f924f351-d12c-438b-91e0-71f9a316b14d\") " pod="openshift-ingress/e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl"
Oct 30 21:47:11.659606 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:11.659579    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f924f351-d12c-438b-91e0-71f9a316b14d-workload-socket\") pod \"e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl\" (UID: \"f924f351-d12c-438b-91e0-71f9a316b14d\") " pod="openshift-ingress/e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl"
Oct 30 21:47:11.659606 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:11.659601    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f924f351-d12c-438b-91e0-71f9a316b14d-istio-envoy\") pod \"e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl\" (UID: \"f924f351-d12c-438b-91e0-71f9a316b14d\") " pod="openshift-ingress/e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl"
Oct 30 21:47:11.761672 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:11.761639    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f924f351-d12c-438b-91e0-71f9a316b14d-istio-data\") pod \"e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl\" (UID: \"f924f351-d12c-438b-91e0-71f9a316b14d\") " pod="openshift-ingress/e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl"
Oct 30 21:47:11.761861 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:11.761681    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f924f351-d12c-438b-91e0-71f9a316b14d-istiod-ca-cert\") pod \"e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl\" (UID: \"f924f351-d12c-438b-91e0-71f9a316b14d\") " pod="openshift-ingress/e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl"
Oct 30 21:47:11.761861 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:11.761794    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmn8w\" (UniqueName: \"kubernetes.io/projected/f924f351-d12c-438b-91e0-71f9a316b14d-kube-api-access-lmn8w\") pod \"e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl\" (UID: \"f924f351-d12c-438b-91e0-71f9a316b14d\") " pod="openshift-ingress/e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl"
Oct 30 21:47:11.761956 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:11.761890    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f924f351-d12c-438b-91e0-71f9a316b14d-istio-token\") pod \"e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl\" (UID: \"f924f351-d12c-438b-91e0-71f9a316b14d\") " pod="openshift-ingress/e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl"
Oct 30 21:47:11.761956 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:11.761941    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f924f351-d12c-438b-91e0-71f9a316b14d-credential-socket\") pod \"e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl\" (UID: \"f924f351-d12c-438b-91e0-71f9a316b14d\") " pod="openshift-ingress/e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl"
Oct 30 21:47:11.762076 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:11.762028    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f924f351-d12c-438b-91e0-71f9a316b14d-istio-podinfo\") pod \"e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl\" (UID: \"f924f351-d12c-438b-91e0-71f9a316b14d\") " pod="openshift-ingress/e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl"
Oct 30 21:47:11.762151 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:11.762109    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f924f351-d12c-438b-91e0-71f9a316b14d-istio-data\") pod \"e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl\" (UID: \"f924f351-d12c-438b-91e0-71f9a316b14d\") " pod="openshift-ingress/e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl"
Oct 30 21:47:11.762151 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:11.762119    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f924f351-d12c-438b-91e0-71f9a316b14d-workload-certs\") pod \"e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl\" (UID: \"f924f351-d12c-438b-91e0-71f9a316b14d\") " pod="openshift-ingress/e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl"
Oct 30 21:47:11.762232 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:11.762158    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f924f351-d12c-438b-91e0-71f9a316b14d-workload-socket\") pod \"e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl\" (UID: \"f924f351-d12c-438b-91e0-71f9a316b14d\") " pod="openshift-ingress/e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl"
Oct 30 21:47:11.762232 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:11.762193    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f924f351-d12c-438b-91e0-71f9a316b14d-istio-envoy\") pod \"e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl\" (UID: \"f924f351-d12c-438b-91e0-71f9a316b14d\") " pod="openshift-ingress/e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl"
Oct 30 21:47:11.762524 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:11.762481    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f924f351-d12c-438b-91e0-71f9a316b14d-workload-certs\") pod \"e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl\" (UID: \"f924f351-d12c-438b-91e0-71f9a316b14d\") " pod="openshift-ingress/e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl"
Oct 30 21:47:11.762639 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:11.762602    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f924f351-d12c-438b-91e0-71f9a316b14d-istiod-ca-cert\") pod \"e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl\" (UID: \"f924f351-d12c-438b-91e0-71f9a316b14d\") " pod="openshift-ingress/e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl"
Oct 30 21:47:11.762639 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:11.762615    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f924f351-d12c-438b-91e0-71f9a316b14d-workload-socket\") pod \"e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl\" (UID: \"f924f351-d12c-438b-91e0-71f9a316b14d\") " pod="openshift-ingress/e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl"
Oct 30 21:47:11.762755 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:11.762649    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f924f351-d12c-438b-91e0-71f9a316b14d-credential-socket\") pod \"e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl\" (UID: \"f924f351-d12c-438b-91e0-71f9a316b14d\") " pod="openshift-ingress/e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl"
Oct 30 21:47:11.765044 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:11.765024    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f924f351-d12c-438b-91e0-71f9a316b14d-istio-envoy\") pod \"e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl\" (UID: \"f924f351-d12c-438b-91e0-71f9a316b14d\") " pod="openshift-ingress/e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl"
Oct 30 21:47:11.765227 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:11.765210    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f924f351-d12c-438b-91e0-71f9a316b14d-istio-podinfo\") pod \"e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl\" (UID: \"f924f351-d12c-438b-91e0-71f9a316b14d\") " pod="openshift-ingress/e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl"
Oct 30 21:47:11.781837 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:11.781814    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f924f351-d12c-438b-91e0-71f9a316b14d-istio-token\") pod \"e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl\" (UID: \"f924f351-d12c-438b-91e0-71f9a316b14d\") " pod="openshift-ingress/e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl"
Oct 30 21:47:11.781951 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:11.781888    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmn8w\" (UniqueName: \"kubernetes.io/projected/f924f351-d12c-438b-91e0-71f9a316b14d-kube-api-access-lmn8w\") pod \"e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl\" (UID: \"f924f351-d12c-438b-91e0-71f9a316b14d\") " pod="openshift-ingress/e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl"
Oct 30 21:47:11.903617 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:11.903527    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl"
Oct 30 21:47:11.906270 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:11.906246    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-aad4dd-679948c8f9-bwf4l"
Oct 30 21:47:12.259633 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:12.258935    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:47:12.259633 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:12.259036    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:47:12.259633 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:12.259076    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:47:12.272408 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:12.268524    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl"]
Oct 30 21:47:12.992476 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:12.992431    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl" event={"ID":"f924f351-d12c-438b-91e0-71f9a316b14d","Type":"ContainerStarted","Data":"a81ba9892b2708c8e230ee9e9092ab020387104ab1959e3442d0a1bd10edc051"}
Oct 30 21:47:12.992900 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:12.992503    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl" event={"ID":"f924f351-d12c-438b-91e0-71f9a316b14d","Type":"ContainerStarted","Data":"f63624967e64559a4c43eba7a350f2b4ab1fdd6bc694c149c43771ceafbb55c6"}
Oct 30 21:47:13.031962 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:13.031891    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl" podStartSLOduration=2.031862686 podStartE2EDuration="2.031862686s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:47:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:47:13.023757966 +0000 UTC m=+1145.825570430" watchObservedRunningTime="2026-10-30 21:47:13.031862686 +0000 UTC m=+1145.833675151"
Oct 30 21:47:13.904046 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:13.904013    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl"
Oct 30 21:47:13.905774 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:13.905744    5032 patch_prober.go:29] interesting pod/e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.112:15021/healthz/ready\": dial tcp 10.132.0.112:15021: connect: connection refused" start-of-body=
Oct 30 21:47:13.905925 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:13.905817    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl" podUID="f924f351-d12c-438b-91e0-71f9a316b14d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.112:15021/healthz/ready\": dial tcp 10.132.0.112:15021: connect: connection refused"
Oct 30 21:47:14.904450 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:14.904404    5032 patch_prober.go:29] interesting pod/e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.112:15021/healthz/ready\": dial tcp 10.132.0.112:15021: connect: connection refused" start-of-body=
Oct 30 21:47:14.904866 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:14.904478    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl" podUID="f924f351-d12c-438b-91e0-71f9a316b14d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.112:15021/healthz/ready\": dial tcp 10.132.0.112:15021: connect: connection refused"
Oct 30 21:47:15.905034 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:15.904988    5032 patch_prober.go:29] interesting pod/e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.112:15021/healthz/ready\": dial tcp 10.132.0.112:15021: connect: connection refused" start-of-body=
Oct 30 21:47:15.905805 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:15.905053    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl" podUID="f924f351-d12c-438b-91e0-71f9a316b14d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.112:15021/healthz/ready\": dial tcp 10.132.0.112:15021: connect: connection refused"
Oct 30 21:47:16.914771 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:16.914736    5032 patch_prober.go:29] interesting pod/e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.112:15021/healthz/ready\": dial tcp 10.132.0.112:15021: connect: connection refused" start-of-body=
Oct 30 21:47:16.915346 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:16.915313    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl" podUID="f924f351-d12c-438b-91e0-71f9a316b14d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.112:15021/healthz/ready\": dial tcp 10.132.0.112:15021: connect: connection refused"
Oct 30 21:47:17.905266 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:17.905229    5032 patch_prober.go:29] interesting pod/e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.112:15021/healthz/ready\": dial tcp 10.132.0.112:15021: connect: connection refused" start-of-body=
Oct 30 21:47:17.905472 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:17.905283    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl" podUID="f924f351-d12c-438b-91e0-71f9a316b14d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.112:15021/healthz/ready\": dial tcp 10.132.0.112:15021: connect: connection refused"
Oct 30 21:47:18.365020 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:47:18.364984    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 21:47:18.986947 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:18.986915    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl"
Oct 30 21:47:18.987840 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:18.987819    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl"
Oct 30 21:47:18.988853 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:18.988828    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl"
Oct 30 21:47:29.449390 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:29.443055    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-cd3c4c-56cb554497-g2vk9"]
Oct 30 21:47:29.464670 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:29.462886    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-cd3c4c-56cb554497-g2vk9"]
Oct 30 21:47:29.464670 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:29.463038    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cd3c4c-56cb554497-g2vk9"
Oct 30 21:47:29.500527 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:29.500494    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-cd3c4c-serving-cert\""
Oct 30 21:47:29.548487 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:29.548448    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3193c8ab-5a3f-47cb-b558-0cf84125afa8-openshift-service-ca\") pod \"maas-api-e2e-api-a-cd3c4c-56cb554497-g2vk9\" (UID: \"3193c8ab-5a3f-47cb-b558-0cf84125afa8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cd3c4c-56cb554497-g2vk9"
Oct 30 21:47:29.548782 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:29.548759    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3193c8ab-5a3f-47cb-b558-0cf84125afa8-metrics-certs\") pod \"maas-api-e2e-api-a-cd3c4c-56cb554497-g2vk9\" (UID: \"3193c8ab-5a3f-47cb-b558-0cf84125afa8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cd3c4c-56cb554497-g2vk9"
Oct 30 21:47:29.550677 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:29.550647    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3193c8ab-5a3f-47cb-b558-0cf84125afa8-maas-api-tls\") pod \"maas-api-e2e-api-a-cd3c4c-56cb554497-g2vk9\" (UID: \"3193c8ab-5a3f-47cb-b558-0cf84125afa8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cd3c4c-56cb554497-g2vk9"
Oct 30 21:47:29.567680 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:29.567473    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7r48x\" (UniqueName: \"kubernetes.io/projected/3193c8ab-5a3f-47cb-b558-0cf84125afa8-kube-api-access-7r48x\") pod \"maas-api-e2e-api-a-cd3c4c-56cb554497-g2vk9\" (UID: \"3193c8ab-5a3f-47cb-b558-0cf84125afa8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cd3c4c-56cb554497-g2vk9"
Oct 30 21:47:29.677386 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:29.675349    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3193c8ab-5a3f-47cb-b558-0cf84125afa8-openshift-service-ca\") pod \"maas-api-e2e-api-a-cd3c4c-56cb554497-g2vk9\" (UID: \"3193c8ab-5a3f-47cb-b558-0cf84125afa8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cd3c4c-56cb554497-g2vk9"
Oct 30 21:47:29.677386 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:29.675442    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3193c8ab-5a3f-47cb-b558-0cf84125afa8-metrics-certs\") pod \"maas-api-e2e-api-a-cd3c4c-56cb554497-g2vk9\" (UID: \"3193c8ab-5a3f-47cb-b558-0cf84125afa8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cd3c4c-56cb554497-g2vk9"
Oct 30 21:47:29.677386 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:29.675676    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3193c8ab-5a3f-47cb-b558-0cf84125afa8-maas-api-tls\") pod \"maas-api-e2e-api-a-cd3c4c-56cb554497-g2vk9\" (UID: \"3193c8ab-5a3f-47cb-b558-0cf84125afa8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cd3c4c-56cb554497-g2vk9"
Oct 30 21:47:29.677386 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:29.675730    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7r48x\" (UniqueName: \"kubernetes.io/projected/3193c8ab-5a3f-47cb-b558-0cf84125afa8-kube-api-access-7r48x\") pod \"maas-api-e2e-api-a-cd3c4c-56cb554497-g2vk9\" (UID: \"3193c8ab-5a3f-47cb-b558-0cf84125afa8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cd3c4c-56cb554497-g2vk9"
Oct 30 21:47:29.690261 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:29.690210    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3193c8ab-5a3f-47cb-b558-0cf84125afa8-openshift-service-ca\") pod \"maas-api-e2e-api-a-cd3c4c-56cb554497-g2vk9\" (UID: \"3193c8ab-5a3f-47cb-b558-0cf84125afa8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cd3c4c-56cb554497-g2vk9"
Oct 30 21:47:29.691928 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:47:29.691858    5032 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-cd3c4c-serving-cert: secret "maas-api-e2e-api-a-cd3c4c-serving-cert" not found
Oct 30 21:47:29.692067 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:47:29.691953    5032 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3193c8ab-5a3f-47cb-b558-0cf84125afa8-maas-api-tls podName:3193c8ab-5a3f-47cb-b558-0cf84125afa8 nodeName:}" failed. No retries permitted until 2026-10-30 21:47:30.191931069 +0000 UTC m=+1162.993743525 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3193c8ab-5a3f-47cb-b558-0cf84125afa8-maas-api-tls") pod "maas-api-e2e-api-a-cd3c4c-56cb554497-g2vk9" (UID: "3193c8ab-5a3f-47cb-b558-0cf84125afa8") : secret "maas-api-e2e-api-a-cd3c4c-serving-cert" not found
Oct 30 21:47:29.695641 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:29.695549    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7r48x\" (UniqueName: \"kubernetes.io/projected/3193c8ab-5a3f-47cb-b558-0cf84125afa8-kube-api-access-7r48x\") pod \"maas-api-e2e-api-a-cd3c4c-56cb554497-g2vk9\" (UID: \"3193c8ab-5a3f-47cb-b558-0cf84125afa8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cd3c4c-56cb554497-g2vk9"
Oct 30 21:47:29.754760 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:29.752674    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3193c8ab-5a3f-47cb-b558-0cf84125afa8-metrics-certs\") pod \"maas-api-e2e-api-a-cd3c4c-56cb554497-g2vk9\" (UID: \"3193c8ab-5a3f-47cb-b558-0cf84125afa8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cd3c4c-56cb554497-g2vk9"
Oct 30 21:47:30.288801 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:30.288762    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3193c8ab-5a3f-47cb-b558-0cf84125afa8-maas-api-tls\") pod \"maas-api-e2e-api-a-cd3c4c-56cb554497-g2vk9\" (UID: \"3193c8ab-5a3f-47cb-b558-0cf84125afa8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cd3c4c-56cb554497-g2vk9"
Oct 30 21:47:30.291782 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:30.291749    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3193c8ab-5a3f-47cb-b558-0cf84125afa8-maas-api-tls\") pod \"maas-api-e2e-api-a-cd3c4c-56cb554497-g2vk9\" (UID: \"3193c8ab-5a3f-47cb-b558-0cf84125afa8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cd3c4c-56cb554497-g2vk9"
Oct 30 21:47:30.468893 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:30.468862    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cd3c4c-56cb554497-g2vk9"
Oct 30 21:47:30.797870 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:47:30.797836    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3193c8ab_5a3f_47cb_b558_0cf84125afa8.slice/crio-cfa7ff62409867e8382746c7caa4ae6e28d3a4e7e7bbeba4b2067c60ae3f96b6 WatchSource:0}: Error finding container cfa7ff62409867e8382746c7caa4ae6e28d3a4e7e7bbeba4b2067c60ae3f96b6: Status 404 returned error can't find the container with id cfa7ff62409867e8382746c7caa4ae6e28d3a4e7e7bbeba4b2067c60ae3f96b6
Oct 30 21:47:30.799025 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:30.799000    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-cd3c4c-56cb554497-g2vk9"]
Oct 30 21:47:31.441675 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:31.441568    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cd3c4c-56cb554497-g2vk9" event={"ID":"3193c8ab-5a3f-47cb-b558-0cf84125afa8","Type":"ContainerStarted","Data":"883c556bc1876f55e5c7d02ccb6f6bd74aa80b06737e5432f7c4009b2c557925"}
Oct 30 21:47:31.441675 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:31.441635    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cd3c4c-56cb554497-g2vk9" event={"ID":"3193c8ab-5a3f-47cb-b558-0cf84125afa8","Type":"ContainerStarted","Data":"cfa7ff62409867e8382746c7caa4ae6e28d3a4e7e7bbeba4b2067c60ae3f96b6"}
Oct 30 21:47:31.441675 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:31.441661    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cd3c4c-56cb554497-g2vk9"
Oct 30 21:47:31.465113 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:31.465062    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cd3c4c-56cb554497-g2vk9" podStartSLOduration=2.098805283 podStartE2EDuration="2.465046833s" totalImagesPullingTime="366.24155ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:47:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:47:31.462492824 +0000 UTC m=+1164.264305291" watchObservedRunningTime="2026-10-30 21:47:31.465046833 +0000 UTC m=+1164.266859325"
Oct 30 21:47:37.462400 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:37.462355    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cd3c4c-56cb554497-g2vk9"
Oct 30 21:47:48.453391 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:47:48.451686    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 21:47:50.251379 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:50.251071    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-dd5bf8-77fb44c66d-28b9d"]
Oct 30 21:47:50.292431 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:50.280765    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dd5bf8-77fb44c66d-28b9d"
Oct 30 21:47:50.292431 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:50.287844    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-dd5bf8-77fb44c66d-28b9d"]
Oct 30 21:47:50.297075 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:50.296988    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-dd5bf8-serving-cert\""
Oct 30 21:47:50.458052 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:50.458017    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f53fa2b2-206d-4342-9e26-edc43dbdf7b7-metrics-certs\") pod \"maas-api-e2e-api-a-dd5bf8-77fb44c66d-28b9d\" (UID: \"f53fa2b2-206d-4342-9e26-edc43dbdf7b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dd5bf8-77fb44c66d-28b9d"
Oct 30 21:47:50.458424 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:50.458403    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f53fa2b2-206d-4342-9e26-edc43dbdf7b7-openshift-service-ca\") pod \"maas-api-e2e-api-a-dd5bf8-77fb44c66d-28b9d\" (UID: \"f53fa2b2-206d-4342-9e26-edc43dbdf7b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dd5bf8-77fb44c66d-28b9d"
Oct 30 21:47:50.458795 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:50.458686    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpwls\" (UniqueName: \"kubernetes.io/projected/f53fa2b2-206d-4342-9e26-edc43dbdf7b7-kube-api-access-xpwls\") pod \"maas-api-e2e-api-a-dd5bf8-77fb44c66d-28b9d\" (UID: \"f53fa2b2-206d-4342-9e26-edc43dbdf7b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dd5bf8-77fb44c66d-28b9d"
Oct 30 21:47:50.458999 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:50.458977    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f53fa2b2-206d-4342-9e26-edc43dbdf7b7-maas-api-tls\") pod \"maas-api-e2e-api-a-dd5bf8-77fb44c66d-28b9d\" (UID: \"f53fa2b2-206d-4342-9e26-edc43dbdf7b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dd5bf8-77fb44c66d-28b9d"
Oct 30 21:47:50.561123 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:50.561033    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f53fa2b2-206d-4342-9e26-edc43dbdf7b7-metrics-certs\") pod \"maas-api-e2e-api-a-dd5bf8-77fb44c66d-28b9d\" (UID: \"f53fa2b2-206d-4342-9e26-edc43dbdf7b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dd5bf8-77fb44c66d-28b9d"
Oct 30 21:47:50.561373 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:50.561210    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f53fa2b2-206d-4342-9e26-edc43dbdf7b7-openshift-service-ca\") pod \"maas-api-e2e-api-a-dd5bf8-77fb44c66d-28b9d\" (UID: \"f53fa2b2-206d-4342-9e26-edc43dbdf7b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dd5bf8-77fb44c66d-28b9d"
Oct 30 21:47:50.561373 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:50.561318    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpwls\" (UniqueName: \"kubernetes.io/projected/f53fa2b2-206d-4342-9e26-edc43dbdf7b7-kube-api-access-xpwls\") pod \"maas-api-e2e-api-a-dd5bf8-77fb44c66d-28b9d\" (UID: \"f53fa2b2-206d-4342-9e26-edc43dbdf7b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dd5bf8-77fb44c66d-28b9d"
Oct 30 21:47:50.561500 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:50.561383    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f53fa2b2-206d-4342-9e26-edc43dbdf7b7-maas-api-tls\") pod \"maas-api-e2e-api-a-dd5bf8-77fb44c66d-28b9d\" (UID: \"f53fa2b2-206d-4342-9e26-edc43dbdf7b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dd5bf8-77fb44c66d-28b9d"
Oct 30 21:47:50.563985 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:50.563952    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f53fa2b2-206d-4342-9e26-edc43dbdf7b7-openshift-service-ca\") pod \"maas-api-e2e-api-a-dd5bf8-77fb44c66d-28b9d\" (UID: \"f53fa2b2-206d-4342-9e26-edc43dbdf7b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dd5bf8-77fb44c66d-28b9d"
Oct 30 21:47:50.568812 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:50.568783    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f53fa2b2-206d-4342-9e26-edc43dbdf7b7-maas-api-tls\") pod \"maas-api-e2e-api-a-dd5bf8-77fb44c66d-28b9d\" (UID: \"f53fa2b2-206d-4342-9e26-edc43dbdf7b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dd5bf8-77fb44c66d-28b9d"
Oct 30 21:47:50.568926 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:50.568830    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f53fa2b2-206d-4342-9e26-edc43dbdf7b7-metrics-certs\") pod \"maas-api-e2e-api-a-dd5bf8-77fb44c66d-28b9d\" (UID: \"f53fa2b2-206d-4342-9e26-edc43dbdf7b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dd5bf8-77fb44c66d-28b9d"
Oct 30 21:47:50.575595 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:50.575564    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpwls\" (UniqueName: \"kubernetes.io/projected/f53fa2b2-206d-4342-9e26-edc43dbdf7b7-kube-api-access-xpwls\") pod \"maas-api-e2e-api-a-dd5bf8-77fb44c66d-28b9d\" (UID: \"f53fa2b2-206d-4342-9e26-edc43dbdf7b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dd5bf8-77fb44c66d-28b9d"
Oct 30 21:47:50.678474 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:50.670036    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dd5bf8-77fb44c66d-28b9d"
Oct 30 21:47:51.230641 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:51.221084    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-dd5bf8-77fb44c66d-28b9d"]
Oct 30 21:47:51.895621 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:51.895574    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dd5bf8-77fb44c66d-28b9d" event={"ID":"f53fa2b2-206d-4342-9e26-edc43dbdf7b7","Type":"ContainerStarted","Data":"788fae58e6a7b4eeb910d6b169b504823af1640728201ef93ec09826fc6f84ef"}
Oct 30 21:47:51.896006 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:51.895646    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dd5bf8-77fb44c66d-28b9d" event={"ID":"f53fa2b2-206d-4342-9e26-edc43dbdf7b7","Type":"ContainerStarted","Data":"bc8769f5f383bbab5ac89375ad36b46ffc4091761ae247b85197b83f5bd2eca4"}
Oct 30 21:47:51.896975 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:51.896954    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dd5bf8-77fb44c66d-28b9d"
Oct 30 21:47:51.926340 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:51.926078    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dd5bf8-77fb44c66d-28b9d" podStartSLOduration=1.502479568 podStartE2EDuration="1.926061535s" totalImagesPullingTime="423.581967ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:47:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:47:51.919448714 +0000 UTC m=+1184.721261182" watchObservedRunningTime="2026-10-30 21:47:51.926061535 +0000 UTC m=+1184.727873992"
Oct 30 21:47:58.951727 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:47:58.951698    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dd5bf8-77fb44c66d-28b9d"
Oct 30 21:48:10.001654 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:10.001619    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-sklxd_67292bdc-7fe1-449d-af75-2baa90c16ea9/console-operator/1.log"
Oct 30 21:48:10.154748 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:10.154676    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-sklxd_67292bdc-7fe1-449d-af75-2baa90c16ea9/console-operator/1.log"
Oct 30 21:48:18.354211 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:48:18.354080    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 21:48:24.546264 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:24.546230    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv"]
Oct 30 21:48:24.546806 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:24.546485    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv" podUID="30ea822a-85f1-4db5-a3a2-b8edd78d1200" containerName="istio-proxy" containerID="cri-o://ca5eafd98d90fdc3a6c340da0657c9c5ef55304bd7ea8e22277dc6bb83be92d0" gracePeriod=30
Oct 30 21:48:24.551652 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:24.551601    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj"]
Oct 30 21:48:24.563495 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:24.551887    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj" podUID="4577b3de-b9e4-47a7-87e0-bcba13b11bbe" containerName="istio-proxy" containerID="cri-o://e61b8147c0670a991ce18515008b69f74f5279f964883befb24bdc04a625a2b7" gracePeriod=30
Oct 30 21:48:29.914969 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:29.914942    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv"
Oct 30 21:48:29.971424 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:29.971399    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj"
Oct 30 21:48:29.987246 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:29.987215    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/30ea822a-85f1-4db5-a3a2-b8edd78d1200-kube-api-access-xhkx9\" (UniqueName: \"kubernetes.io/projected/30ea822a-85f1-4db5-a3a2-b8edd78d1200-kube-api-access-xhkx9\") pod \"30ea822a-85f1-4db5-a3a2-b8edd78d1200\" (UID: \"30ea822a-85f1-4db5-a3a2-b8edd78d1200\") "
Oct 30 21:48:29.987445 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:29.987350    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/30ea822a-85f1-4db5-a3a2-b8edd78d1200-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30ea822a-85f1-4db5-a3a2-b8edd78d1200-workload-socket\") pod \"30ea822a-85f1-4db5-a3a2-b8edd78d1200\" (UID: \"30ea822a-85f1-4db5-a3a2-b8edd78d1200\") "
Oct 30 21:48:29.987664 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:29.987531    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/30ea822a-85f1-4db5-a3a2-b8edd78d1200-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30ea822a-85f1-4db5-a3a2-b8edd78d1200-workload-certs\") pod \"30ea822a-85f1-4db5-a3a2-b8edd78d1200\" (UID: \"30ea822a-85f1-4db5-a3a2-b8edd78d1200\") "
Oct 30 21:48:29.987664 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:29.987605    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/30ea822a-85f1-4db5-a3a2-b8edd78d1200-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30ea822a-85f1-4db5-a3a2-b8edd78d1200-istio-podinfo\") pod \"30ea822a-85f1-4db5-a3a2-b8edd78d1200\" (UID: \"30ea822a-85f1-4db5-a3a2-b8edd78d1200\") "
Oct 30 21:48:29.987886 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:29.987722    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/30ea822a-85f1-4db5-a3a2-b8edd78d1200-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30ea822a-85f1-4db5-a3a2-b8edd78d1200-istio-data\") pod \"30ea822a-85f1-4db5-a3a2-b8edd78d1200\" (UID: \"30ea822a-85f1-4db5-a3a2-b8edd78d1200\") "
Oct 30 21:48:29.987886 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:29.987798    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/30ea822a-85f1-4db5-a3a2-b8edd78d1200-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30ea822a-85f1-4db5-a3a2-b8edd78d1200-credential-socket\") pod \"30ea822a-85f1-4db5-a3a2-b8edd78d1200\" (UID: \"30ea822a-85f1-4db5-a3a2-b8edd78d1200\") "
Oct 30 21:48:29.987886 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:29.987817    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/30ea822a-85f1-4db5-a3a2-b8edd78d1200-istio-token\" (UniqueName: \"kubernetes.io/projected/30ea822a-85f1-4db5-a3a2-b8edd78d1200-istio-token\") pod \"30ea822a-85f1-4db5-a3a2-b8edd78d1200\" (UID: \"30ea822a-85f1-4db5-a3a2-b8edd78d1200\") "
Oct 30 21:48:29.987886 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:29.987854    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/30ea822a-85f1-4db5-a3a2-b8edd78d1200-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30ea822a-85f1-4db5-a3a2-b8edd78d1200-istio-envoy\") pod \"30ea822a-85f1-4db5-a3a2-b8edd78d1200\" (UID: \"30ea822a-85f1-4db5-a3a2-b8edd78d1200\") "
Oct 30 21:48:29.988139 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:29.987893    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/30ea822a-85f1-4db5-a3a2-b8edd78d1200-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30ea822a-85f1-4db5-a3a2-b8edd78d1200-istiod-ca-cert\") pod \"30ea822a-85f1-4db5-a3a2-b8edd78d1200\" (UID: \"30ea822a-85f1-4db5-a3a2-b8edd78d1200\") "
Oct 30 21:48:29.988455 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:29.988351    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30ea822a-85f1-4db5-a3a2-b8edd78d1200-istio-data" pod "30ea822a-85f1-4db5-a3a2-b8edd78d1200" (UID: "30ea822a-85f1-4db5-a3a2-b8edd78d1200"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:48:29.988702 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:29.988681    5032 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30ea822a-85f1-4db5-a3a2-b8edd78d1200-istio-data\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:48:29.988789 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:29.988707    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30ea822a-85f1-4db5-a3a2-b8edd78d1200-istiod-ca-cert" pod "30ea822a-85f1-4db5-a3a2-b8edd78d1200" (UID: "30ea822a-85f1-4db5-a3a2-b8edd78d1200"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:48:29.988929 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:29.988906    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30ea822a-85f1-4db5-a3a2-b8edd78d1200-workload-socket" pod "30ea822a-85f1-4db5-a3a2-b8edd78d1200" (UID: "30ea822a-85f1-4db5-a3a2-b8edd78d1200"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:48:29.988993 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:29.988957    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30ea822a-85f1-4db5-a3a2-b8edd78d1200-credential-socket" pod "30ea822a-85f1-4db5-a3a2-b8edd78d1200" (UID: "30ea822a-85f1-4db5-a3a2-b8edd78d1200"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:48:29.989133 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:29.989113    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30ea822a-85f1-4db5-a3a2-b8edd78d1200-workload-certs" pod "30ea822a-85f1-4db5-a3a2-b8edd78d1200" (UID: "30ea822a-85f1-4db5-a3a2-b8edd78d1200"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:48:29.989916 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:29.989887    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30ea822a-85f1-4db5-a3a2-b8edd78d1200-kube-api-access-xhkx9" pod "30ea822a-85f1-4db5-a3a2-b8edd78d1200" (UID: "30ea822a-85f1-4db5-a3a2-b8edd78d1200"). InnerVolumeSpecName "kube-api-access-xhkx9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:48:29.991414 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:29.991391    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/30ea822a-85f1-4db5-a3a2-b8edd78d1200-istio-podinfo" pod "30ea822a-85f1-4db5-a3a2-b8edd78d1200" (UID: "30ea822a-85f1-4db5-a3a2-b8edd78d1200"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:48:29.991504 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:29.991476    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30ea822a-85f1-4db5-a3a2-b8edd78d1200-istio-token" pod "30ea822a-85f1-4db5-a3a2-b8edd78d1200" (UID: "30ea822a-85f1-4db5-a3a2-b8edd78d1200"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:48:29.991504 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:29.991486    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30ea822a-85f1-4db5-a3a2-b8edd78d1200-istio-envoy" pod "30ea822a-85f1-4db5-a3a2-b8edd78d1200" (UID: "30ea822a-85f1-4db5-a3a2-b8edd78d1200"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:48:30.089304 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:30.089224    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4577b3de-b9e4-47a7-87e0-bcba13b11bbe-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4577b3de-b9e4-47a7-87e0-bcba13b11bbe-istio-envoy\") pod \"4577b3de-b9e4-47a7-87e0-bcba13b11bbe\" (UID: \"4577b3de-b9e4-47a7-87e0-bcba13b11bbe\") "
Oct 30 21:48:30.089304 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:30.089278    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4577b3de-b9e4-47a7-87e0-bcba13b11bbe-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4577b3de-b9e4-47a7-87e0-bcba13b11bbe-workload-certs\") pod \"4577b3de-b9e4-47a7-87e0-bcba13b11bbe\" (UID: \"4577b3de-b9e4-47a7-87e0-bcba13b11bbe\") "
Oct 30 21:48:30.089542 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:30.089306    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4577b3de-b9e4-47a7-87e0-bcba13b11bbe-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4577b3de-b9e4-47a7-87e0-bcba13b11bbe-istio-data\") pod \"4577b3de-b9e4-47a7-87e0-bcba13b11bbe\" (UID: \"4577b3de-b9e4-47a7-87e0-bcba13b11bbe\") "
Oct 30 21:48:30.089542 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:30.089333    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4577b3de-b9e4-47a7-87e0-bcba13b11bbe-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4577b3de-b9e4-47a7-87e0-bcba13b11bbe-workload-socket\") pod \"4577b3de-b9e4-47a7-87e0-bcba13b11bbe\" (UID: \"4577b3de-b9e4-47a7-87e0-bcba13b11bbe\") "
Oct 30 21:48:30.089542 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:30.089450    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4577b3de-b9e4-47a7-87e0-bcba13b11bbe-kube-api-access-nwr7d\" (UniqueName: \"kubernetes.io/projected/4577b3de-b9e4-47a7-87e0-bcba13b11bbe-kube-api-access-nwr7d\") pod \"4577b3de-b9e4-47a7-87e0-bcba13b11bbe\" (UID: \"4577b3de-b9e4-47a7-87e0-bcba13b11bbe\") "
Oct 30 21:48:30.089698 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:30.089575    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4577b3de-b9e4-47a7-87e0-bcba13b11bbe-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4577b3de-b9e4-47a7-87e0-bcba13b11bbe-istio-podinfo\") pod \"4577b3de-b9e4-47a7-87e0-bcba13b11bbe\" (UID: \"4577b3de-b9e4-47a7-87e0-bcba13b11bbe\") "
Oct 30 21:48:30.089698 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:30.089616    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4577b3de-b9e4-47a7-87e0-bcba13b11bbe-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4577b3de-b9e4-47a7-87e0-bcba13b11bbe-credential-socket\") pod \"4577b3de-b9e4-47a7-87e0-bcba13b11bbe\" (UID: \"4577b3de-b9e4-47a7-87e0-bcba13b11bbe\") "
Oct 30 21:48:30.089698 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:30.089636    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4577b3de-b9e4-47a7-87e0-bcba13b11bbe-istio-data" pod "4577b3de-b9e4-47a7-87e0-bcba13b11bbe" (UID: "4577b3de-b9e4-47a7-87e0-bcba13b11bbe"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:48:30.089698 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:30.089659    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4577b3de-b9e4-47a7-87e0-bcba13b11bbe-workload-certs" pod "4577b3de-b9e4-47a7-87e0-bcba13b11bbe" (UID: "4577b3de-b9e4-47a7-87e0-bcba13b11bbe"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:48:30.089911 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:30.089649    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4577b3de-b9e4-47a7-87e0-bcba13b11bbe-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4577b3de-b9e4-47a7-87e0-bcba13b11bbe-istiod-ca-cert\") pod \"4577b3de-b9e4-47a7-87e0-bcba13b11bbe\" (UID: \"4577b3de-b9e4-47a7-87e0-bcba13b11bbe\") "
Oct 30 21:48:30.089911 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:30.089745    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4577b3de-b9e4-47a7-87e0-bcba13b11bbe-workload-socket" pod "4577b3de-b9e4-47a7-87e0-bcba13b11bbe" (UID: "4577b3de-b9e4-47a7-87e0-bcba13b11bbe"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:48:30.089911 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:30.089781    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4577b3de-b9e4-47a7-87e0-bcba13b11bbe-istio-token\" (UniqueName: \"kubernetes.io/projected/4577b3de-b9e4-47a7-87e0-bcba13b11bbe-istio-token\") pod \"4577b3de-b9e4-47a7-87e0-bcba13b11bbe\" (UID: \"4577b3de-b9e4-47a7-87e0-bcba13b11bbe\") "
Oct 30 21:48:30.089911 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:30.089831    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4577b3de-b9e4-47a7-87e0-bcba13b11bbe-credential-socket" pod "4577b3de-b9e4-47a7-87e0-bcba13b11bbe" (UID: "4577b3de-b9e4-47a7-87e0-bcba13b11bbe"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:48:30.090117 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:30.089976    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4577b3de-b9e4-47a7-87e0-bcba13b11bbe-istiod-ca-cert" pod "4577b3de-b9e4-47a7-87e0-bcba13b11bbe" (UID: "4577b3de-b9e4-47a7-87e0-bcba13b11bbe"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:48:30.091398 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:30.091314    5032 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30ea822a-85f1-4db5-a3a2-b8edd78d1200-workload-certs\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:48:30.091398 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:30.091356    5032 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4577b3de-b9e4-47a7-87e0-bcba13b11bbe-credential-socket\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:48:30.091398 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:30.091386    5032 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4577b3de-b9e4-47a7-87e0-bcba13b11bbe-istiod-ca-cert\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:48:30.091732 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:30.091404    5032 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30ea822a-85f1-4db5-a3a2-b8edd78d1200-istio-podinfo\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:48:30.091732 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:30.091436    5032 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30ea822a-85f1-4db5-a3a2-b8edd78d1200-credential-socket\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:48:30.091732 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:30.091452    5032 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30ea822a-85f1-4db5-a3a2-b8edd78d1200-istio-token\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:48:30.091732 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:30.091466    5032 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30ea822a-85f1-4db5-a3a2-b8edd78d1200-istio-envoy\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:48:30.091732 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:30.091480    5032 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30ea822a-85f1-4db5-a3a2-b8edd78d1200-istiod-ca-cert\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:48:30.091732 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:30.091494    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xhkx9\" (UniqueName: \"kubernetes.io/projected/30ea822a-85f1-4db5-a3a2-b8edd78d1200-kube-api-access-xhkx9\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:48:30.091732 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:30.091509    5032 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4577b3de-b9e4-47a7-87e0-bcba13b11bbe-workload-certs\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:48:30.091732 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:30.091522    5032 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4577b3de-b9e4-47a7-87e0-bcba13b11bbe-istio-data\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:48:30.091732 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:30.091536    5032 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4577b3de-b9e4-47a7-87e0-bcba13b11bbe-workload-socket\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:48:30.091732 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:30.091555    5032 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30ea822a-85f1-4db5-a3a2-b8edd78d1200-workload-socket\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:48:30.092587 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:30.092559    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4577b3de-b9e4-47a7-87e0-bcba13b11bbe-istio-podinfo" pod "4577b3de-b9e4-47a7-87e0-bcba13b11bbe" (UID: "4577b3de-b9e4-47a7-87e0-bcba13b11bbe"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:48:30.092826 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:30.092795    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4577b3de-b9e4-47a7-87e0-bcba13b11bbe-istio-envoy" pod "4577b3de-b9e4-47a7-87e0-bcba13b11bbe" (UID: "4577b3de-b9e4-47a7-87e0-bcba13b11bbe"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:48:30.092914 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:30.092806    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4577b3de-b9e4-47a7-87e0-bcba13b11bbe-kube-api-access-nwr7d" pod "4577b3de-b9e4-47a7-87e0-bcba13b11bbe" (UID: "4577b3de-b9e4-47a7-87e0-bcba13b11bbe"). InnerVolumeSpecName "kube-api-access-nwr7d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:48:30.093471 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:30.093449    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4577b3de-b9e4-47a7-87e0-bcba13b11bbe-istio-token" pod "4577b3de-b9e4-47a7-87e0-bcba13b11bbe" (UID: "4577b3de-b9e4-47a7-87e0-bcba13b11bbe"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:48:30.208036 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:30.193658    5032 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4577b3de-b9e4-47a7-87e0-bcba13b11bbe-istio-podinfo\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:48:30.208036 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:30.193694    5032 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4577b3de-b9e4-47a7-87e0-bcba13b11bbe-istio-token\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:48:30.208036 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:30.193704    5032 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4577b3de-b9e4-47a7-87e0-bcba13b11bbe-istio-envoy\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:48:30.208036 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:30.193718    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nwr7d\" (UniqueName: \"kubernetes.io/projected/4577b3de-b9e4-47a7-87e0-bcba13b11bbe-kube-api-access-nwr7d\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:48:30.370637 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:30.370552    5032 generic.go:356] "Generic (PLEG): container finished" podID="30ea822a-85f1-4db5-a3a2-b8edd78d1200" containerID="ca5eafd98d90fdc3a6c340da0657c9c5ef55304bd7ea8e22277dc6bb83be92d0" exitCode=0
Oct 30 21:48:30.370637 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:30.370602    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv"
Oct 30 21:48:30.370843 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:30.370630    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv" event={"ID":"30ea822a-85f1-4db5-a3a2-b8edd78d1200","Type":"ContainerDied","Data":"ca5eafd98d90fdc3a6c340da0657c9c5ef55304bd7ea8e22277dc6bb83be92d0"}
Oct 30 21:48:30.370843 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:30.370686    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv" event={"ID":"30ea822a-85f1-4db5-a3a2-b8edd78d1200","Type":"ContainerDied","Data":"59e71aa45ab38a5e1a2b02fcba4ea14c4391970e9bed69c55de03cad879144c2"}
Oct 30 21:48:30.370843 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:30.370707    5032 scope.go:122] "RemoveContainer" containerID="ca5eafd98d90fdc3a6c340da0657c9c5ef55304bd7ea8e22277dc6bb83be92d0"
Oct 30 21:48:30.372224 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:30.372177    5032 generic.go:356] "Generic (PLEG): container finished" podID="4577b3de-b9e4-47a7-87e0-bcba13b11bbe" containerID="e61b8147c0670a991ce18515008b69f74f5279f964883befb24bdc04a625a2b7" exitCode=0
Oct 30 21:48:30.372340 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:30.372224    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj"
Oct 30 21:48:30.372340 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:30.372262    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj" event={"ID":"4577b3de-b9e4-47a7-87e0-bcba13b11bbe","Type":"ContainerDied","Data":"e61b8147c0670a991ce18515008b69f74f5279f964883befb24bdc04a625a2b7"}
Oct 30 21:48:30.372340 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:30.372313    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj" event={"ID":"4577b3de-b9e4-47a7-87e0-bcba13b11bbe","Type":"ContainerDied","Data":"57c7c539da9fc4d0ce4e272f8504d17cfc38051e78794d02ced2ca7bc8643e96"}
Oct 30 21:48:30.383666 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:30.383645    5032 scope.go:122] "RemoveContainer" containerID="ca5eafd98d90fdc3a6c340da0657c9c5ef55304bd7ea8e22277dc6bb83be92d0"
Oct 30 21:48:30.384497 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:48:30.384100    5032 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca5eafd98d90fdc3a6c340da0657c9c5ef55304bd7ea8e22277dc6bb83be92d0\": container with ID starting with ca5eafd98d90fdc3a6c340da0657c9c5ef55304bd7ea8e22277dc6bb83be92d0 not found: ID does not exist" containerID="ca5eafd98d90fdc3a6c340da0657c9c5ef55304bd7ea8e22277dc6bb83be92d0"
Oct 30 21:48:30.384497 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:30.384126    5032 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca5eafd98d90fdc3a6c340da0657c9c5ef55304bd7ea8e22277dc6bb83be92d0"} err="failed to get container status \"ca5eafd98d90fdc3a6c340da0657c9c5ef55304bd7ea8e22277dc6bb83be92d0\": rpc error: code = NotFound desc = could not find container \"ca5eafd98d90fdc3a6c340da0657c9c5ef55304bd7ea8e22277dc6bb83be92d0\": container with ID starting with ca5eafd98d90fdc3a6c340da0657c9c5ef55304bd7ea8e22277dc6bb83be92d0 not found: ID does not exist"
Oct 30 21:48:30.384497 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:30.384143    5032 scope.go:122] "RemoveContainer" containerID="e61b8147c0670a991ce18515008b69f74f5279f964883befb24bdc04a625a2b7"
Oct 30 21:48:30.396069 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:30.396045    5032 scope.go:122] "RemoveContainer" containerID="e61b8147c0670a991ce18515008b69f74f5279f964883befb24bdc04a625a2b7"
Oct 30 21:48:30.396449 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:48:30.396425    5032 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e61b8147c0670a991ce18515008b69f74f5279f964883befb24bdc04a625a2b7\": container with ID starting with e61b8147c0670a991ce18515008b69f74f5279f964883befb24bdc04a625a2b7 not found: ID does not exist" containerID="e61b8147c0670a991ce18515008b69f74f5279f964883befb24bdc04a625a2b7"
Oct 30 21:48:30.396546 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:30.396461    5032 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e61b8147c0670a991ce18515008b69f74f5279f964883befb24bdc04a625a2b7"} err="failed to get container status \"e61b8147c0670a991ce18515008b69f74f5279f964883befb24bdc04a625a2b7\": rpc error: code = NotFound desc = could not find container \"e61b8147c0670a991ce18515008b69f74f5279f964883befb24bdc04a625a2b7\": container with ID starting with e61b8147c0670a991ce18515008b69f74f5279f964883befb24bdc04a625a2b7 not found: ID does not exist"
Oct 30 21:48:30.412291 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:30.412263    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj"]
Oct 30 21:48:30.420515 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:30.420483    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-ls8dj"]
Oct 30 21:48:30.430050 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:30.430015    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv"]
Oct 30 21:48:30.438775 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:30.438746    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-jgmbv"]
Oct 30 21:48:31.349243 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:31.349210    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="30ea822a-85f1-4db5-a3a2-b8edd78d1200" path="/var/lib/kubelet/pods/30ea822a-85f1-4db5-a3a2-b8edd78d1200/volumes"
Oct 30 21:48:31.349798 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:31.349782    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4577b3de-b9e4-47a7-87e0-bcba13b11bbe" path="/var/lib/kubelet/pods/4577b3de-b9e4-47a7-87e0-bcba13b11bbe/volumes"
Oct 30 21:48:33.335904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:33.335862    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 21:48:37.365386 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:37.364642    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7386906d-5b9645d4df-cwtsd"]
Oct 30 21:48:37.365386 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:37.364882    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7386906d-5b9645d4df-cwtsd" podUID="a12e67c9-820d-48a8-b5e0-c9656ee8d199" containerName="maas-api" containerID="cri-o://a22043c396d0c96d7a9bec07ec2e85b4bec83dd385b1e8bef7e6381bb50fea07" gracePeriod=30
Oct 30 21:48:37.889382 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:37.887389    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7386906d-5b9645d4df-cwtsd"
Oct 30 21:48:38.018071 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:38.013222    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a12e67c9-820d-48a8-b5e0-c9656ee8d199-metrics-certs\" (UniqueName: \"kubernetes.io/secret/a12e67c9-820d-48a8-b5e0-c9656ee8d199-metrics-certs\") pod \"a12e67c9-820d-48a8-b5e0-c9656ee8d199\" (UID: \"a12e67c9-820d-48a8-b5e0-c9656ee8d199\") "
Oct 30 21:48:38.018071 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:38.013350    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a12e67c9-820d-48a8-b5e0-c9656ee8d199-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a12e67c9-820d-48a8-b5e0-c9656ee8d199-openshift-service-ca\") pod \"a12e67c9-820d-48a8-b5e0-c9656ee8d199\" (UID: \"a12e67c9-820d-48a8-b5e0-c9656ee8d199\") "
Oct 30 21:48:38.018071 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:38.013419    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a12e67c9-820d-48a8-b5e0-c9656ee8d199-kube-api-access-8x9qh\" (UniqueName: \"kubernetes.io/projected/a12e67c9-820d-48a8-b5e0-c9656ee8d199-kube-api-access-8x9qh\") pod \"a12e67c9-820d-48a8-b5e0-c9656ee8d199\" (UID: \"a12e67c9-820d-48a8-b5e0-c9656ee8d199\") "
Oct 30 21:48:38.018071 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:38.013580    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a12e67c9-820d-48a8-b5e0-c9656ee8d199-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a12e67c9-820d-48a8-b5e0-c9656ee8d199-maas-api-tls\") pod \"a12e67c9-820d-48a8-b5e0-c9656ee8d199\" (UID: \"a12e67c9-820d-48a8-b5e0-c9656ee8d199\") "
Oct 30 21:48:38.018071 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:38.015252    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a12e67c9-820d-48a8-b5e0-c9656ee8d199-openshift-service-ca" pod "a12e67c9-820d-48a8-b5e0-c9656ee8d199" (UID: "a12e67c9-820d-48a8-b5e0-c9656ee8d199"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:48:38.029125 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:38.027650    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a12e67c9-820d-48a8-b5e0-c9656ee8d199-kube-api-access-8x9qh" pod "a12e67c9-820d-48a8-b5e0-c9656ee8d199" (UID: "a12e67c9-820d-48a8-b5e0-c9656ee8d199"). InnerVolumeSpecName "kube-api-access-8x9qh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:48:38.029125 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:38.027740    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a12e67c9-820d-48a8-b5e0-c9656ee8d199-metrics-certs" pod "a12e67c9-820d-48a8-b5e0-c9656ee8d199" (UID: "a12e67c9-820d-48a8-b5e0-c9656ee8d199"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:48:38.034233 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:38.034199    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a12e67c9-820d-48a8-b5e0-c9656ee8d199-maas-api-tls" pod "a12e67c9-820d-48a8-b5e0-c9656ee8d199" (UID: "a12e67c9-820d-48a8-b5e0-c9656ee8d199"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:48:38.117288 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:38.117249    5032 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a12e67c9-820d-48a8-b5e0-c9656ee8d199-maas-api-tls\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:48:38.117288 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:38.117290    5032 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a12e67c9-820d-48a8-b5e0-c9656ee8d199-metrics-certs\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:48:38.117529 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:38.117306    5032 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a12e67c9-820d-48a8-b5e0-c9656ee8d199-openshift-service-ca\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:48:38.117529 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:38.117321    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8x9qh\" (UniqueName: \"kubernetes.io/projected/a12e67c9-820d-48a8-b5e0-c9656ee8d199-kube-api-access-8x9qh\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:48:38.468138 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:38.468105    5032 generic.go:356] "Generic (PLEG): container finished" podID="a12e67c9-820d-48a8-b5e0-c9656ee8d199" containerID="a22043c396d0c96d7a9bec07ec2e85b4bec83dd385b1e8bef7e6381bb50fea07" exitCode=0
Oct 30 21:48:38.468611 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:38.468150    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7386906d-5b9645d4df-cwtsd"
Oct 30 21:48:38.468611 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:38.468191    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7386906d-5b9645d4df-cwtsd" event={"ID":"a12e67c9-820d-48a8-b5e0-c9656ee8d199","Type":"ContainerDied","Data":"a22043c396d0c96d7a9bec07ec2e85b4bec83dd385b1e8bef7e6381bb50fea07"}
Oct 30 21:48:38.468611 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:38.468246    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7386906d-5b9645d4df-cwtsd" event={"ID":"a12e67c9-820d-48a8-b5e0-c9656ee8d199","Type":"ContainerDied","Data":"53324da251af317d8382afed8d5a0c90c6d44f01fbe86640c25794f5ce61dfc0"}
Oct 30 21:48:38.468611 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:38.468270    5032 scope.go:122] "RemoveContainer" containerID="a22043c396d0c96d7a9bec07ec2e85b4bec83dd385b1e8bef7e6381bb50fea07"
Oct 30 21:48:38.480865 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:38.480840    5032 scope.go:122] "RemoveContainer" containerID="a22043c396d0c96d7a9bec07ec2e85b4bec83dd385b1e8bef7e6381bb50fea07"
Oct 30 21:48:38.481179 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:48:38.481157    5032 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a22043c396d0c96d7a9bec07ec2e85b4bec83dd385b1e8bef7e6381bb50fea07\": container with ID starting with a22043c396d0c96d7a9bec07ec2e85b4bec83dd385b1e8bef7e6381bb50fea07 not found: ID does not exist" containerID="a22043c396d0c96d7a9bec07ec2e85b4bec83dd385b1e8bef7e6381bb50fea07"
Oct 30 21:48:38.481251 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:38.481194    5032 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a22043c396d0c96d7a9bec07ec2e85b4bec83dd385b1e8bef7e6381bb50fea07"} err="failed to get container status \"a22043c396d0c96d7a9bec07ec2e85b4bec83dd385b1e8bef7e6381bb50fea07\": rpc error: code = NotFound desc = could not find container \"a22043c396d0c96d7a9bec07ec2e85b4bec83dd385b1e8bef7e6381bb50fea07\": container with ID starting with a22043c396d0c96d7a9bec07ec2e85b4bec83dd385b1e8bef7e6381bb50fea07 not found: ID does not exist"
Oct 30 21:48:38.497604 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:38.497576    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7386906d-5b9645d4df-cwtsd"]
Oct 30 21:48:38.504934 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:38.504909    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7386906d-5b9645d4df-cwtsd"]
Oct 30 21:48:39.351645 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:39.351614    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a12e67c9-820d-48a8-b5e0-c9656ee8d199" path="/var/lib/kubelet/pods/a12e67c9-820d-48a8-b5e0-c9656ee8d199/volumes"
Oct 30 21:48:45.138600 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:45.138567    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl"]
Oct 30 21:48:45.149771 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:45.139246    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl" podUID="f924f351-d12c-438b-91e0-71f9a316b14d" containerName="istio-proxy" containerID="cri-o://a81ba9892b2708c8e230ee9e9092ab020387104ab1959e3442d0a1bd10edc051" gracePeriod=30
Oct 30 21:48:48.349688 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:48:48.349652    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 21:48:48.990668 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:48.990635    5032 patch_prober.go:29] interesting pod/e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:48:48.990841 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:48.990696    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl" podUID="f924f351-d12c-438b-91e0-71f9a316b14d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:48:50.465444 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:50.465418    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl"
Oct 30 21:48:50.515701 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:50.515666    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f924f351-d12c-438b-91e0-71f9a316b14d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f924f351-d12c-438b-91e0-71f9a316b14d-istio-envoy\") pod \"f924f351-d12c-438b-91e0-71f9a316b14d\" (UID: \"f924f351-d12c-438b-91e0-71f9a316b14d\") "
Oct 30 21:48:50.515894 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:50.515739    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f924f351-d12c-438b-91e0-71f9a316b14d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f924f351-d12c-438b-91e0-71f9a316b14d-istio-data\") pod \"f924f351-d12c-438b-91e0-71f9a316b14d\" (UID: \"f924f351-d12c-438b-91e0-71f9a316b14d\") "
Oct 30 21:48:50.515964 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:50.515938    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f924f351-d12c-438b-91e0-71f9a316b14d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f924f351-d12c-438b-91e0-71f9a316b14d-workload-certs\") pod \"f924f351-d12c-438b-91e0-71f9a316b14d\" (UID: \"f924f351-d12c-438b-91e0-71f9a316b14d\") "
Oct 30 21:48:50.516188 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:50.516127    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f924f351-d12c-438b-91e0-71f9a316b14d-istio-data" pod "f924f351-d12c-438b-91e0-71f9a316b14d" (UID: "f924f351-d12c-438b-91e0-71f9a316b14d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:48:50.516352 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:50.516208    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f924f351-d12c-438b-91e0-71f9a316b14d-workload-certs" pod "f924f351-d12c-438b-91e0-71f9a316b14d" (UID: "f924f351-d12c-438b-91e0-71f9a316b14d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:48:50.516352 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:50.516275    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f924f351-d12c-438b-91e0-71f9a316b14d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f924f351-d12c-438b-91e0-71f9a316b14d-workload-socket\") pod \"f924f351-d12c-438b-91e0-71f9a316b14d\" (UID: \"f924f351-d12c-438b-91e0-71f9a316b14d\") "
Oct 30 21:48:50.516565 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:50.516459    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f924f351-d12c-438b-91e0-71f9a316b14d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f924f351-d12c-438b-91e0-71f9a316b14d-istiod-ca-cert\") pod \"f924f351-d12c-438b-91e0-71f9a316b14d\" (UID: \"f924f351-d12c-438b-91e0-71f9a316b14d\") "
Oct 30 21:48:50.516565 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:50.516475    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f924f351-d12c-438b-91e0-71f9a316b14d-workload-socket" pod "f924f351-d12c-438b-91e0-71f9a316b14d" (UID: "f924f351-d12c-438b-91e0-71f9a316b14d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:48:50.516565 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:50.516493    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f924f351-d12c-438b-91e0-71f9a316b14d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f924f351-d12c-438b-91e0-71f9a316b14d-istio-podinfo\") pod \"f924f351-d12c-438b-91e0-71f9a316b14d\" (UID: \"f924f351-d12c-438b-91e0-71f9a316b14d\") "
Oct 30 21:48:50.516565 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:50.516545    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f924f351-d12c-438b-91e0-71f9a316b14d-kube-api-access-lmn8w\" (UniqueName: \"kubernetes.io/projected/f924f351-d12c-438b-91e0-71f9a316b14d-kube-api-access-lmn8w\") pod \"f924f351-d12c-438b-91e0-71f9a316b14d\" (UID: \"f924f351-d12c-438b-91e0-71f9a316b14d\") "
Oct 30 21:48:50.516765 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:50.516614    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f924f351-d12c-438b-91e0-71f9a316b14d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f924f351-d12c-438b-91e0-71f9a316b14d-credential-socket\") pod \"f924f351-d12c-438b-91e0-71f9a316b14d\" (UID: \"f924f351-d12c-438b-91e0-71f9a316b14d\") "
Oct 30 21:48:50.516765 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:50.516684    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f924f351-d12c-438b-91e0-71f9a316b14d-istio-token\" (UniqueName: \"kubernetes.io/projected/f924f351-d12c-438b-91e0-71f9a316b14d-istio-token\") pod \"f924f351-d12c-438b-91e0-71f9a316b14d\" (UID: \"f924f351-d12c-438b-91e0-71f9a316b14d\") "
Oct 30 21:48:50.517056 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:50.517034    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f924f351-d12c-438b-91e0-71f9a316b14d-istiod-ca-cert" pod "f924f351-d12c-438b-91e0-71f9a316b14d" (UID: "f924f351-d12c-438b-91e0-71f9a316b14d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:48:50.518245 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:50.517370    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f924f351-d12c-438b-91e0-71f9a316b14d-credential-socket" pod "f924f351-d12c-438b-91e0-71f9a316b14d" (UID: "f924f351-d12c-438b-91e0-71f9a316b14d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:48:50.518245 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:50.517645    5032 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f924f351-d12c-438b-91e0-71f9a316b14d-credential-socket\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:48:50.518245 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:50.517668    5032 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f924f351-d12c-438b-91e0-71f9a316b14d-istio-data\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:48:50.518245 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:50.517682    5032 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f924f351-d12c-438b-91e0-71f9a316b14d-workload-certs\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:48:50.518245 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:50.517697    5032 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f924f351-d12c-438b-91e0-71f9a316b14d-workload-socket\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:48:50.518245 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:50.517711    5032 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f924f351-d12c-438b-91e0-71f9a316b14d-istiod-ca-cert\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:48:50.519474 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:50.519446    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f924f351-d12c-438b-91e0-71f9a316b14d-istio-podinfo" pod "f924f351-d12c-438b-91e0-71f9a316b14d" (UID: "f924f351-d12c-438b-91e0-71f9a316b14d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:48:50.520777 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:50.520750    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f924f351-d12c-438b-91e0-71f9a316b14d-istio-envoy" pod "f924f351-d12c-438b-91e0-71f9a316b14d" (UID: "f924f351-d12c-438b-91e0-71f9a316b14d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:48:50.520879 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:50.520838    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f924f351-d12c-438b-91e0-71f9a316b14d-istio-token" pod "f924f351-d12c-438b-91e0-71f9a316b14d" (UID: "f924f351-d12c-438b-91e0-71f9a316b14d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:48:50.521077 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:50.521056    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f924f351-d12c-438b-91e0-71f9a316b14d-kube-api-access-lmn8w" pod "f924f351-d12c-438b-91e0-71f9a316b14d" (UID: "f924f351-d12c-438b-91e0-71f9a316b14d"). InnerVolumeSpecName "kube-api-access-lmn8w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:48:50.604420 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:50.604369    5032 generic.go:356] "Generic (PLEG): container finished" podID="f924f351-d12c-438b-91e0-71f9a316b14d" containerID="a81ba9892b2708c8e230ee9e9092ab020387104ab1959e3442d0a1bd10edc051" exitCode=0
Oct 30 21:48:50.604582 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:50.604439    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl"
Oct 30 21:48:50.604582 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:50.604455    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl" event={"ID":"f924f351-d12c-438b-91e0-71f9a316b14d","Type":"ContainerDied","Data":"a81ba9892b2708c8e230ee9e9092ab020387104ab1959e3442d0a1bd10edc051"}
Oct 30 21:48:50.604582 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:50.604560    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl" event={"ID":"f924f351-d12c-438b-91e0-71f9a316b14d","Type":"ContainerDied","Data":"f63624967e64559a4c43eba7a350f2b4ab1fdd6bc694c149c43771ceafbb55c6"}
Oct 30 21:48:50.604712 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:50.604594    5032 scope.go:122] "RemoveContainer" containerID="a81ba9892b2708c8e230ee9e9092ab020387104ab1959e3442d0a1bd10edc051"
Oct 30 21:48:50.620518 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:50.617510    5032 scope.go:122] "RemoveContainer" containerID="a81ba9892b2708c8e230ee9e9092ab020387104ab1959e3442d0a1bd10edc051"
Oct 30 21:48:50.620518 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:48:50.617849    5032 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a81ba9892b2708c8e230ee9e9092ab020387104ab1959e3442d0a1bd10edc051\": container with ID starting with a81ba9892b2708c8e230ee9e9092ab020387104ab1959e3442d0a1bd10edc051 not found: ID does not exist" containerID="a81ba9892b2708c8e230ee9e9092ab020387104ab1959e3442d0a1bd10edc051"
Oct 30 21:48:50.620518 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:50.617876    5032 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a81ba9892b2708c8e230ee9e9092ab020387104ab1959e3442d0a1bd10edc051"} err="failed to get container status \"a81ba9892b2708c8e230ee9e9092ab020387104ab1959e3442d0a1bd10edc051\": rpc error: code = NotFound desc = could not find container \"a81ba9892b2708c8e230ee9e9092ab020387104ab1959e3442d0a1bd10edc051\": container with ID starting with a81ba9892b2708c8e230ee9e9092ab020387104ab1959e3442d0a1bd10edc051 not found: ID does not exist"
Oct 30 21:48:50.620518 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:50.618977    5032 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f924f351-d12c-438b-91e0-71f9a316b14d-istio-envoy\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:48:50.620518 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:50.619002    5032 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f924f351-d12c-438b-91e0-71f9a316b14d-istio-podinfo\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:48:50.620518 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:50.619019    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lmn8w\" (UniqueName: \"kubernetes.io/projected/f924f351-d12c-438b-91e0-71f9a316b14d-kube-api-access-lmn8w\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:48:50.620518 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:50.619033    5032 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f924f351-d12c-438b-91e0-71f9a316b14d-istio-token\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:48:50.640323 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:50.640288    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl"]
Oct 30 21:48:50.646445 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:50.646417    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-7386906d-openshift-default-77bf4c6869-m2rvl"]
Oct 30 21:48:51.353870 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:51.353838    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f924f351-d12c-438b-91e0-71f9a316b14d" path="/var/lib/kubelet/pods/f924f351-d12c-438b-91e0-71f9a316b14d/volumes"
Oct 30 21:48:55.471691 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:48:55.471659    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw"]
Oct 30 21:49:00.958013 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:00.957959    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-7dfff0-579b4f5ddc-kxqlf"]
Oct 30 21:49:05.960982 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:05.960886    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-7dfff0-579b4f5ddc-kxqlf" podUID="b53e75a7-c215-4fc9-a64b-8720382c6acd" containerName="payload-pre-processing" containerID="cri-o://65496a6fcedbf59644f1d4de477af46439ec1b1bc1e7a1c567c89211aa24d2fe" gracePeriod=25
Oct 30 21:49:07.412780 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:07.412747    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-7dfff0-579b4f5ddc-kxqlf" podUID="b53e75a7-c215-4fc9-a64b-8720382c6acd" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:49:10.494804 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:10.494753    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw" podUID="2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c" containerName="main" containerID="cri-o://51dc5925f7d3db9b4cf18e7e05f841ea126c46e10ceb6d77d137207f2c52ae78" gracePeriod=45
Oct 30 21:49:10.880041 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:10.880004    5032 generic.go:356] "Generic (PLEG): container finished" podID="2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c" containerID="51dc5925f7d3db9b4cf18e7e05f841ea126c46e10ceb6d77d137207f2c52ae78" exitCode=0
Oct 30 21:49:10.880213 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:10.880054    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw" event={"ID":"2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c","Type":"ContainerDied","Data":"51dc5925f7d3db9b4cf18e7e05f841ea126c46e10ceb6d77d137207f2c52ae78"}
Oct 30 21:49:10.880213 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:10.880104    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw" event={"ID":"2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c","Type":"ContainerDied","Data":"df140a9c931051d41094f139b02ca4a937a7ea41fa55f227332720391ecd1280"}
Oct 30 21:49:10.880213 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:10.880120    5032 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="df140a9c931051d41094f139b02ca4a937a7ea41fa55f227332720391ecd1280"
Oct 30 21:49:10.888783 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:10.888761    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw"
Oct 30 21:49:10.936431 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:10.936378    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c-dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c-dshm\") pod \"2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c\" (UID: \"2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c\") "
Oct 30 21:49:10.936595 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:10.936441    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c-kube-api-access-hd2pl\" (UniqueName: \"kubernetes.io/projected/2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c-kube-api-access-hd2pl\") pod \"2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c\" (UID: \"2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c\") "
Oct 30 21:49:10.936595 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:10.936509    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c-tmp-dir\") pod \"2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c\" (UID: \"2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c\") "
Oct 30 21:49:10.936911 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:10.936881    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c-tmp-dir" pod "2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c" (UID: "2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:10.937009 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:10.936969    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c-home\" (UniqueName: \"kubernetes.io/empty-dir/2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c-home\") pod \"2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c\" (UID: \"2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c\") "
Oct 30 21:49:10.937228 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:10.937205    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c-home" pod "2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c" (UID: "2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:10.937305 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:10.937293    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c-tls-certs\" (UniqueName: \"kubernetes.io/secret/2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c-tls-certs\") pod \"2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c\" (UID: \"2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c\") "
Oct 30 21:49:10.937703 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:10.937679    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c-model-cache\") pod \"2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c\" (UID: \"2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c\") "
Oct 30 21:49:10.938022 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:10.937996    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c-model-cache" pod "2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c" (UID: "2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:10.938792 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:10.938767    5032 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c-home\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:10.938881 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:10.938797    5032 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c-model-cache\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:10.938881 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:10.938813    5032 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c-tmp-dir\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:10.939604 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:10.939558    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c-kube-api-access-hd2pl" pod "2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c" (UID: "2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c"). InnerVolumeSpecName "kube-api-access-hd2pl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:49:10.939708 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:10.939633    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c-dshm" pod "2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c" (UID: "2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:10.943521 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:10.943493    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c-tls-certs" pod "2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c" (UID: "2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:49:11.040419 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:11.040390    5032 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c-dshm\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:11.040419 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:11.040419    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hd2pl\" (UniqueName: \"kubernetes.io/projected/2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c-kube-api-access-hd2pl\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:11.040628 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:11.040434    5032 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c-tls-certs\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:11.887774 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:11.887739    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw"
Oct 30 21:49:11.912637 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:11.912601    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw"]
Oct 30 21:49:11.916774 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:11.916743    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-7dfff0/distinct-w6-7dfff0-kserve-674b4cf84b-qsclw"]
Oct 30 21:49:13.347997 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:13.347962    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c" path="/var/lib/kubelet/pods/2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c/volumes"
Oct 30 21:49:17.412829 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:17.412789    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-7dfff0-579b4f5ddc-kxqlf" podUID="b53e75a7-c215-4fc9-a64b-8720382c6acd" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:49:18.347512 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:49:18.347479    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 21:49:22.725350 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:22.725311    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-cd3c4c-56cb554497-g2vk9"]
Oct 30 21:49:22.725843 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:22.725655    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cd3c4c-56cb554497-g2vk9" podUID="3193c8ab-5a3f-47cb-b558-0cf84125afa8" containerName="maas-api" containerID="cri-o://883c556bc1876f55e5c7d02ccb6f6bd74aa80b06737e5432f7c4009b2c557925" gracePeriod=30
Oct 30 21:49:23.251471 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:23.251434    5032 generic.go:356] "Generic (PLEG): container finished" podID="3193c8ab-5a3f-47cb-b558-0cf84125afa8" containerID="883c556bc1876f55e5c7d02ccb6f6bd74aa80b06737e5432f7c4009b2c557925" exitCode=0
Oct 30 21:49:23.251698 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:23.251676    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cd3c4c-56cb554497-g2vk9" event={"ID":"3193c8ab-5a3f-47cb-b558-0cf84125afa8","Type":"ContainerDied","Data":"883c556bc1876f55e5c7d02ccb6f6bd74aa80b06737e5432f7c4009b2c557925"}
Oct 30 21:49:23.297607 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:23.297577    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cd3c4c-56cb554497-g2vk9"
Oct 30 21:49:23.336627 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:23.336587    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3193c8ab-5a3f-47cb-b558-0cf84125afa8-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3193c8ab-5a3f-47cb-b558-0cf84125afa8-maas-api-tls\") pod \"3193c8ab-5a3f-47cb-b558-0cf84125afa8\" (UID: \"3193c8ab-5a3f-47cb-b558-0cf84125afa8\") "
Oct 30 21:49:23.336627 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:23.336669    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3193c8ab-5a3f-47cb-b558-0cf84125afa8-kube-api-access-7r48x\" (UniqueName: \"kubernetes.io/projected/3193c8ab-5a3f-47cb-b558-0cf84125afa8-kube-api-access-7r48x\") pod \"3193c8ab-5a3f-47cb-b558-0cf84125afa8\" (UID: \"3193c8ab-5a3f-47cb-b558-0cf84125afa8\") "
Oct 30 21:49:23.336901 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:23.336812    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3193c8ab-5a3f-47cb-b558-0cf84125afa8-metrics-certs\" (UniqueName: \"kubernetes.io/secret/3193c8ab-5a3f-47cb-b558-0cf84125afa8-metrics-certs\") pod \"3193c8ab-5a3f-47cb-b558-0cf84125afa8\" (UID: \"3193c8ab-5a3f-47cb-b558-0cf84125afa8\") "
Oct 30 21:49:23.336901 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:23.336891    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3193c8ab-5a3f-47cb-b558-0cf84125afa8-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3193c8ab-5a3f-47cb-b558-0cf84125afa8-openshift-service-ca\") pod \"3193c8ab-5a3f-47cb-b558-0cf84125afa8\" (UID: \"3193c8ab-5a3f-47cb-b558-0cf84125afa8\") "
Oct 30 21:49:23.337657 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:23.337627    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3193c8ab-5a3f-47cb-b558-0cf84125afa8-openshift-service-ca" pod "3193c8ab-5a3f-47cb-b558-0cf84125afa8" (UID: "3193c8ab-5a3f-47cb-b558-0cf84125afa8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:49:23.338300 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:23.338275    5032 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3193c8ab-5a3f-47cb-b558-0cf84125afa8-openshift-service-ca\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:23.340329 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:23.340292    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3193c8ab-5a3f-47cb-b558-0cf84125afa8-kube-api-access-7r48x" pod "3193c8ab-5a3f-47cb-b558-0cf84125afa8" (UID: "3193c8ab-5a3f-47cb-b558-0cf84125afa8"). InnerVolumeSpecName "kube-api-access-7r48x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:49:23.340514 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:23.340488    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3193c8ab-5a3f-47cb-b558-0cf84125afa8-metrics-certs" pod "3193c8ab-5a3f-47cb-b558-0cf84125afa8" (UID: "3193c8ab-5a3f-47cb-b558-0cf84125afa8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:49:23.344836 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:23.344812    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3193c8ab-5a3f-47cb-b558-0cf84125afa8-maas-api-tls" pod "3193c8ab-5a3f-47cb-b558-0cf84125afa8" (UID: "3193c8ab-5a3f-47cb-b558-0cf84125afa8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:49:23.371627 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:23.371595    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-000f64-6576d48f75-5rdtz"]
Oct 30 21:49:23.387110 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:23.387042    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-000f64-85b9f75579-vc4bc"]
Oct 30 21:49:23.445336 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:23.445300    5032 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3193c8ab-5a3f-47cb-b558-0cf84125afa8-maas-api-tls\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:23.445336 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:23.445333    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7r48x\" (UniqueName: \"kubernetes.io/projected/3193c8ab-5a3f-47cb-b558-0cf84125afa8-kube-api-access-7r48x\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:23.445592 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:23.445349    5032 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3193c8ab-5a3f-47cb-b558-0cf84125afa8-metrics-certs\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:24.260602 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:24.260570    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cd3c4c-56cb554497-g2vk9"
Oct 30 21:49:24.261035 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:24.260607    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cd3c4c-56cb554497-g2vk9" event={"ID":"3193c8ab-5a3f-47cb-b558-0cf84125afa8","Type":"ContainerDied","Data":"cfa7ff62409867e8382746c7caa4ae6e28d3a4e7e7bbeba4b2067c60ae3f96b6"}
Oct 30 21:49:24.261035 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:24.260671    5032 scope.go:122] "RemoveContainer" containerID="883c556bc1876f55e5c7d02ccb6f6bd74aa80b06737e5432f7c4009b2c557925"
Oct 30 21:49:24.282389 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:24.282341    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-cd3c4c-56cb554497-g2vk9"]
Oct 30 21:49:24.288478 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:24.288452    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-cd3c4c-56cb554497-g2vk9"]
Oct 30 21:49:25.370991 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:25.370955    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3193c8ab-5a3f-47cb-b558-0cf84125afa8" path="/var/lib/kubelet/pods/3193c8ab-5a3f-47cb-b558-0cf84125afa8/volumes"
Oct 30 21:49:25.674261 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:25.674183    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg"]
Oct 30 21:49:25.674473 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:25.674388    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg" podUID="516c1abb-b9c5-4164-959b-4a565999db86" containerName="istio-proxy" containerID="cri-o://ac9bedb91419f3bd1c2cd8f9d1e899dbba2d139f4cc35930d99881328eb48191" gracePeriod=30
Oct 30 21:49:26.305305 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:26.305271    5032 generic.go:356] "Generic (PLEG): container finished" podID="b53e75a7-c215-4fc9-a64b-8720382c6acd" containerID="65496a6fcedbf59644f1d4de477af46439ec1b1bc1e7a1c567c89211aa24d2fe" exitCode=0
Oct 30 21:49:26.305556 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:26.305535    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-7dfff0-579b4f5ddc-kxqlf" event={"ID":"b53e75a7-c215-4fc9-a64b-8720382c6acd","Type":"ContainerDied","Data":"65496a6fcedbf59644f1d4de477af46439ec1b1bc1e7a1c567c89211aa24d2fe"}
Oct 30 21:49:26.329786 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:26.329758    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-7dfff0-579b4f5ddc-kxqlf"
Oct 30 21:49:26.404781 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:26.402764    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b53e75a7-c215-4fc9-a64b-8720382c6acd-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b53e75a7-c215-4fc9-a64b-8720382c6acd-plugins-config-volume\") pod \"b53e75a7-c215-4fc9-a64b-8720382c6acd\" (UID: \"b53e75a7-c215-4fc9-a64b-8720382c6acd\") "
Oct 30 21:49:26.404781 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:26.403029    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b53e75a7-c215-4fc9-a64b-8720382c6acd-kube-api-access-jn22k\" (UniqueName: \"kubernetes.io/projected/b53e75a7-c215-4fc9-a64b-8720382c6acd-kube-api-access-jn22k\") pod \"b53e75a7-c215-4fc9-a64b-8720382c6acd\" (UID: \"b53e75a7-c215-4fc9-a64b-8720382c6acd\") "
Oct 30 21:49:26.404781 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:26.403474    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b53e75a7-c215-4fc9-a64b-8720382c6acd-plugins-config-volume" pod "b53e75a7-c215-4fc9-a64b-8720382c6acd" (UID: "b53e75a7-c215-4fc9-a64b-8720382c6acd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:49:26.404781 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:26.404077    5032 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b53e75a7-c215-4fc9-a64b-8720382c6acd-plugins-config-volume\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:26.406045 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:26.406016    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b53e75a7-c215-4fc9-a64b-8720382c6acd-kube-api-access-jn22k" pod "b53e75a7-c215-4fc9-a64b-8720382c6acd" (UID: "b53e75a7-c215-4fc9-a64b-8720382c6acd"). InnerVolumeSpecName "kube-api-access-jn22k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:49:26.506073 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:26.506037    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jn22k\" (UniqueName: \"kubernetes.io/projected/b53e75a7-c215-4fc9-a64b-8720382c6acd-kube-api-access-jn22k\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:27.314884 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:27.314853    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-7dfff0-579b4f5ddc-kxqlf"
Oct 30 21:49:27.315057 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:27.314898    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-7dfff0-579b4f5ddc-kxqlf" event={"ID":"b53e75a7-c215-4fc9-a64b-8720382c6acd","Type":"ContainerDied","Data":"f88e9c0ec5587071b365e2604ad975131a1bf31b384497d2cad3d97840a09520"}
Oct 30 21:49:27.315057 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:27.314975    5032 scope.go:122] "RemoveContainer" containerID="65496a6fcedbf59644f1d4de477af46439ec1b1bc1e7a1c567c89211aa24d2fe"
Oct 30 21:49:27.350839 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:27.350812    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-7dfff0-579b4f5ddc-kxqlf"]
Oct 30 21:49:27.350980 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:27.350848    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-7dfff0-579b4f5ddc-kxqlf"]
Oct 30 21:49:28.372976 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:28.372934    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w0-000f64-6576d48f75-5rdtz" podUID="c9e3ad2b-93c6-46fd-8b0c-87e24852081e" containerName="payload-processing" containerID="cri-o://e23310c51897580dcafb822b29983854a16210e0688ddc6da4a082275bc747df" gracePeriod=25
Oct 30 21:49:28.388076 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:28.388040    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-000f64-85b9f75579-vc4bc" podUID="d3bbb2d6-2cff-40ad-90fa-7e232d26a860" containerName="payload-pre-processing" containerID="cri-o://b31f5ee41ac60d32dab66e4d10fbb926e533ab4ff94a8d34169a9921248b9d15" gracePeriod=25
Oct 30 21:49:29.034674 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:29.034639    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 21:49:29.421756 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:29.421660    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b53e75a7-c215-4fc9-a64b-8720382c6acd" path="/var/lib/kubelet/pods/b53e75a7-c215-4fc9-a64b-8720382c6acd/volumes"
Oct 30 21:49:30.059618 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:30.059580    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-7dfff0-6ccb87d846-7ml8d"]
Oct 30 21:49:30.059890 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:30.059840    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-7dfff0-6ccb87d846-7ml8d" podUID="b5f6a7d6-010d-4bb5-a333-6f299a5022b5" containerName="maas-api" containerID="cri-o://aa9393190b852941e794ff7c91f89d633c495233afd09ade66eb9f4afe9393b0" gracePeriod=30
Oct 30 21:49:30.389277 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:30.389245    5032 generic.go:356] "Generic (PLEG): container finished" podID="b5f6a7d6-010d-4bb5-a333-6f299a5022b5" containerID="aa9393190b852941e794ff7c91f89d633c495233afd09ade66eb9f4afe9393b0" exitCode=0
Oct 30 21:49:30.389432 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:30.389315    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-7dfff0-6ccb87d846-7ml8d" event={"ID":"b5f6a7d6-010d-4bb5-a333-6f299a5022b5","Type":"ContainerDied","Data":"aa9393190b852941e794ff7c91f89d633c495233afd09ade66eb9f4afe9393b0"}
Oct 30 21:49:30.389432 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:30.389413    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-7dfff0-6ccb87d846-7ml8d" event={"ID":"b5f6a7d6-010d-4bb5-a333-6f299a5022b5","Type":"ContainerDied","Data":"f636da81d2ddb1a50bd565a6f4f25834e4b174beda11874d8428b1f38204b613"}
Oct 30 21:49:30.389513 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:30.389434    5032 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f636da81d2ddb1a50bd565a6f4f25834e4b174beda11874d8428b1f38204b613"
Oct 30 21:49:30.410212 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:30.410186    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-7dfff0-6ccb87d846-7ml8d"
Oct 30 21:49:30.505470 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:30.505429    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b5f6a7d6-010d-4bb5-a333-6f299a5022b5-kube-api-access-ch94b\" (UniqueName: \"kubernetes.io/projected/b5f6a7d6-010d-4bb5-a333-6f299a5022b5-kube-api-access-ch94b\") pod \"b5f6a7d6-010d-4bb5-a333-6f299a5022b5\" (UID: \"b5f6a7d6-010d-4bb5-a333-6f299a5022b5\") "
Oct 30 21:49:30.506589 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:30.506555    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b5f6a7d6-010d-4bb5-a333-6f299a5022b5-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5f6a7d6-010d-4bb5-a333-6f299a5022b5-openshift-service-ca\") pod \"b5f6a7d6-010d-4bb5-a333-6f299a5022b5\" (UID: \"b5f6a7d6-010d-4bb5-a333-6f299a5022b5\") "
Oct 30 21:49:30.506905 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:30.506887    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b5f6a7d6-010d-4bb5-a333-6f299a5022b5-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5f6a7d6-010d-4bb5-a333-6f299a5022b5-maas-api-tls\") pod \"b5f6a7d6-010d-4bb5-a333-6f299a5022b5\" (UID: \"b5f6a7d6-010d-4bb5-a333-6f299a5022b5\") "
Oct 30 21:49:30.507118 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:30.507104    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b5f6a7d6-010d-4bb5-a333-6f299a5022b5-metrics-certs\" (UniqueName: \"kubernetes.io/secret/b5f6a7d6-010d-4bb5-a333-6f299a5022b5-metrics-certs\") pod \"b5f6a7d6-010d-4bb5-a333-6f299a5022b5\" (UID: \"b5f6a7d6-010d-4bb5-a333-6f299a5022b5\") "
Oct 30 21:49:30.525426 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:30.521194    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5f6a7d6-010d-4bb5-a333-6f299a5022b5-openshift-service-ca" pod "b5f6a7d6-010d-4bb5-a333-6f299a5022b5" (UID: "b5f6a7d6-010d-4bb5-a333-6f299a5022b5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:49:30.578623 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:30.562616    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5f6a7d6-010d-4bb5-a333-6f299a5022b5-maas-api-tls" pod "b5f6a7d6-010d-4bb5-a333-6f299a5022b5" (UID: "b5f6a7d6-010d-4bb5-a333-6f299a5022b5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:49:30.629400 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:30.626935    5032 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5f6a7d6-010d-4bb5-a333-6f299a5022b5-openshift-service-ca\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:30.629400 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:30.626974    5032 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5f6a7d6-010d-4bb5-a333-6f299a5022b5-maas-api-tls\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:30.697386 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:30.688778    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5f6a7d6-010d-4bb5-a333-6f299a5022b5-kube-api-access-ch94b" pod "b5f6a7d6-010d-4bb5-a333-6f299a5022b5" (UID: "b5f6a7d6-010d-4bb5-a333-6f299a5022b5"). InnerVolumeSpecName "kube-api-access-ch94b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:49:30.761731 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:30.754660    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5f6a7d6-010d-4bb5-a333-6f299a5022b5-metrics-certs" pod "b5f6a7d6-010d-4bb5-a333-6f299a5022b5" (UID: "b5f6a7d6-010d-4bb5-a333-6f299a5022b5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:49:30.762544 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:30.762442    5032 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b5f6a7d6-010d-4bb5-a333-6f299a5022b5-metrics-certs\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:30.762544 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:30.762477    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ch94b\" (UniqueName: \"kubernetes.io/projected/b5f6a7d6-010d-4bb5-a333-6f299a5022b5-kube-api-access-ch94b\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:31.243694 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:31.243659    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg"
Oct 30 21:49:31.280119 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:31.279490    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/516c1abb-b9c5-4164-959b-4a565999db86-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/516c1abb-b9c5-4164-959b-4a565999db86-istio-envoy\") pod \"516c1abb-b9c5-4164-959b-4a565999db86\" (UID: \"516c1abb-b9c5-4164-959b-4a565999db86\") "
Oct 30 21:49:31.280119 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:31.279550    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/516c1abb-b9c5-4164-959b-4a565999db86-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/516c1abb-b9c5-4164-959b-4a565999db86-istio-data\") pod \"516c1abb-b9c5-4164-959b-4a565999db86\" (UID: \"516c1abb-b9c5-4164-959b-4a565999db86\") "
Oct 30 21:49:31.280119 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:31.279646    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/516c1abb-b9c5-4164-959b-4a565999db86-istio-token\" (UniqueName: \"kubernetes.io/projected/516c1abb-b9c5-4164-959b-4a565999db86-istio-token\") pod \"516c1abb-b9c5-4164-959b-4a565999db86\" (UID: \"516c1abb-b9c5-4164-959b-4a565999db86\") "
Oct 30 21:49:31.280119 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:31.279682    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/516c1abb-b9c5-4164-959b-4a565999db86-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/516c1abb-b9c5-4164-959b-4a565999db86-istio-podinfo\") pod \"516c1abb-b9c5-4164-959b-4a565999db86\" (UID: \"516c1abb-b9c5-4164-959b-4a565999db86\") "
Oct 30 21:49:31.280119 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:31.279739    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/516c1abb-b9c5-4164-959b-4a565999db86-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/516c1abb-b9c5-4164-959b-4a565999db86-workload-socket\") pod \"516c1abb-b9c5-4164-959b-4a565999db86\" (UID: \"516c1abb-b9c5-4164-959b-4a565999db86\") "
Oct 30 21:49:31.280119 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:31.279806    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/516c1abb-b9c5-4164-959b-4a565999db86-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/516c1abb-b9c5-4164-959b-4a565999db86-workload-certs\") pod \"516c1abb-b9c5-4164-959b-4a565999db86\" (UID: \"516c1abb-b9c5-4164-959b-4a565999db86\") "
Oct 30 21:49:31.280119 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:31.279879    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/516c1abb-b9c5-4164-959b-4a565999db86-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/516c1abb-b9c5-4164-959b-4a565999db86-istiod-ca-cert\") pod \"516c1abb-b9c5-4164-959b-4a565999db86\" (UID: \"516c1abb-b9c5-4164-959b-4a565999db86\") "
Oct 30 21:49:31.280119 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:31.279943    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/516c1abb-b9c5-4164-959b-4a565999db86-kube-api-access-4mrkp\" (UniqueName: \"kubernetes.io/projected/516c1abb-b9c5-4164-959b-4a565999db86-kube-api-access-4mrkp\") pod \"516c1abb-b9c5-4164-959b-4a565999db86\" (UID: \"516c1abb-b9c5-4164-959b-4a565999db86\") "
Oct 30 21:49:31.280119 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:31.280093    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/516c1abb-b9c5-4164-959b-4a565999db86-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/516c1abb-b9c5-4164-959b-4a565999db86-credential-socket\") pod \"516c1abb-b9c5-4164-959b-4a565999db86\" (UID: \"516c1abb-b9c5-4164-959b-4a565999db86\") "
Oct 30 21:49:31.283122 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:31.281109    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/516c1abb-b9c5-4164-959b-4a565999db86-credential-socket" pod "516c1abb-b9c5-4164-959b-4a565999db86" (UID: "516c1abb-b9c5-4164-959b-4a565999db86"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:31.284300 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:31.284281    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/516c1abb-b9c5-4164-959b-4a565999db86-workload-socket" pod "516c1abb-b9c5-4164-959b-4a565999db86" (UID: "516c1abb-b9c5-4164-959b-4a565999db86"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:31.284885 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:31.284868    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/516c1abb-b9c5-4164-959b-4a565999db86-istio-data" pod "516c1abb-b9c5-4164-959b-4a565999db86" (UID: "516c1abb-b9c5-4164-959b-4a565999db86"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:31.285200 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:31.285175    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/516c1abb-b9c5-4164-959b-4a565999db86-workload-certs" pod "516c1abb-b9c5-4164-959b-4a565999db86" (UID: "516c1abb-b9c5-4164-959b-4a565999db86"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:31.288956 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:31.288345    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/516c1abb-b9c5-4164-959b-4a565999db86-istiod-ca-cert" pod "516c1abb-b9c5-4164-959b-4a565999db86" (UID: "516c1abb-b9c5-4164-959b-4a565999db86"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:49:31.289655 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:31.289634    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/516c1abb-b9c5-4164-959b-4a565999db86-istio-token" pod "516c1abb-b9c5-4164-959b-4a565999db86" (UID: "516c1abb-b9c5-4164-959b-4a565999db86"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:49:31.290099 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:31.290075    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/516c1abb-b9c5-4164-959b-4a565999db86-istio-envoy" pod "516c1abb-b9c5-4164-959b-4a565999db86" (UID: "516c1abb-b9c5-4164-959b-4a565999db86"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:31.294036 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:31.292073    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/516c1abb-b9c5-4164-959b-4a565999db86-kube-api-access-4mrkp" pod "516c1abb-b9c5-4164-959b-4a565999db86" (UID: "516c1abb-b9c5-4164-959b-4a565999db86"). InnerVolumeSpecName "kube-api-access-4mrkp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:49:31.294225 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:31.294204    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/516c1abb-b9c5-4164-959b-4a565999db86-istio-podinfo" pod "516c1abb-b9c5-4164-959b-4a565999db86" (UID: "516c1abb-b9c5-4164-959b-4a565999db86"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:49:31.383498 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:31.383450    5032 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/516c1abb-b9c5-4164-959b-4a565999db86-credential-socket\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:31.383498 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:31.383487    5032 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/516c1abb-b9c5-4164-959b-4a565999db86-istio-envoy\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:31.383498 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:31.383501    5032 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/516c1abb-b9c5-4164-959b-4a565999db86-istio-data\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:31.383874 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:31.383513    5032 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/516c1abb-b9c5-4164-959b-4a565999db86-istio-token\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:31.383874 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:31.383525    5032 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/516c1abb-b9c5-4164-959b-4a565999db86-istio-podinfo\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:31.383874 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:31.383537    5032 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/516c1abb-b9c5-4164-959b-4a565999db86-workload-socket\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:31.383874 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:31.383552    5032 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/516c1abb-b9c5-4164-959b-4a565999db86-workload-certs\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:31.383874 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:31.383565    5032 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/516c1abb-b9c5-4164-959b-4a565999db86-istiod-ca-cert\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:31.383874 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:31.383579    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4mrkp\" (UniqueName: \"kubernetes.io/projected/516c1abb-b9c5-4164-959b-4a565999db86-kube-api-access-4mrkp\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:31.475893 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:31.454676    5032 generic.go:356] "Generic (PLEG): container finished" podID="516c1abb-b9c5-4164-959b-4a565999db86" containerID="ac9bedb91419f3bd1c2cd8f9d1e899dbba2d139f4cc35930d99881328eb48191" exitCode=0
Oct 30 21:49:31.475893 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:31.454783    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-7dfff0-6ccb87d846-7ml8d"
Oct 30 21:49:31.476394 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:31.476341    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg"
Oct 30 21:49:31.482291 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:31.476457    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg" event={"ID":"516c1abb-b9c5-4164-959b-4a565999db86","Type":"ContainerDied","Data":"ac9bedb91419f3bd1c2cd8f9d1e899dbba2d139f4cc35930d99881328eb48191"}
Oct 30 21:49:31.482291 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:31.476931    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg" event={"ID":"516c1abb-b9c5-4164-959b-4a565999db86","Type":"ContainerDied","Data":"60d872f16bcd12192aa1cb51eba571d228308ec92e497b818f88a27523d1eed9"}
Oct 30 21:49:31.482291 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:31.476965    5032 scope.go:122] "RemoveContainer" containerID="ac9bedb91419f3bd1c2cd8f9d1e899dbba2d139f4cc35930d99881328eb48191"
Oct 30 21:49:31.599387 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:31.591128    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg"]
Oct 30 21:49:31.599387 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:31.591190    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-qsjmg"]
Oct 30 21:49:31.606430 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:31.601806    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-7dfff0-6ccb87d846-7ml8d"]
Oct 30 21:49:31.620819 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:31.615024    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-7dfff0-6ccb87d846-7ml8d"]
Oct 30 21:49:31.681594 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:31.674979    5032 scope.go:122] "RemoveContainer" containerID="ac9bedb91419f3bd1c2cd8f9d1e899dbba2d139f4cc35930d99881328eb48191"
Oct 30 21:49:31.684946 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:49:31.684869    5032 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac9bedb91419f3bd1c2cd8f9d1e899dbba2d139f4cc35930d99881328eb48191\": container with ID starting with ac9bedb91419f3bd1c2cd8f9d1e899dbba2d139f4cc35930d99881328eb48191 not found: ID does not exist" containerID="ac9bedb91419f3bd1c2cd8f9d1e899dbba2d139f4cc35930d99881328eb48191"
Oct 30 21:49:31.684946 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:31.684913    5032 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac9bedb91419f3bd1c2cd8f9d1e899dbba2d139f4cc35930d99881328eb48191"} err="failed to get container status \"ac9bedb91419f3bd1c2cd8f9d1e899dbba2d139f4cc35930d99881328eb48191\": rpc error: code = NotFound desc = could not find container \"ac9bedb91419f3bd1c2cd8f9d1e899dbba2d139f4cc35930d99881328eb48191\": container with ID starting with ac9bedb91419f3bd1c2cd8f9d1e899dbba2d139f4cc35930d99881328eb48191 not found: ID does not exist"
Oct 30 21:49:33.462775 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:33.462721    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="516c1abb-b9c5-4164-959b-4a565999db86" path="/var/lib/kubelet/pods/516c1abb-b9c5-4164-959b-4a565999db86/volumes"
Oct 30 21:49:33.464156 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:33.464129    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b5f6a7d6-010d-4bb5-a333-6f299a5022b5" path="/var/lib/kubelet/pods/b5f6a7d6-010d-4bb5-a333-6f299a5022b5/volumes"
Oct 30 21:49:36.403455 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:36.403287    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-000f64-85b9f75579-vc4bc" podUID="d3bbb2d6-2cff-40ad-90fa-7e232d26a860" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:49:36.404211 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:36.404179    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w0-000f64-6576d48f75-5rdtz" podUID="c9e3ad2b-93c6-46fd-8b0c-87e24852081e" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:49:38.871241 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:38.871184    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-dd5bf8-77fb44c66d-28b9d"]
Oct 30 21:49:38.874855 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:38.874810    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dd5bf8-77fb44c66d-28b9d" podUID="f53fa2b2-206d-4342-9e26-edc43dbdf7b7" containerName="maas-api" containerID="cri-o://788fae58e6a7b4eeb910d6b169b504823af1640728201ef93ec09826fc6f84ef" gracePeriod=30
Oct 30 21:49:38.946447 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:38.946391    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dd5bf8-77fb44c66d-28b9d" podUID="f53fa2b2-206d-4342-9e26-edc43dbdf7b7" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.114:8443/health\": dial tcp 10.132.0.114:8443: connect: connection refused"
Oct 30 21:49:39.330599 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:39.330569    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dd5bf8-77fb44c66d-28b9d"
Oct 30 21:49:39.439412 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:39.437215    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f53fa2b2-206d-4342-9e26-edc43dbdf7b7-kube-api-access-xpwls\" (UniqueName: \"kubernetes.io/projected/f53fa2b2-206d-4342-9e26-edc43dbdf7b7-kube-api-access-xpwls\") pod \"f53fa2b2-206d-4342-9e26-edc43dbdf7b7\" (UID: \"f53fa2b2-206d-4342-9e26-edc43dbdf7b7\") "
Oct 30 21:49:39.439412 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:39.437312    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f53fa2b2-206d-4342-9e26-edc43dbdf7b7-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f53fa2b2-206d-4342-9e26-edc43dbdf7b7-maas-api-tls\") pod \"f53fa2b2-206d-4342-9e26-edc43dbdf7b7\" (UID: \"f53fa2b2-206d-4342-9e26-edc43dbdf7b7\") "
Oct 30 21:49:39.439412 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:39.437412    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f53fa2b2-206d-4342-9e26-edc43dbdf7b7-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f53fa2b2-206d-4342-9e26-edc43dbdf7b7-openshift-service-ca\") pod \"f53fa2b2-206d-4342-9e26-edc43dbdf7b7\" (UID: \"f53fa2b2-206d-4342-9e26-edc43dbdf7b7\") "
Oct 30 21:49:39.439412 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:39.437526    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f53fa2b2-206d-4342-9e26-edc43dbdf7b7-metrics-certs\" (UniqueName: \"kubernetes.io/secret/f53fa2b2-206d-4342-9e26-edc43dbdf7b7-metrics-certs\") pod \"f53fa2b2-206d-4342-9e26-edc43dbdf7b7\" (UID: \"f53fa2b2-206d-4342-9e26-edc43dbdf7b7\") "
Oct 30 21:49:39.439412 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:39.439232    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f53fa2b2-206d-4342-9e26-edc43dbdf7b7-openshift-service-ca" pod "f53fa2b2-206d-4342-9e26-edc43dbdf7b7" (UID: "f53fa2b2-206d-4342-9e26-edc43dbdf7b7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:49:39.463681 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:39.463516    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f53fa2b2-206d-4342-9e26-edc43dbdf7b7-metrics-certs" pod "f53fa2b2-206d-4342-9e26-edc43dbdf7b7" (UID: "f53fa2b2-206d-4342-9e26-edc43dbdf7b7"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:49:39.493567 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:39.492070    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f53fa2b2-206d-4342-9e26-edc43dbdf7b7-kube-api-access-xpwls" pod "f53fa2b2-206d-4342-9e26-edc43dbdf7b7" (UID: "f53fa2b2-206d-4342-9e26-edc43dbdf7b7"). InnerVolumeSpecName "kube-api-access-xpwls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:49:39.509594 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:39.508294    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f53fa2b2-206d-4342-9e26-edc43dbdf7b7-maas-api-tls" pod "f53fa2b2-206d-4342-9e26-edc43dbdf7b7" (UID: "f53fa2b2-206d-4342-9e26-edc43dbdf7b7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:49:39.541951 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:39.541770    5032 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f53fa2b2-206d-4342-9e26-edc43dbdf7b7-metrics-certs\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:39.541951 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:39.541810    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xpwls\" (UniqueName: \"kubernetes.io/projected/f53fa2b2-206d-4342-9e26-edc43dbdf7b7-kube-api-access-xpwls\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:39.541951 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:39.541867    5032 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f53fa2b2-206d-4342-9e26-edc43dbdf7b7-maas-api-tls\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:39.541951 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:39.541880    5032 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f53fa2b2-206d-4342-9e26-edc43dbdf7b7-openshift-service-ca\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:39.577748 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:39.577711    5032 generic.go:356] "Generic (PLEG): container finished" podID="f53fa2b2-206d-4342-9e26-edc43dbdf7b7" containerID="788fae58e6a7b4eeb910d6b169b504823af1640728201ef93ec09826fc6f84ef" exitCode=0
Oct 30 21:49:39.577957 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:39.577789    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dd5bf8-77fb44c66d-28b9d"
Oct 30 21:49:39.577957 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:39.577819    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dd5bf8-77fb44c66d-28b9d" event={"ID":"f53fa2b2-206d-4342-9e26-edc43dbdf7b7","Type":"ContainerDied","Data":"788fae58e6a7b4eeb910d6b169b504823af1640728201ef93ec09826fc6f84ef"}
Oct 30 21:49:39.577957 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:39.577872    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dd5bf8-77fb44c66d-28b9d" event={"ID":"f53fa2b2-206d-4342-9e26-edc43dbdf7b7","Type":"ContainerDied","Data":"bc8769f5f383bbab5ac89375ad36b46ffc4091761ae247b85197b83f5bd2eca4"}
Oct 30 21:49:39.577957 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:39.577906    5032 scope.go:122] "RemoveContainer" containerID="788fae58e6a7b4eeb910d6b169b504823af1640728201ef93ec09826fc6f84ef"
Oct 30 21:49:39.597169 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:39.596614    5032 scope.go:122] "RemoveContainer" containerID="788fae58e6a7b4eeb910d6b169b504823af1640728201ef93ec09826fc6f84ef"
Oct 30 21:49:39.597715 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:49:39.597680    5032 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"788fae58e6a7b4eeb910d6b169b504823af1640728201ef93ec09826fc6f84ef\": container with ID starting with 788fae58e6a7b4eeb910d6b169b504823af1640728201ef93ec09826fc6f84ef not found: ID does not exist" containerID="788fae58e6a7b4eeb910d6b169b504823af1640728201ef93ec09826fc6f84ef"
Oct 30 21:49:39.597832 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:39.597745    5032 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"788fae58e6a7b4eeb910d6b169b504823af1640728201ef93ec09826fc6f84ef"} err="failed to get container status \"788fae58e6a7b4eeb910d6b169b504823af1640728201ef93ec09826fc6f84ef\": rpc error: code = NotFound desc = could not find container \"788fae58e6a7b4eeb910d6b169b504823af1640728201ef93ec09826fc6f84ef\": container with ID starting with 788fae58e6a7b4eeb910d6b169b504823af1640728201ef93ec09826fc6f84ef not found: ID does not exist"
Oct 30 21:49:39.619352 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:39.619307    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-dd5bf8-77fb44c66d-28b9d"]
Oct 30 21:49:39.625511 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:39.625478    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-dd5bf8-77fb44c66d-28b9d"]
Oct 30 21:49:39.925437 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:39.918979    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc"]
Oct 30 21:49:39.925437 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:39.920347    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc" podUID="3b92d898-765d-445d-9056-9f256030162e" containerName="istio-proxy" containerID="cri-o://bc19f76ff61a6b70cb04279dd2eca09b3218c48b6d2d6b731ed8e6fb6da03e66" gracePeriod=30
Oct 30 21:49:40.408868 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:40.408833    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n"]
Oct 30 21:49:40.409101 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:40.409074    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n" podUID="74856e06-9b6f-425d-90f8-e973d0649f90" containerName="istio-proxy" containerID="cri-o://0b6751f9d63c1c06407f8c38e60aa49c0ccd14d070648e6e8d7e55147b6c1aac" gracePeriod=30
Oct 30 21:49:40.686280 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:40.686179    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s"]
Oct 30 21:49:40.686492 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:40.686432    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s" podUID="222c96b4-9c61-49c0-a3ed-450ea7d17b67" containerName="istio-proxy" containerID="cri-o://a0cc01dd8a6e2215c301be962219d059ff1634bf72bac7e9f43fab59f4f5abc4" gracePeriod=30
Oct 30 21:49:40.748386 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:40.747384    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2"]
Oct 30 21:49:40.748386 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:40.747476    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww"]
Oct 30 21:49:40.748386 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:40.747683    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww" podUID="63f89cda-14de-416c-9263-00ffdb8c9ec9" containerName="istio-proxy" containerID="cri-o://329f325c035c1901134d37ebff509d6feb27eecda94f08290199a7b843bbb21a" gracePeriod=30
Oct 30 21:49:40.748386 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:40.747970    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2" podUID="62000861-c573-4dba-8bca-c6f962057f5b" containerName="istio-proxy" containerID="cri-o://ff7e7fa1ccecb3bf7d7b3431f975cbaf5fa1442a004336175284517e055f4b07" gracePeriod=30
Oct 30 21:49:40.778238 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:40.778196    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2"]
Oct 30 21:49:40.778460 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:40.778435    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2" podUID="57dfd470-9c35-404f-b967-14a9103adeea" containerName="istio-proxy" containerID="cri-o://573e81d49ed938098ecb7f82d952e67c858bdee65faa9ad51332ddd424dcdbcd" gracePeriod=30
Oct 30 21:49:41.352563 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:41.352527    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f53fa2b2-206d-4342-9e26-edc43dbdf7b7" path="/var/lib/kubelet/pods/f53fa2b2-206d-4342-9e26-edc43dbdf7b7/volumes"
Oct 30 21:49:41.530034 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:41.529995    5032 patch_prober.go:29] interesting pod/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:49:41.530196 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:41.530066    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc" podUID="3b92d898-765d-445d-9056-9f256030162e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:49:41.845063 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:41.845022    5032 patch_prober.go:29] interesting pod/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:49:41.845311 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:41.845093    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww" podUID="63f89cda-14de-416c-9263-00ffdb8c9ec9" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:49:42.297173 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:42.297138    5032 patch_prober.go:29] interesting pod/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:49:42.297344 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:42.297194    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2" podUID="57dfd470-9c35-404f-b967-14a9103adeea" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:49:42.575933 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:42.575821    5032 patch_prober.go:29] interesting pod/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:49:42.575933 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:42.575888    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s" podUID="222c96b4-9c61-49c0-a3ed-450ea7d17b67" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:49:43.791575 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:43.791541    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb"]
Oct 30 21:49:43.792345 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:43.792320    5032 cpu_manager.go:414] "containerMap: removing container" podUID="4577b3de-b9e4-47a7-87e0-bcba13b11bbe" containerName="istio-proxy"
Oct 30 21:49:43.792345 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:43.792337    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4577b3de-b9e4-47a7-87e0-bcba13b11bbe" containerName="istio-proxy"
Oct 30 21:49:43.792443 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:43.792371    5032 cpu_manager.go:414] "containerMap: removing container" podUID="b5f6a7d6-010d-4bb5-a333-6f299a5022b5" containerName="maas-api"
Oct 30 21:49:43.792443 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:43.792380    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b5f6a7d6-010d-4bb5-a333-6f299a5022b5" containerName="maas-api"
Oct 30 21:49:43.792443 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:43.792388    5032 cpu_manager.go:414] "containerMap: removing container" podUID="b53e75a7-c215-4fc9-a64b-8720382c6acd" containerName="payload-pre-processing"
Oct 30 21:49:43.792443 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:43.792393    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b53e75a7-c215-4fc9-a64b-8720382c6acd" containerName="payload-pre-processing"
Oct 30 21:49:43.792443 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:43.792415    5032 cpu_manager.go:414] "containerMap: removing container" podUID="a12e67c9-820d-48a8-b5e0-c9656ee8d199" containerName="maas-api"
Oct 30 21:49:43.792443 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:43.792422    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a12e67c9-820d-48a8-b5e0-c9656ee8d199" containerName="maas-api"
Oct 30 21:49:43.792443 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:43.792431    5032 cpu_manager.go:414] "containerMap: removing container" podUID="f924f351-d12c-438b-91e0-71f9a316b14d" containerName="istio-proxy"
Oct 30 21:49:43.792443 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:43.792436    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f924f351-d12c-438b-91e0-71f9a316b14d" containerName="istio-proxy"
Oct 30 21:49:43.792443 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:43.792446    5032 cpu_manager.go:414] "containerMap: removing container" podUID="f53fa2b2-206d-4342-9e26-edc43dbdf7b7" containerName="maas-api"
Oct 30 21:49:43.792703 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:43.792450    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f53fa2b2-206d-4342-9e26-edc43dbdf7b7" containerName="maas-api"
Oct 30 21:49:43.792703 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:43.792459    5032 cpu_manager.go:414] "containerMap: removing container" podUID="3193c8ab-5a3f-47cb-b558-0cf84125afa8" containerName="maas-api"
Oct 30 21:49:43.792703 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:43.792463    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3193c8ab-5a3f-47cb-b558-0cf84125afa8" containerName="maas-api"
Oct 30 21:49:43.792703 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:43.792476    5032 cpu_manager.go:414] "containerMap: removing container" podUID="30ea822a-85f1-4db5-a3a2-b8edd78d1200" containerName="istio-proxy"
Oct 30 21:49:43.792703 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:43.792480    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="30ea822a-85f1-4db5-a3a2-b8edd78d1200" containerName="istio-proxy"
Oct 30 21:49:43.792703 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:43.792489    5032 cpu_manager.go:414] "containerMap: removing container" podUID="516c1abb-b9c5-4164-959b-4a565999db86" containerName="istio-proxy"
Oct 30 21:49:43.792703 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:43.792493    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="516c1abb-b9c5-4164-959b-4a565999db86" containerName="istio-proxy"
Oct 30 21:49:43.792703 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:43.792499    5032 cpu_manager.go:414] "containerMap: removing container" podUID="2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c" containerName="main"
Oct 30 21:49:43.792703 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:43.792503    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c" containerName="main"
Oct 30 21:49:43.792703 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:43.792632    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="30ea822a-85f1-4db5-a3a2-b8edd78d1200" containerName="istio-proxy"
Oct 30 21:49:43.792703 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:43.792642    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="516c1abb-b9c5-4164-959b-4a565999db86" containerName="istio-proxy"
Oct 30 21:49:43.792703 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:43.792648    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="2ce9b0b7-ff4f-4bd4-ab29-18e2b340c50c" containerName="main"
Oct 30 21:49:43.792703 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:43.792660    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="4577b3de-b9e4-47a7-87e0-bcba13b11bbe" containerName="istio-proxy"
Oct 30 21:49:43.792703 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:43.792664    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="b5f6a7d6-010d-4bb5-a333-6f299a5022b5" containerName="maas-api"
Oct 30 21:49:43.792703 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:43.792670    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="b53e75a7-c215-4fc9-a64b-8720382c6acd" containerName="payload-pre-processing"
Oct 30 21:49:43.792703 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:43.792678    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="a12e67c9-820d-48a8-b5e0-c9656ee8d199" containerName="maas-api"
Oct 30 21:49:43.792703 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:43.792682    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="f924f351-d12c-438b-91e0-71f9a316b14d" containerName="istio-proxy"
Oct 30 21:49:43.792703 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:43.792688    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="f53fa2b2-206d-4342-9e26-edc43dbdf7b7" containerName="maas-api"
Oct 30 21:49:43.792703 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:43.792694    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="3193c8ab-5a3f-47cb-b558-0cf84125afa8" containerName="maas-api"
Oct 30 21:49:43.798583 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:43.798552    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb"
Oct 30 21:49:43.807979 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:43.807950    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-08aafeac-openshift-default-dockercfg-qg8xv\""
Oct 30 21:49:43.812912 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:43.812610    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb"]
Oct 30 21:49:43.859630 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:43.859586    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/629450ab-662b-43c5-b93d-e264b1ce2034-workload-certs\") pod \"e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb\" (UID: \"629450ab-662b-43c5-b93d-e264b1ce2034\") " pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb"
Oct 30 21:49:43.859820 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:43.859655    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/629450ab-662b-43c5-b93d-e264b1ce2034-istio-envoy\") pod \"e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb\" (UID: \"629450ab-662b-43c5-b93d-e264b1ce2034\") " pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb"
Oct 30 21:49:43.859820 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:43.859782    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/629450ab-662b-43c5-b93d-e264b1ce2034-workload-socket\") pod \"e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb\" (UID: \"629450ab-662b-43c5-b93d-e264b1ce2034\") " pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb"
Oct 30 21:49:43.859929 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:43.859855    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/629450ab-662b-43c5-b93d-e264b1ce2034-istio-token\") pod \"e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb\" (UID: \"629450ab-662b-43c5-b93d-e264b1ce2034\") " pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb"
Oct 30 21:49:43.860085 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:43.860061    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/629450ab-662b-43c5-b93d-e264b1ce2034-istiod-ca-cert\") pod \"e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb\" (UID: \"629450ab-662b-43c5-b93d-e264b1ce2034\") " pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb"
Oct 30 21:49:43.860189 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:43.860173    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/629450ab-662b-43c5-b93d-e264b1ce2034-istio-data\") pod \"e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb\" (UID: \"629450ab-662b-43c5-b93d-e264b1ce2034\") " pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb"
Oct 30 21:49:43.860337 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:43.860310    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/629450ab-662b-43c5-b93d-e264b1ce2034-credential-socket\") pod \"e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb\" (UID: \"629450ab-662b-43c5-b93d-e264b1ce2034\") " pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb"
Oct 30 21:49:43.860509 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:43.860342    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/629450ab-662b-43c5-b93d-e264b1ce2034-istio-podinfo\") pod \"e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb\" (UID: \"629450ab-662b-43c5-b93d-e264b1ce2034\") " pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb"
Oct 30 21:49:43.860509 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:43.860474    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2gmb\" (UniqueName: \"kubernetes.io/projected/629450ab-662b-43c5-b93d-e264b1ce2034-kube-api-access-m2gmb\") pod \"e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb\" (UID: \"629450ab-662b-43c5-b93d-e264b1ce2034\") " pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb"
Oct 30 21:49:43.962272 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:43.962232    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/629450ab-662b-43c5-b93d-e264b1ce2034-istiod-ca-cert\") pod \"e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb\" (UID: \"629450ab-662b-43c5-b93d-e264b1ce2034\") " pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb"
Oct 30 21:49:43.962496 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:43.962297    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/629450ab-662b-43c5-b93d-e264b1ce2034-istio-data\") pod \"e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb\" (UID: \"629450ab-662b-43c5-b93d-e264b1ce2034\") " pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb"
Oct 30 21:49:43.962496 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:43.962409    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/629450ab-662b-43c5-b93d-e264b1ce2034-credential-socket\") pod \"e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb\" (UID: \"629450ab-662b-43c5-b93d-e264b1ce2034\") " pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb"
Oct 30 21:49:43.962672 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:43.962435    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/629450ab-662b-43c5-b93d-e264b1ce2034-istio-podinfo\") pod \"e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb\" (UID: \"629450ab-662b-43c5-b93d-e264b1ce2034\") " pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb"
Oct 30 21:49:43.962744 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:43.962731    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2gmb\" (UniqueName: \"kubernetes.io/projected/629450ab-662b-43c5-b93d-e264b1ce2034-kube-api-access-m2gmb\") pod \"e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb\" (UID: \"629450ab-662b-43c5-b93d-e264b1ce2034\") " pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb"
Oct 30 21:49:43.962837 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:43.962817    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/629450ab-662b-43c5-b93d-e264b1ce2034-workload-certs\") pod \"e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb\" (UID: \"629450ab-662b-43c5-b93d-e264b1ce2034\") " pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb"
Oct 30 21:49:43.962890 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:43.962880    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/629450ab-662b-43c5-b93d-e264b1ce2034-istio-envoy\") pod \"e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb\" (UID: \"629450ab-662b-43c5-b93d-e264b1ce2034\") " pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb"
Oct 30 21:49:43.963004 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:43.962987    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/629450ab-662b-43c5-b93d-e264b1ce2034-workload-socket\") pod \"e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb\" (UID: \"629450ab-662b-43c5-b93d-e264b1ce2034\") " pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb"
Oct 30 21:49:43.963121 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:43.963106    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/629450ab-662b-43c5-b93d-e264b1ce2034-istio-token\") pod \"e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb\" (UID: \"629450ab-662b-43c5-b93d-e264b1ce2034\") " pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb"
Oct 30 21:49:43.963335 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:43.963233    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/629450ab-662b-43c5-b93d-e264b1ce2034-credential-socket\") pod \"e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb\" (UID: \"629450ab-662b-43c5-b93d-e264b1ce2034\") " pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb"
Oct 30 21:49:43.963996 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:43.963417    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/629450ab-662b-43c5-b93d-e264b1ce2034-istio-data\") pod \"e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb\" (UID: \"629450ab-662b-43c5-b93d-e264b1ce2034\") " pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb"
Oct 30 21:49:43.963996 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:43.963620    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/629450ab-662b-43c5-b93d-e264b1ce2034-workload-certs\") pod \"e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb\" (UID: \"629450ab-662b-43c5-b93d-e264b1ce2034\") " pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb"
Oct 30 21:49:43.963996 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:43.963877    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/629450ab-662b-43c5-b93d-e264b1ce2034-workload-socket\") pod \"e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb\" (UID: \"629450ab-662b-43c5-b93d-e264b1ce2034\") " pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb"
Oct 30 21:49:43.965141 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:43.965116    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/629450ab-662b-43c5-b93d-e264b1ce2034-istiod-ca-cert\") pod \"e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb\" (UID: \"629450ab-662b-43c5-b93d-e264b1ce2034\") " pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb"
Oct 30 21:49:43.966497 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:43.966473    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/629450ab-662b-43c5-b93d-e264b1ce2034-istio-envoy\") pod \"e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb\" (UID: \"629450ab-662b-43c5-b93d-e264b1ce2034\") " pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb"
Oct 30 21:49:43.968511 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:43.968482    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/629450ab-662b-43c5-b93d-e264b1ce2034-istio-podinfo\") pod \"e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb\" (UID: \"629450ab-662b-43c5-b93d-e264b1ce2034\") " pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb"
Oct 30 21:49:43.977948 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:43.977921    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/629450ab-662b-43c5-b93d-e264b1ce2034-istio-token\") pod \"e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb\" (UID: \"629450ab-662b-43c5-b93d-e264b1ce2034\") " pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb"
Oct 30 21:49:43.985762 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:43.979155    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2gmb\" (UniqueName: \"kubernetes.io/projected/629450ab-662b-43c5-b93d-e264b1ce2034-kube-api-access-m2gmb\") pod \"e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb\" (UID: \"629450ab-662b-43c5-b93d-e264b1ce2034\") " pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb"
Oct 30 21:49:44.150850 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:44.150762    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb"
Oct 30 21:49:44.476230 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:44.476196    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb"]
Oct 30 21:49:44.484495 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:44.484461    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:49:44.484602 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:44.484529    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:49:44.484602 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:44.484562    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:49:44.643671 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:44.643632    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb" event={"ID":"629450ab-662b-43c5-b93d-e264b1ce2034","Type":"ContainerStarted","Data":"9b6b8ea7cbe715277dfd7789482c5bc09d4ec4ffb1e88addfebb451618561677"}
Oct 30 21:49:44.643934 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:44.643714    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb" event={"ID":"629450ab-662b-43c5-b93d-e264b1ce2034","Type":"ContainerStarted","Data":"2a156f8f68a3ab4cf5e5f14d4f10773edf39e9b8f23df4841a72f01cbf0ec3b4"}
Oct 30 21:49:45.151938 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:45.151907    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb"
Oct 30 21:49:45.154208 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:45.154182    5032 patch_prober.go:29] interesting pod/e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused" start-of-body=
Oct 30 21:49:45.154327 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:45.154239    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb" podUID="629450ab-662b-43c5-b93d-e264b1ce2034" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused"
Oct 30 21:49:45.278380 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:45.278342    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc"
Oct 30 21:49:45.307348 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:45.307287    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb" podStartSLOduration=2.307265241 podStartE2EDuration="2.307265241s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:49:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:49:44.733286006 +0000 UTC m=+1297.535098474" watchObservedRunningTime="2026-10-30 21:49:45.307265241 +0000 UTC m=+1298.109077708"
Oct 30 21:49:45.388252 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:45.387304    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3b92d898-765d-445d-9056-9f256030162e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b92d898-765d-445d-9056-9f256030162e-workload-certs\") pod \"3b92d898-765d-445d-9056-9f256030162e\" (UID: \"3b92d898-765d-445d-9056-9f256030162e\") "
Oct 30 21:49:45.388252 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:45.387355    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3b92d898-765d-445d-9056-9f256030162e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b92d898-765d-445d-9056-9f256030162e-istio-podinfo\") pod \"3b92d898-765d-445d-9056-9f256030162e\" (UID: \"3b92d898-765d-445d-9056-9f256030162e\") "
Oct 30 21:49:45.388252 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:45.387439    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3b92d898-765d-445d-9056-9f256030162e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b92d898-765d-445d-9056-9f256030162e-istio-envoy\") pod \"3b92d898-765d-445d-9056-9f256030162e\" (UID: \"3b92d898-765d-445d-9056-9f256030162e\") "
Oct 30 21:49:45.388252 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:45.387488    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3b92d898-765d-445d-9056-9f256030162e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b92d898-765d-445d-9056-9f256030162e-credential-socket\") pod \"3b92d898-765d-445d-9056-9f256030162e\" (UID: \"3b92d898-765d-445d-9056-9f256030162e\") "
Oct 30 21:49:45.388252 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:45.387625    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3b92d898-765d-445d-9056-9f256030162e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b92d898-765d-445d-9056-9f256030162e-workload-socket\") pod \"3b92d898-765d-445d-9056-9f256030162e\" (UID: \"3b92d898-765d-445d-9056-9f256030162e\") "
Oct 30 21:49:45.388252 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:45.387738    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3b92d898-765d-445d-9056-9f256030162e-kube-api-access-7hvh5\" (UniqueName: \"kubernetes.io/projected/3b92d898-765d-445d-9056-9f256030162e-kube-api-access-7hvh5\") pod \"3b92d898-765d-445d-9056-9f256030162e\" (UID: \"3b92d898-765d-445d-9056-9f256030162e\") "
Oct 30 21:49:45.388252 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:45.387788    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3b92d898-765d-445d-9056-9f256030162e-istio-token\" (UniqueName: \"kubernetes.io/projected/3b92d898-765d-445d-9056-9f256030162e-istio-token\") pod \"3b92d898-765d-445d-9056-9f256030162e\" (UID: \"3b92d898-765d-445d-9056-9f256030162e\") "
Oct 30 21:49:45.388252 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:45.387843    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3b92d898-765d-445d-9056-9f256030162e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b92d898-765d-445d-9056-9f256030162e-istiod-ca-cert\") pod \"3b92d898-765d-445d-9056-9f256030162e\" (UID: \"3b92d898-765d-445d-9056-9f256030162e\") "
Oct 30 21:49:45.388252 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:45.387873    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3b92d898-765d-445d-9056-9f256030162e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b92d898-765d-445d-9056-9f256030162e-istio-data\") pod \"3b92d898-765d-445d-9056-9f256030162e\" (UID: \"3b92d898-765d-445d-9056-9f256030162e\") "
Oct 30 21:49:45.389210 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:45.389150    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b92d898-765d-445d-9056-9f256030162e-istio-data" pod "3b92d898-765d-445d-9056-9f256030162e" (UID: "3b92d898-765d-445d-9056-9f256030162e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:45.389437 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:45.389413    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b92d898-765d-445d-9056-9f256030162e-workload-socket" pod "3b92d898-765d-445d-9056-9f256030162e" (UID: "3b92d898-765d-445d-9056-9f256030162e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:45.390183 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:45.390144    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b92d898-765d-445d-9056-9f256030162e-credential-socket" pod "3b92d898-765d-445d-9056-9f256030162e" (UID: "3b92d898-765d-445d-9056-9f256030162e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:45.392498 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:45.392469    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b92d898-765d-445d-9056-9f256030162e-istiod-ca-cert" pod "3b92d898-765d-445d-9056-9f256030162e" (UID: "3b92d898-765d-445d-9056-9f256030162e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:49:45.392631 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:45.392604    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3b92d898-765d-445d-9056-9f256030162e-istio-podinfo" pod "3b92d898-765d-445d-9056-9f256030162e" (UID: "3b92d898-765d-445d-9056-9f256030162e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:49:45.392782 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:45.392758    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b92d898-765d-445d-9056-9f256030162e-workload-certs" pod "3b92d898-765d-445d-9056-9f256030162e" (UID: "3b92d898-765d-445d-9056-9f256030162e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:45.397986 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:45.397956    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b92d898-765d-445d-9056-9f256030162e-kube-api-access-7hvh5" pod "3b92d898-765d-445d-9056-9f256030162e" (UID: "3b92d898-765d-445d-9056-9f256030162e"). InnerVolumeSpecName "kube-api-access-7hvh5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:49:45.399588 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:45.399561    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b92d898-765d-445d-9056-9f256030162e-istio-token" pod "3b92d898-765d-445d-9056-9f256030162e" (UID: "3b92d898-765d-445d-9056-9f256030162e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:49:45.399812 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:45.399765    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b92d898-765d-445d-9056-9f256030162e-istio-envoy" pod "3b92d898-765d-445d-9056-9f256030162e" (UID: "3b92d898-765d-445d-9056-9f256030162e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:45.493593 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:45.492644    5032 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b92d898-765d-445d-9056-9f256030162e-workload-certs\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:45.493593 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:45.492677    5032 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b92d898-765d-445d-9056-9f256030162e-istio-podinfo\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:45.493593 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:45.492691    5032 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b92d898-765d-445d-9056-9f256030162e-istio-envoy\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:45.493593 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:45.492706    5032 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b92d898-765d-445d-9056-9f256030162e-credential-socket\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:45.493593 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:45.492719    5032 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b92d898-765d-445d-9056-9f256030162e-workload-socket\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:45.493593 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:45.492732    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7hvh5\" (UniqueName: \"kubernetes.io/projected/3b92d898-765d-445d-9056-9f256030162e-kube-api-access-7hvh5\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:45.493593 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:45.492745    5032 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b92d898-765d-445d-9056-9f256030162e-istio-token\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:45.493593 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:45.492758    5032 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b92d898-765d-445d-9056-9f256030162e-istiod-ca-cert\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:45.493593 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:45.492772    5032 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b92d898-765d-445d-9056-9f256030162e-istio-data\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:45.654916 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:45.654878    5032 generic.go:356] "Generic (PLEG): container finished" podID="74856e06-9b6f-425d-90f8-e973d0649f90" containerID="0b6751f9d63c1c06407f8c38e60aa49c0ccd14d070648e6e8d7e55147b6c1aac" exitCode=0
Oct 30 21:49:45.655084 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:45.655054    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n" event={"ID":"74856e06-9b6f-425d-90f8-e973d0649f90","Type":"ContainerDied","Data":"0b6751f9d63c1c06407f8c38e60aa49c0ccd14d070648e6e8d7e55147b6c1aac"}
Oct 30 21:49:45.659967 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:45.659933    5032 generic.go:356] "Generic (PLEG): container finished" podID="3b92d898-765d-445d-9056-9f256030162e" containerID="bc19f76ff61a6b70cb04279dd2eca09b3218c48b6d2d6b731ed8e6fb6da03e66" exitCode=0
Oct 30 21:49:45.660336 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:45.660308    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc"
Oct 30 21:49:45.661482 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:45.661446    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc" event={"ID":"3b92d898-765d-445d-9056-9f256030162e","Type":"ContainerDied","Data":"bc19f76ff61a6b70cb04279dd2eca09b3218c48b6d2d6b731ed8e6fb6da03e66"}
Oct 30 21:49:45.661593 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:45.661561    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc" event={"ID":"3b92d898-765d-445d-9056-9f256030162e","Type":"ContainerDied","Data":"01ca058bc33c80e0bf7c34d70704e463a3fb120936834e4dcc94b5f93d5b00c7"}
Oct 30 21:49:45.661660 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:45.661596    5032 scope.go:122] "RemoveContainer" containerID="bc19f76ff61a6b70cb04279dd2eca09b3218c48b6d2d6b731ed8e6fb6da03e66"
Oct 30 21:49:45.685848 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:45.685823    5032 scope.go:122] "RemoveContainer" containerID="bc19f76ff61a6b70cb04279dd2eca09b3218c48b6d2d6b731ed8e6fb6da03e66"
Oct 30 21:49:45.686197 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:49:45.686134    5032 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc19f76ff61a6b70cb04279dd2eca09b3218c48b6d2d6b731ed8e6fb6da03e66\": container with ID starting with bc19f76ff61a6b70cb04279dd2eca09b3218c48b6d2d6b731ed8e6fb6da03e66 not found: ID does not exist" containerID="bc19f76ff61a6b70cb04279dd2eca09b3218c48b6d2d6b731ed8e6fb6da03e66"
Oct 30 21:49:45.686197 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:45.686168    5032 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc19f76ff61a6b70cb04279dd2eca09b3218c48b6d2d6b731ed8e6fb6da03e66"} err="failed to get container status \"bc19f76ff61a6b70cb04279dd2eca09b3218c48b6d2d6b731ed8e6fb6da03e66\": rpc error: code = NotFound desc = could not find container \"bc19f76ff61a6b70cb04279dd2eca09b3218c48b6d2d6b731ed8e6fb6da03e66\": container with ID starting with bc19f76ff61a6b70cb04279dd2eca09b3218c48b6d2d6b731ed8e6fb6da03e66 not found: ID does not exist"
Oct 30 21:49:45.706924 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:45.706891    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc"]
Oct 30 21:49:45.732850 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:45.713457    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-vkvvc"]
Oct 30 21:49:45.986560 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:45.986174    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n"
Oct 30 21:49:46.052872 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.027745    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/74856e06-9b6f-425d-90f8-e973d0649f90-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74856e06-9b6f-425d-90f8-e973d0649f90-workload-socket\") pod \"74856e06-9b6f-425d-90f8-e973d0649f90\" (UID: \"74856e06-9b6f-425d-90f8-e973d0649f90\") "
Oct 30 21:49:46.052872 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.027867    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/74856e06-9b6f-425d-90f8-e973d0649f90-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74856e06-9b6f-425d-90f8-e973d0649f90-credential-socket\") pod \"74856e06-9b6f-425d-90f8-e973d0649f90\" (UID: \"74856e06-9b6f-425d-90f8-e973d0649f90\") "
Oct 30 21:49:46.052872 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.027901    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/74856e06-9b6f-425d-90f8-e973d0649f90-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74856e06-9b6f-425d-90f8-e973d0649f90-istio-envoy\") pod \"74856e06-9b6f-425d-90f8-e973d0649f90\" (UID: \"74856e06-9b6f-425d-90f8-e973d0649f90\") "
Oct 30 21:49:46.052872 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.028085    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/74856e06-9b6f-425d-90f8-e973d0649f90-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74856e06-9b6f-425d-90f8-e973d0649f90-istiod-ca-cert\") pod \"74856e06-9b6f-425d-90f8-e973d0649f90\" (UID: \"74856e06-9b6f-425d-90f8-e973d0649f90\") "
Oct 30 21:49:46.052872 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.028115    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/74856e06-9b6f-425d-90f8-e973d0649f90-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74856e06-9b6f-425d-90f8-e973d0649f90-istio-data\") pod \"74856e06-9b6f-425d-90f8-e973d0649f90\" (UID: \"74856e06-9b6f-425d-90f8-e973d0649f90\") "
Oct 30 21:49:46.052872 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.028203    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/74856e06-9b6f-425d-90f8-e973d0649f90-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74856e06-9b6f-425d-90f8-e973d0649f90-workload-certs\") pod \"74856e06-9b6f-425d-90f8-e973d0649f90\" (UID: \"74856e06-9b6f-425d-90f8-e973d0649f90\") "
Oct 30 21:49:46.052872 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.028236    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/74856e06-9b6f-425d-90f8-e973d0649f90-istio-token\" (UniqueName: \"kubernetes.io/projected/74856e06-9b6f-425d-90f8-e973d0649f90-istio-token\") pod \"74856e06-9b6f-425d-90f8-e973d0649f90\" (UID: \"74856e06-9b6f-425d-90f8-e973d0649f90\") "
Oct 30 21:49:46.052872 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.028334    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/74856e06-9b6f-425d-90f8-e973d0649f90-kube-api-access-2n82w\" (UniqueName: \"kubernetes.io/projected/74856e06-9b6f-425d-90f8-e973d0649f90-kube-api-access-2n82w\") pod \"74856e06-9b6f-425d-90f8-e973d0649f90\" (UID: \"74856e06-9b6f-425d-90f8-e973d0649f90\") "
Oct 30 21:49:46.052872 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.028402    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/74856e06-9b6f-425d-90f8-e973d0649f90-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74856e06-9b6f-425d-90f8-e973d0649f90-istio-podinfo\") pod \"74856e06-9b6f-425d-90f8-e973d0649f90\" (UID: \"74856e06-9b6f-425d-90f8-e973d0649f90\") "
Oct 30 21:49:46.056502 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.055443    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74856e06-9b6f-425d-90f8-e973d0649f90-istio-data" pod "74856e06-9b6f-425d-90f8-e973d0649f90" (UID: "74856e06-9b6f-425d-90f8-e973d0649f90"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:46.056502 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.055662    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74856e06-9b6f-425d-90f8-e973d0649f90-credential-socket" pod "74856e06-9b6f-425d-90f8-e973d0649f90" (UID: "74856e06-9b6f-425d-90f8-e973d0649f90"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:46.056502 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.055851    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74856e06-9b6f-425d-90f8-e973d0649f90-workload-certs" pod "74856e06-9b6f-425d-90f8-e973d0649f90" (UID: "74856e06-9b6f-425d-90f8-e973d0649f90"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:46.056502 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.056210    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74856e06-9b6f-425d-90f8-e973d0649f90-workload-socket" pod "74856e06-9b6f-425d-90f8-e973d0649f90" (UID: "74856e06-9b6f-425d-90f8-e973d0649f90"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:46.076701 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.070293    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74856e06-9b6f-425d-90f8-e973d0649f90-istiod-ca-cert" pod "74856e06-9b6f-425d-90f8-e973d0649f90" (UID: "74856e06-9b6f-425d-90f8-e973d0649f90"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:49:46.079093 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.077035    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74856e06-9b6f-425d-90f8-e973d0649f90-istio-token" pod "74856e06-9b6f-425d-90f8-e973d0649f90" (UID: "74856e06-9b6f-425d-90f8-e973d0649f90"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:49:46.082171 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.082125    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74856e06-9b6f-425d-90f8-e973d0649f90-istio-envoy" pod "74856e06-9b6f-425d-90f8-e973d0649f90" (UID: "74856e06-9b6f-425d-90f8-e973d0649f90"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:46.082793 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.082754    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/74856e06-9b6f-425d-90f8-e973d0649f90-istio-podinfo" pod "74856e06-9b6f-425d-90f8-e973d0649f90" (UID: "74856e06-9b6f-425d-90f8-e973d0649f90"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:49:46.083826 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.083496    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74856e06-9b6f-425d-90f8-e973d0649f90-kube-api-access-2n82w" pod "74856e06-9b6f-425d-90f8-e973d0649f90" (UID: "74856e06-9b6f-425d-90f8-e973d0649f90"). InnerVolumeSpecName "kube-api-access-2n82w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:49:46.151424 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.135122    5032 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74856e06-9b6f-425d-90f8-e973d0649f90-workload-certs\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:46.151424 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.135164    5032 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74856e06-9b6f-425d-90f8-e973d0649f90-istio-token\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:46.151424 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.135181    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2n82w\" (UniqueName: \"kubernetes.io/projected/74856e06-9b6f-425d-90f8-e973d0649f90-kube-api-access-2n82w\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:46.151424 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.135196    5032 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74856e06-9b6f-425d-90f8-e973d0649f90-istio-podinfo\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:46.151424 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.135210    5032 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74856e06-9b6f-425d-90f8-e973d0649f90-workload-socket\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:46.151424 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.135225    5032 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74856e06-9b6f-425d-90f8-e973d0649f90-credential-socket\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:46.151424 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.135239    5032 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74856e06-9b6f-425d-90f8-e973d0649f90-istio-envoy\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:46.151424 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.135251    5032 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74856e06-9b6f-425d-90f8-e973d0649f90-istiod-ca-cert\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:46.151424 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.135265    5032 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74856e06-9b6f-425d-90f8-e973d0649f90-istio-data\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:46.154928 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.154487    5032 patch_prober.go:29] interesting pod/e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused" start-of-body=
Oct 30 21:49:46.154928 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.154548    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb" podUID="629450ab-662b-43c5-b93d-e264b1ce2034" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused"
Oct 30 21:49:46.280555 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.280528    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2"
Oct 30 21:49:46.346201 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.346168    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/62000861-c573-4dba-8bca-c6f962057f5b-istio-token\" (UniqueName: \"kubernetes.io/projected/62000861-c573-4dba-8bca-c6f962057f5b-istio-token\") pod \"62000861-c573-4dba-8bca-c6f962057f5b\" (UID: \"62000861-c573-4dba-8bca-c6f962057f5b\") "
Oct 30 21:49:46.346507 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.346482    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/62000861-c573-4dba-8bca-c6f962057f5b-kube-api-access-rbcgk\" (UniqueName: \"kubernetes.io/projected/62000861-c573-4dba-8bca-c6f962057f5b-kube-api-access-rbcgk\") pod \"62000861-c573-4dba-8bca-c6f962057f5b\" (UID: \"62000861-c573-4dba-8bca-c6f962057f5b\") "
Oct 30 21:49:46.346683 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.346661    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/62000861-c573-4dba-8bca-c6f962057f5b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62000861-c573-4dba-8bca-c6f962057f5b-credential-socket\") pod \"62000861-c573-4dba-8bca-c6f962057f5b\" (UID: \"62000861-c573-4dba-8bca-c6f962057f5b\") "
Oct 30 21:49:46.346915 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.346885    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/62000861-c573-4dba-8bca-c6f962057f5b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62000861-c573-4dba-8bca-c6f962057f5b-istio-data\") pod \"62000861-c573-4dba-8bca-c6f962057f5b\" (UID: \"62000861-c573-4dba-8bca-c6f962057f5b\") "
Oct 30 21:49:46.346995 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.346948    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/62000861-c573-4dba-8bca-c6f962057f5b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62000861-c573-4dba-8bca-c6f962057f5b-workload-socket\") pod \"62000861-c573-4dba-8bca-c6f962057f5b\" (UID: \"62000861-c573-4dba-8bca-c6f962057f5b\") "
Oct 30 21:49:46.346995 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.346986    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/62000861-c573-4dba-8bca-c6f962057f5b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62000861-c573-4dba-8bca-c6f962057f5b-istiod-ca-cert\") pod \"62000861-c573-4dba-8bca-c6f962057f5b\" (UID: \"62000861-c573-4dba-8bca-c6f962057f5b\") "
Oct 30 21:49:46.347108 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.347023    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/62000861-c573-4dba-8bca-c6f962057f5b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62000861-c573-4dba-8bca-c6f962057f5b-istio-podinfo\") pod \"62000861-c573-4dba-8bca-c6f962057f5b\" (UID: \"62000861-c573-4dba-8bca-c6f962057f5b\") "
Oct 30 21:49:46.347155 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.347116    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/62000861-c573-4dba-8bca-c6f962057f5b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62000861-c573-4dba-8bca-c6f962057f5b-istio-envoy\") pod \"62000861-c573-4dba-8bca-c6f962057f5b\" (UID: \"62000861-c573-4dba-8bca-c6f962057f5b\") "
Oct 30 21:49:46.347246 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.347233    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/62000861-c573-4dba-8bca-c6f962057f5b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62000861-c573-4dba-8bca-c6f962057f5b-workload-certs\") pod \"62000861-c573-4dba-8bca-c6f962057f5b\" (UID: \"62000861-c573-4dba-8bca-c6f962057f5b\") "
Oct 30 21:49:46.348450 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.347710    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62000861-c573-4dba-8bca-c6f962057f5b-credential-socket" pod "62000861-c573-4dba-8bca-c6f962057f5b" (UID: "62000861-c573-4dba-8bca-c6f962057f5b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:46.348450 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.348023    5032 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62000861-c573-4dba-8bca-c6f962057f5b-credential-socket\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:46.348625 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.348484    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62000861-c573-4dba-8bca-c6f962057f5b-istio-data" pod "62000861-c573-4dba-8bca-c6f962057f5b" (UID: "62000861-c573-4dba-8bca-c6f962057f5b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:46.349337 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.349067    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62000861-c573-4dba-8bca-c6f962057f5b-workload-certs" pod "62000861-c573-4dba-8bca-c6f962057f5b" (UID: "62000861-c573-4dba-8bca-c6f962057f5b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:46.349337 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.349303    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62000861-c573-4dba-8bca-c6f962057f5b-workload-socket" pod "62000861-c573-4dba-8bca-c6f962057f5b" (UID: "62000861-c573-4dba-8bca-c6f962057f5b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:46.349513 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.349388    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62000861-c573-4dba-8bca-c6f962057f5b-istiod-ca-cert" pod "62000861-c573-4dba-8bca-c6f962057f5b" (UID: "62000861-c573-4dba-8bca-c6f962057f5b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:49:46.352838 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.352777    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/62000861-c573-4dba-8bca-c6f962057f5b-istio-podinfo" pod "62000861-c573-4dba-8bca-c6f962057f5b" (UID: "62000861-c573-4dba-8bca-c6f962057f5b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:49:46.352940 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.352829    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62000861-c573-4dba-8bca-c6f962057f5b-istio-token" pod "62000861-c573-4dba-8bca-c6f962057f5b" (UID: "62000861-c573-4dba-8bca-c6f962057f5b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:49:46.353099 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.353069    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62000861-c573-4dba-8bca-c6f962057f5b-istio-envoy" pod "62000861-c573-4dba-8bca-c6f962057f5b" (UID: "62000861-c573-4dba-8bca-c6f962057f5b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:46.353400 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.353351    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62000861-c573-4dba-8bca-c6f962057f5b-kube-api-access-rbcgk" pod "62000861-c573-4dba-8bca-c6f962057f5b" (UID: "62000861-c573-4dba-8bca-c6f962057f5b"). InnerVolumeSpecName "kube-api-access-rbcgk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:49:46.408068 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.407924    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w0-000f64-6576d48f75-5rdtz" podUID="c9e3ad2b-93c6-46fd-8b0c-87e24852081e" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:49:46.439386 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.426464    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-000f64-85b9f75579-vc4bc" podUID="d3bbb2d6-2cff-40ad-90fa-7e232d26a860" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:49:46.450021 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.449866    5032 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62000861-c573-4dba-8bca-c6f962057f5b-istio-data\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:46.450021 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.449903    5032 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62000861-c573-4dba-8bca-c6f962057f5b-workload-socket\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:46.450021 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.449919    5032 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62000861-c573-4dba-8bca-c6f962057f5b-istiod-ca-cert\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:46.450021 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.449934    5032 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62000861-c573-4dba-8bca-c6f962057f5b-istio-podinfo\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:46.450021 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.449949    5032 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62000861-c573-4dba-8bca-c6f962057f5b-istio-envoy\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:46.450021 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.449964    5032 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62000861-c573-4dba-8bca-c6f962057f5b-workload-certs\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:46.450021 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.449979    5032 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62000861-c573-4dba-8bca-c6f962057f5b-istio-token\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:46.450021 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.449993    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rbcgk\" (UniqueName: \"kubernetes.io/projected/62000861-c573-4dba-8bca-c6f962057f5b-kube-api-access-rbcgk\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:46.473000 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.472973    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s"
Oct 30 21:49:46.485759 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.485725    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww"
Oct 30 21:49:46.491079 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.491049    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2"
Oct 30 21:49:46.550673 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.550642    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/63f89cda-14de-416c-9263-00ffdb8c9ec9-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63f89cda-14de-416c-9263-00ffdb8c9ec9-workload-socket\") pod \"63f89cda-14de-416c-9263-00ffdb8c9ec9\" (UID: \"63f89cda-14de-416c-9263-00ffdb8c9ec9\") "
Oct 30 21:49:46.550900 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.550879    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/57dfd470-9c35-404f-b967-14a9103adeea-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/57dfd470-9c35-404f-b967-14a9103adeea-istio-podinfo\") pod \"57dfd470-9c35-404f-b967-14a9103adeea\" (UID: \"57dfd470-9c35-404f-b967-14a9103adeea\") "
Oct 30 21:49:46.550988 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.550933    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/63f89cda-14de-416c-9263-00ffdb8c9ec9-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63f89cda-14de-416c-9263-00ffdb8c9ec9-istiod-ca-cert\") pod \"63f89cda-14de-416c-9263-00ffdb8c9ec9\" (UID: \"63f89cda-14de-416c-9263-00ffdb8c9ec9\") "
Oct 30 21:49:46.550988 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.550962    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/63f89cda-14de-416c-9263-00ffdb8c9ec9-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63f89cda-14de-416c-9263-00ffdb8c9ec9-istio-data\") pod \"63f89cda-14de-416c-9263-00ffdb8c9ec9\" (UID: \"63f89cda-14de-416c-9263-00ffdb8c9ec9\") "
Oct 30 21:49:46.551089 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.550976    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63f89cda-14de-416c-9263-00ffdb8c9ec9-workload-socket" pod "63f89cda-14de-416c-9263-00ffdb8c9ec9" (UID: "63f89cda-14de-416c-9263-00ffdb8c9ec9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:46.551148 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.551090    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/63f89cda-14de-416c-9263-00ffdb8c9ec9-kube-api-access-spprn\" (UniqueName: \"kubernetes.io/projected/63f89cda-14de-416c-9263-00ffdb8c9ec9-kube-api-access-spprn\") pod \"63f89cda-14de-416c-9263-00ffdb8c9ec9\" (UID: \"63f89cda-14de-416c-9263-00ffdb8c9ec9\") "
Oct 30 21:49:46.551148 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.551128    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/63f89cda-14de-416c-9263-00ffdb8c9ec9-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63f89cda-14de-416c-9263-00ffdb8c9ec9-istio-podinfo\") pod \"63f89cda-14de-416c-9263-00ffdb8c9ec9\" (UID: \"63f89cda-14de-416c-9263-00ffdb8c9ec9\") "
Oct 30 21:49:46.551248 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.551168    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/222c96b4-9c61-49c0-a3ed-450ea7d17b67-kube-api-access-spnlf\" (UniqueName: \"kubernetes.io/projected/222c96b4-9c61-49c0-a3ed-450ea7d17b67-kube-api-access-spnlf\") pod \"222c96b4-9c61-49c0-a3ed-450ea7d17b67\" (UID: \"222c96b4-9c61-49c0-a3ed-450ea7d17b67\") "
Oct 30 21:49:46.551248 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.551201    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/57dfd470-9c35-404f-b967-14a9103adeea-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/57dfd470-9c35-404f-b967-14a9103adeea-workload-certs\") pod \"57dfd470-9c35-404f-b967-14a9103adeea\" (UID: \"57dfd470-9c35-404f-b967-14a9103adeea\") "
Oct 30 21:49:46.551350 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.551258    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/57dfd470-9c35-404f-b967-14a9103adeea-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/57dfd470-9c35-404f-b967-14a9103adeea-istio-envoy\") pod \"57dfd470-9c35-404f-b967-14a9103adeea\" (UID: \"57dfd470-9c35-404f-b967-14a9103adeea\") "
Oct 30 21:49:46.551350 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.551295    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/222c96b4-9c61-49c0-a3ed-450ea7d17b67-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/222c96b4-9c61-49c0-a3ed-450ea7d17b67-istio-envoy\") pod \"222c96b4-9c61-49c0-a3ed-450ea7d17b67\" (UID: \"222c96b4-9c61-49c0-a3ed-450ea7d17b67\") "
Oct 30 21:49:46.551350 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.551327    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/57dfd470-9c35-404f-b967-14a9103adeea-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/57dfd470-9c35-404f-b967-14a9103adeea-istio-data\") pod \"57dfd470-9c35-404f-b967-14a9103adeea\" (UID: \"57dfd470-9c35-404f-b967-14a9103adeea\") "
Oct 30 21:49:46.551521 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.551375    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/57dfd470-9c35-404f-b967-14a9103adeea-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/57dfd470-9c35-404f-b967-14a9103adeea-istiod-ca-cert\") pod \"57dfd470-9c35-404f-b967-14a9103adeea\" (UID: \"57dfd470-9c35-404f-b967-14a9103adeea\") "
Oct 30 21:49:46.551521 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.551414    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/222c96b4-9c61-49c0-a3ed-450ea7d17b67-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/222c96b4-9c61-49c0-a3ed-450ea7d17b67-workload-socket\") pod \"222c96b4-9c61-49c0-a3ed-450ea7d17b67\" (UID: \"222c96b4-9c61-49c0-a3ed-450ea7d17b67\") "
Oct 30 21:49:46.551521 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.551464    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/63f89cda-14de-416c-9263-00ffdb8c9ec9-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63f89cda-14de-416c-9263-00ffdb8c9ec9-istio-envoy\") pod \"63f89cda-14de-416c-9263-00ffdb8c9ec9\" (UID: \"63f89cda-14de-416c-9263-00ffdb8c9ec9\") "
Oct 30 21:49:46.551669 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.551555    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/63f89cda-14de-416c-9263-00ffdb8c9ec9-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63f89cda-14de-416c-9263-00ffdb8c9ec9-workload-certs\") pod \"63f89cda-14de-416c-9263-00ffdb8c9ec9\" (UID: \"63f89cda-14de-416c-9263-00ffdb8c9ec9\") "
Oct 30 21:49:46.551669 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.551585    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/57dfd470-9c35-404f-b967-14a9103adeea-kube-api-access-mnvsp\" (UniqueName: \"kubernetes.io/projected/57dfd470-9c35-404f-b967-14a9103adeea-kube-api-access-mnvsp\") pod \"57dfd470-9c35-404f-b967-14a9103adeea\" (UID: \"57dfd470-9c35-404f-b967-14a9103adeea\") "
Oct 30 21:49:46.551669 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.551637    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/222c96b4-9c61-49c0-a3ed-450ea7d17b67-istio-token\" (UniqueName: \"kubernetes.io/projected/222c96b4-9c61-49c0-a3ed-450ea7d17b67-istio-token\") pod \"222c96b4-9c61-49c0-a3ed-450ea7d17b67\" (UID: \"222c96b4-9c61-49c0-a3ed-450ea7d17b67\") "
Oct 30 21:49:46.551823 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.551683    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/57dfd470-9c35-404f-b967-14a9103adeea-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/57dfd470-9c35-404f-b967-14a9103adeea-workload-socket\") pod \"57dfd470-9c35-404f-b967-14a9103adeea\" (UID: \"57dfd470-9c35-404f-b967-14a9103adeea\") "
Oct 30 21:49:46.551823 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.551716    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/222c96b4-9c61-49c0-a3ed-450ea7d17b67-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/222c96b4-9c61-49c0-a3ed-450ea7d17b67-istiod-ca-cert\") pod \"222c96b4-9c61-49c0-a3ed-450ea7d17b67\" (UID: \"222c96b4-9c61-49c0-a3ed-450ea7d17b67\") "
Oct 30 21:49:46.551823 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.551748    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/63f89cda-14de-416c-9263-00ffdb8c9ec9-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63f89cda-14de-416c-9263-00ffdb8c9ec9-credential-socket\") pod \"63f89cda-14de-416c-9263-00ffdb8c9ec9\" (UID: \"63f89cda-14de-416c-9263-00ffdb8c9ec9\") "
Oct 30 21:49:46.551966 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.551856    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/63f89cda-14de-416c-9263-00ffdb8c9ec9-istio-token\" (UniqueName: \"kubernetes.io/projected/63f89cda-14de-416c-9263-00ffdb8c9ec9-istio-token\") pod \"63f89cda-14de-416c-9263-00ffdb8c9ec9\" (UID: \"63f89cda-14de-416c-9263-00ffdb8c9ec9\") "
Oct 30 21:49:46.551966 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.551950    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/57dfd470-9c35-404f-b967-14a9103adeea-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/57dfd470-9c35-404f-b967-14a9103adeea-credential-socket\") pod \"57dfd470-9c35-404f-b967-14a9103adeea\" (UID: \"57dfd470-9c35-404f-b967-14a9103adeea\") "
Oct 30 21:49:46.552066 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.551984    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/222c96b4-9c61-49c0-a3ed-450ea7d17b67-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/222c96b4-9c61-49c0-a3ed-450ea7d17b67-credential-socket\") pod \"222c96b4-9c61-49c0-a3ed-450ea7d17b67\" (UID: \"222c96b4-9c61-49c0-a3ed-450ea7d17b67\") "
Oct 30 21:49:46.552066 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.552018    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/57dfd470-9c35-404f-b967-14a9103adeea-istio-token\" (UniqueName: \"kubernetes.io/projected/57dfd470-9c35-404f-b967-14a9103adeea-istio-token\") pod \"57dfd470-9c35-404f-b967-14a9103adeea\" (UID: \"57dfd470-9c35-404f-b967-14a9103adeea\") "
Oct 30 21:49:46.552194 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.552062    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/222c96b4-9c61-49c0-a3ed-450ea7d17b67-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/222c96b4-9c61-49c0-a3ed-450ea7d17b67-workload-certs\") pod \"222c96b4-9c61-49c0-a3ed-450ea7d17b67\" (UID: \"222c96b4-9c61-49c0-a3ed-450ea7d17b67\") "
Oct 30 21:49:46.552194 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.552118    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/222c96b4-9c61-49c0-a3ed-450ea7d17b67-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/222c96b4-9c61-49c0-a3ed-450ea7d17b67-istio-data\") pod \"222c96b4-9c61-49c0-a3ed-450ea7d17b67\" (UID: \"222c96b4-9c61-49c0-a3ed-450ea7d17b67\") "
Oct 30 21:49:46.552194 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.552189    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/222c96b4-9c61-49c0-a3ed-450ea7d17b67-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/222c96b4-9c61-49c0-a3ed-450ea7d17b67-istio-podinfo\") pod \"222c96b4-9c61-49c0-a3ed-450ea7d17b67\" (UID: \"222c96b4-9c61-49c0-a3ed-450ea7d17b67\") "
Oct 30 21:49:46.553329 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.552633    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/222c96b4-9c61-49c0-a3ed-450ea7d17b67-workload-socket" pod "222c96b4-9c61-49c0-a3ed-450ea7d17b67" (UID: "222c96b4-9c61-49c0-a3ed-450ea7d17b67"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:46.553329 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.553036    5032 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/222c96b4-9c61-49c0-a3ed-450ea7d17b67-workload-socket\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:46.553329 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.553058    5032 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63f89cda-14de-416c-9263-00ffdb8c9ec9-workload-socket\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:46.554089 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.553738    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63f89cda-14de-416c-9263-00ffdb8c9ec9-istiod-ca-cert" pod "63f89cda-14de-416c-9263-00ffdb8c9ec9" (UID: "63f89cda-14de-416c-9263-00ffdb8c9ec9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:49:46.554089 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.553905    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/57dfd470-9c35-404f-b967-14a9103adeea-istio-podinfo" pod "57dfd470-9c35-404f-b967-14a9103adeea" (UID: "57dfd470-9c35-404f-b967-14a9103adeea"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:49:46.554509 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.554300    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57dfd470-9c35-404f-b967-14a9103adeea-istio-data" pod "57dfd470-9c35-404f-b967-14a9103adeea" (UID: "57dfd470-9c35-404f-b967-14a9103adeea"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:46.554923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.554650    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63f89cda-14de-416c-9263-00ffdb8c9ec9-credential-socket" pod "63f89cda-14de-416c-9263-00ffdb8c9ec9" (UID: "63f89cda-14de-416c-9263-00ffdb8c9ec9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:46.555232 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.555209    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57dfd470-9c35-404f-b967-14a9103adeea-workload-certs" pod "57dfd470-9c35-404f-b967-14a9103adeea" (UID: "57dfd470-9c35-404f-b967-14a9103adeea"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:46.555477 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.555455    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/222c96b4-9c61-49c0-a3ed-450ea7d17b67-credential-socket" pod "222c96b4-9c61-49c0-a3ed-450ea7d17b67" (UID: "222c96b4-9c61-49c0-a3ed-450ea7d17b67"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:46.555635 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.555591    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63f89cda-14de-416c-9263-00ffdb8c9ec9-istio-data" pod "63f89cda-14de-416c-9263-00ffdb8c9ec9" (UID: "63f89cda-14de-416c-9263-00ffdb8c9ec9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:46.555721 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.555660    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57dfd470-9c35-404f-b967-14a9103adeea-credential-socket" pod "57dfd470-9c35-404f-b967-14a9103adeea" (UID: "57dfd470-9c35-404f-b967-14a9103adeea"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:46.556338 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.555789    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57dfd470-9c35-404f-b967-14a9103adeea-workload-socket" pod "57dfd470-9c35-404f-b967-14a9103adeea" (UID: "57dfd470-9c35-404f-b967-14a9103adeea"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:46.556338 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.555837    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57dfd470-9c35-404f-b967-14a9103adeea-istiod-ca-cert" pod "57dfd470-9c35-404f-b967-14a9103adeea" (UID: "57dfd470-9c35-404f-b967-14a9103adeea"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:49:46.556338 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.556054    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63f89cda-14de-416c-9263-00ffdb8c9ec9-workload-certs" pod "63f89cda-14de-416c-9263-00ffdb8c9ec9" (UID: "63f89cda-14de-416c-9263-00ffdb8c9ec9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:46.556338 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.556119    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/222c96b4-9c61-49c0-a3ed-450ea7d17b67-istiod-ca-cert" pod "222c96b4-9c61-49c0-a3ed-450ea7d17b67" (UID: "222c96b4-9c61-49c0-a3ed-450ea7d17b67"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:49:46.556338 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.556197    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/222c96b4-9c61-49c0-a3ed-450ea7d17b67-workload-certs" pod "222c96b4-9c61-49c0-a3ed-450ea7d17b67" (UID: "222c96b4-9c61-49c0-a3ed-450ea7d17b67"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:46.556338 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.556319    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/222c96b4-9c61-49c0-a3ed-450ea7d17b67-istio-data" pod "222c96b4-9c61-49c0-a3ed-450ea7d17b67" (UID: "222c96b4-9c61-49c0-a3ed-450ea7d17b67"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:46.557754 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.557716    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/222c96b4-9c61-49c0-a3ed-450ea7d17b67-kube-api-access-spnlf" pod "222c96b4-9c61-49c0-a3ed-450ea7d17b67" (UID: "222c96b4-9c61-49c0-a3ed-450ea7d17b67"). InnerVolumeSpecName "kube-api-access-spnlf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:49:46.577369 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.577317    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57dfd470-9c35-404f-b967-14a9103adeea-kube-api-access-mnvsp" pod "57dfd470-9c35-404f-b967-14a9103adeea" (UID: "57dfd470-9c35-404f-b967-14a9103adeea"). InnerVolumeSpecName "kube-api-access-mnvsp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:49:46.580311 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.580256    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/222c96b4-9c61-49c0-a3ed-450ea7d17b67-istio-envoy" pod "222c96b4-9c61-49c0-a3ed-450ea7d17b67" (UID: "222c96b4-9c61-49c0-a3ed-450ea7d17b67"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:46.580311 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.580293    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63f89cda-14de-416c-9263-00ffdb8c9ec9-istio-envoy" pod "63f89cda-14de-416c-9263-00ffdb8c9ec9" (UID: "63f89cda-14de-416c-9263-00ffdb8c9ec9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:46.580495 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.580298    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/222c96b4-9c61-49c0-a3ed-450ea7d17b67-istio-token" pod "222c96b4-9c61-49c0-a3ed-450ea7d17b67" (UID: "222c96b4-9c61-49c0-a3ed-450ea7d17b67"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:49:46.580698 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.580662    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57dfd470-9c35-404f-b967-14a9103adeea-istio-token" pod "57dfd470-9c35-404f-b967-14a9103adeea" (UID: "57dfd470-9c35-404f-b967-14a9103adeea"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:49:46.581553 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.581246    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63f89cda-14de-416c-9263-00ffdb8c9ec9-kube-api-access-spprn" pod "63f89cda-14de-416c-9263-00ffdb8c9ec9" (UID: "63f89cda-14de-416c-9263-00ffdb8c9ec9"). InnerVolumeSpecName "kube-api-access-spprn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:49:46.581553 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.581355    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57dfd470-9c35-404f-b967-14a9103adeea-istio-envoy" pod "57dfd470-9c35-404f-b967-14a9103adeea" (UID: "57dfd470-9c35-404f-b967-14a9103adeea"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:46.581553 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.581522    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/63f89cda-14de-416c-9263-00ffdb8c9ec9-istio-podinfo" pod "63f89cda-14de-416c-9263-00ffdb8c9ec9" (UID: "63f89cda-14de-416c-9263-00ffdb8c9ec9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:49:46.582253 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.582226    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63f89cda-14de-416c-9263-00ffdb8c9ec9-istio-token" pod "63f89cda-14de-416c-9263-00ffdb8c9ec9" (UID: "63f89cda-14de-416c-9263-00ffdb8c9ec9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:49:46.583074 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.583049    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/222c96b4-9c61-49c0-a3ed-450ea7d17b67-istio-podinfo" pod "222c96b4-9c61-49c0-a3ed-450ea7d17b67" (UID: "222c96b4-9c61-49c0-a3ed-450ea7d17b67"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:49:46.655283 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.655239    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-spprn\" (UniqueName: \"kubernetes.io/projected/63f89cda-14de-416c-9263-00ffdb8c9ec9-kube-api-access-spprn\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:46.655283 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.655284    5032 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63f89cda-14de-416c-9263-00ffdb8c9ec9-istio-podinfo\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:46.655593 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.655301    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-spnlf\" (UniqueName: \"kubernetes.io/projected/222c96b4-9c61-49c0-a3ed-450ea7d17b67-kube-api-access-spnlf\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:46.655593 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.655314    5032 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/57dfd470-9c35-404f-b967-14a9103adeea-workload-certs\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:46.655593 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.655327    5032 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/57dfd470-9c35-404f-b967-14a9103adeea-istio-envoy\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:46.655593 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.655340    5032 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/222c96b4-9c61-49c0-a3ed-450ea7d17b67-istio-envoy\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:46.655593 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.655354    5032 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/57dfd470-9c35-404f-b967-14a9103adeea-istio-data\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:46.655593 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.655386    5032 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/57dfd470-9c35-404f-b967-14a9103adeea-istiod-ca-cert\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:46.655593 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.655399    5032 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63f89cda-14de-416c-9263-00ffdb8c9ec9-istio-envoy\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:46.655593 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.655414    5032 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63f89cda-14de-416c-9263-00ffdb8c9ec9-workload-certs\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:46.655593 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.655426    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mnvsp\" (UniqueName: \"kubernetes.io/projected/57dfd470-9c35-404f-b967-14a9103adeea-kube-api-access-mnvsp\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:46.655593 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.655439    5032 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/222c96b4-9c61-49c0-a3ed-450ea7d17b67-istio-token\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:46.655593 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.655452    5032 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/57dfd470-9c35-404f-b967-14a9103adeea-workload-socket\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:46.655593 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.655464    5032 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/222c96b4-9c61-49c0-a3ed-450ea7d17b67-istiod-ca-cert\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:46.655593 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.655478    5032 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63f89cda-14de-416c-9263-00ffdb8c9ec9-credential-socket\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:46.655593 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.655492    5032 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63f89cda-14de-416c-9263-00ffdb8c9ec9-istio-token\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:46.655593 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.655505    5032 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/57dfd470-9c35-404f-b967-14a9103adeea-credential-socket\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:46.655593 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.655518    5032 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/222c96b4-9c61-49c0-a3ed-450ea7d17b67-credential-socket\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:46.655593 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.655533    5032 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57dfd470-9c35-404f-b967-14a9103adeea-istio-token\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:46.655593 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.655547    5032 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/222c96b4-9c61-49c0-a3ed-450ea7d17b67-workload-certs\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:46.655593 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.655562    5032 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/222c96b4-9c61-49c0-a3ed-450ea7d17b67-istio-data\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:46.655593 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.655575    5032 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/222c96b4-9c61-49c0-a3ed-450ea7d17b67-istio-podinfo\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:46.655593 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.655584    5032 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/57dfd470-9c35-404f-b967-14a9103adeea-istio-podinfo\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:46.655593 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.655593    5032 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63f89cda-14de-416c-9263-00ffdb8c9ec9-istiod-ca-cert\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:46.655593 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.655602    5032 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63f89cda-14de-416c-9263-00ffdb8c9ec9-istio-data\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:46.682470 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.676501    5032 generic.go:356] "Generic (PLEG): container finished" podID="62000861-c573-4dba-8bca-c6f962057f5b" containerID="ff7e7fa1ccecb3bf7d7b3431f975cbaf5fa1442a004336175284517e055f4b07" exitCode=0
Oct 30 21:49:46.682470 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.676631    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2" event={"ID":"62000861-c573-4dba-8bca-c6f962057f5b","Type":"ContainerDied","Data":"ff7e7fa1ccecb3bf7d7b3431f975cbaf5fa1442a004336175284517e055f4b07"}
Oct 30 21:49:46.682470 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.676683    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2" event={"ID":"62000861-c573-4dba-8bca-c6f962057f5b","Type":"ContainerDied","Data":"e79ccd071d9d11c318b375907cae2cdd951e23e7d5ff2def6f3b7bdd6274dadd"}
Oct 30 21:49:46.682470 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.676706    5032 scope.go:122] "RemoveContainer" containerID="ff7e7fa1ccecb3bf7d7b3431f975cbaf5fa1442a004336175284517e055f4b07"
Oct 30 21:49:46.682470 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.676802    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2"
Oct 30 21:49:46.682470 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.680527    5032 generic.go:356] "Generic (PLEG): container finished" podID="222c96b4-9c61-49c0-a3ed-450ea7d17b67" containerID="a0cc01dd8a6e2215c301be962219d059ff1634bf72bac7e9f43fab59f4f5abc4" exitCode=0
Oct 30 21:49:46.682470 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.680611    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s" event={"ID":"222c96b4-9c61-49c0-a3ed-450ea7d17b67","Type":"ContainerDied","Data":"a0cc01dd8a6e2215c301be962219d059ff1634bf72bac7e9f43fab59f4f5abc4"}
Oct 30 21:49:46.682470 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.680657    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s" event={"ID":"222c96b4-9c61-49c0-a3ed-450ea7d17b67","Type":"ContainerDied","Data":"ff8c47fbec44be4bdd66bfe02c1b6e7640300faf5da51e3d303845f08fec717b"}
Oct 30 21:49:46.682470 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.680740    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s"
Oct 30 21:49:46.686049 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.686024    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n"
Oct 30 21:49:46.686049 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.686038    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n" event={"ID":"74856e06-9b6f-425d-90f8-e973d0649f90","Type":"ContainerDied","Data":"e14161b3b44b755cfa03f984976e4fb752d989d61df4dec8db5af67806ee9f27"}
Oct 30 21:49:46.691252 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.691221    5032 generic.go:356] "Generic (PLEG): container finished" podID="57dfd470-9c35-404f-b967-14a9103adeea" containerID="573e81d49ed938098ecb7f82d952e67c858bdee65faa9ad51332ddd424dcdbcd" exitCode=0
Oct 30 21:49:46.691885 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.691416    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2" event={"ID":"57dfd470-9c35-404f-b967-14a9103adeea","Type":"ContainerDied","Data":"573e81d49ed938098ecb7f82d952e67c858bdee65faa9ad51332ddd424dcdbcd"}
Oct 30 21:49:46.691885 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.691463    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2" event={"ID":"57dfd470-9c35-404f-b967-14a9103adeea","Type":"ContainerDied","Data":"0b0069e866b89018531851b1bcb7d9e9a190c17843389d89e4f289d18020ba7f"}
Oct 30 21:49:46.691885 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.691470    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2"
Oct 30 21:49:46.694218 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.693304    5032 generic.go:356] "Generic (PLEG): container finished" podID="63f89cda-14de-416c-9263-00ffdb8c9ec9" containerID="329f325c035c1901134d37ebff509d6feb27eecda94f08290199a7b843bbb21a" exitCode=0
Oct 30 21:49:46.694218 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.694001    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww"
Oct 30 21:49:46.694218 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.694078    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww" event={"ID":"63f89cda-14de-416c-9263-00ffdb8c9ec9","Type":"ContainerDied","Data":"329f325c035c1901134d37ebff509d6feb27eecda94f08290199a7b843bbb21a"}
Oct 30 21:49:46.694218 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.694120    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww" event={"ID":"63f89cda-14de-416c-9263-00ffdb8c9ec9","Type":"ContainerDied","Data":"030bed24ac41c7677b3b80d59769dae999f72bbb8567d871b500c5754906c681"}
Oct 30 21:49:46.696182 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.696160    5032 scope.go:122] "RemoveContainer" containerID="ff7e7fa1ccecb3bf7d7b3431f975cbaf5fa1442a004336175284517e055f4b07"
Oct 30 21:49:46.696761 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:49:46.696732    5032 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff7e7fa1ccecb3bf7d7b3431f975cbaf5fa1442a004336175284517e055f4b07\": container with ID starting with ff7e7fa1ccecb3bf7d7b3431f975cbaf5fa1442a004336175284517e055f4b07 not found: ID does not exist" containerID="ff7e7fa1ccecb3bf7d7b3431f975cbaf5fa1442a004336175284517e055f4b07"
Oct 30 21:49:46.696835 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.696772    5032 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff7e7fa1ccecb3bf7d7b3431f975cbaf5fa1442a004336175284517e055f4b07"} err="failed to get container status \"ff7e7fa1ccecb3bf7d7b3431f975cbaf5fa1442a004336175284517e055f4b07\": rpc error: code = NotFound desc = could not find container \"ff7e7fa1ccecb3bf7d7b3431f975cbaf5fa1442a004336175284517e055f4b07\": container with ID starting with ff7e7fa1ccecb3bf7d7b3431f975cbaf5fa1442a004336175284517e055f4b07 not found: ID does not exist"
Oct 30 21:49:46.696835 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.696793    5032 scope.go:122] "RemoveContainer" containerID="a0cc01dd8a6e2215c301be962219d059ff1634bf72bac7e9f43fab59f4f5abc4"
Oct 30 21:49:46.748883 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.744602    5032 scope.go:122] "RemoveContainer" containerID="a0cc01dd8a6e2215c301be962219d059ff1634bf72bac7e9f43fab59f4f5abc4"
Oct 30 21:49:46.748883 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:49:46.746160    5032 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0cc01dd8a6e2215c301be962219d059ff1634bf72bac7e9f43fab59f4f5abc4\": container with ID starting with a0cc01dd8a6e2215c301be962219d059ff1634bf72bac7e9f43fab59f4f5abc4 not found: ID does not exist" containerID="a0cc01dd8a6e2215c301be962219d059ff1634bf72bac7e9f43fab59f4f5abc4"
Oct 30 21:49:46.748883 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.746200    5032 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0cc01dd8a6e2215c301be962219d059ff1634bf72bac7e9f43fab59f4f5abc4"} err="failed to get container status \"a0cc01dd8a6e2215c301be962219d059ff1634bf72bac7e9f43fab59f4f5abc4\": rpc error: code = NotFound desc = could not find container \"a0cc01dd8a6e2215c301be962219d059ff1634bf72bac7e9f43fab59f4f5abc4\": container with ID starting with a0cc01dd8a6e2215c301be962219d059ff1634bf72bac7e9f43fab59f4f5abc4 not found: ID does not exist"
Oct 30 21:49:46.748883 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.746244    5032 scope.go:122] "RemoveContainer" containerID="0b6751f9d63c1c06407f8c38e60aa49c0ccd14d070648e6e8d7e55147b6c1aac"
Oct 30 21:49:46.753639 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.753479    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2"]
Oct 30 21:49:46.761349 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.761327    5032 scope.go:122] "RemoveContainer" containerID="573e81d49ed938098ecb7f82d952e67c858bdee65faa9ad51332ddd424dcdbcd"
Oct 30 21:49:46.762587 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.762556    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-zf5l2"]
Oct 30 21:49:46.778306 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.778259    5032 scope.go:122] "RemoveContainer" containerID="573e81d49ed938098ecb7f82d952e67c858bdee65faa9ad51332ddd424dcdbcd"
Oct 30 21:49:46.779992 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:49:46.779334    5032 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"573e81d49ed938098ecb7f82d952e67c858bdee65faa9ad51332ddd424dcdbcd\": container with ID starting with 573e81d49ed938098ecb7f82d952e67c858bdee65faa9ad51332ddd424dcdbcd not found: ID does not exist" containerID="573e81d49ed938098ecb7f82d952e67c858bdee65faa9ad51332ddd424dcdbcd"
Oct 30 21:49:46.779992 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.779443    5032 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"573e81d49ed938098ecb7f82d952e67c858bdee65faa9ad51332ddd424dcdbcd"} err="failed to get container status \"573e81d49ed938098ecb7f82d952e67c858bdee65faa9ad51332ddd424dcdbcd\": rpc error: code = NotFound desc = could not find container \"573e81d49ed938098ecb7f82d952e67c858bdee65faa9ad51332ddd424dcdbcd\": container with ID starting with 573e81d49ed938098ecb7f82d952e67c858bdee65faa9ad51332ddd424dcdbcd not found: ID does not exist"
Oct 30 21:49:46.779992 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.779663    5032 scope.go:122] "RemoveContainer" containerID="329f325c035c1901134d37ebff509d6feb27eecda94f08290199a7b843bbb21a"
Oct 30 21:49:46.779992 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.779785    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s"]
Oct 30 21:49:46.784226 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.784204    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-8mn6s"]
Oct 30 21:49:46.793327 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.793306    5032 scope.go:122] "RemoveContainer" containerID="329f325c035c1901134d37ebff509d6feb27eecda94f08290199a7b843bbb21a"
Oct 30 21:49:46.793713 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:49:46.793687    5032 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"329f325c035c1901134d37ebff509d6feb27eecda94f08290199a7b843bbb21a\": container with ID starting with 329f325c035c1901134d37ebff509d6feb27eecda94f08290199a7b843bbb21a not found: ID does not exist" containerID="329f325c035c1901134d37ebff509d6feb27eecda94f08290199a7b843bbb21a"
Oct 30 21:49:46.793807 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.793723    5032 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"329f325c035c1901134d37ebff509d6feb27eecda94f08290199a7b843bbb21a"} err="failed to get container status \"329f325c035c1901134d37ebff509d6feb27eecda94f08290199a7b843bbb21a\": rpc error: code = NotFound desc = could not find container \"329f325c035c1901134d37ebff509d6feb27eecda94f08290199a7b843bbb21a\": container with ID starting with 329f325c035c1901134d37ebff509d6feb27eecda94f08290199a7b843bbb21a not found: ID does not exist"
Oct 30 21:49:46.866935 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.866889    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2"]
Oct 30 21:49:46.882869 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.882814    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-v57j2"]
Oct 30 21:49:46.888037 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.888010    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww"]
Oct 30 21:49:46.891654 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.891634    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-7dfff0-openshift-default-559bb574c5-b4lww"]
Oct 30 21:49:46.896002 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.895982    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n"]
Oct 30 21:49:46.899688 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:46.899669    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-pkv5n"]
Oct 30 21:49:47.151435 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:47.151400    5032 patch_prober.go:29] interesting pod/e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused" start-of-body=
Oct 30 21:49:47.151599 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:47.151469    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb" podUID="629450ab-662b-43c5-b93d-e264b1ce2034" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused"
Oct 30 21:49:47.350055 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:47.349978    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="222c96b4-9c61-49c0-a3ed-450ea7d17b67" path="/var/lib/kubelet/pods/222c96b4-9c61-49c0-a3ed-450ea7d17b67/volumes"
Oct 30 21:49:47.350623 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:47.350600    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3b92d898-765d-445d-9056-9f256030162e" path="/var/lib/kubelet/pods/3b92d898-765d-445d-9056-9f256030162e/volumes"
Oct 30 21:49:47.351216 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:47.351196    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="57dfd470-9c35-404f-b967-14a9103adeea" path="/var/lib/kubelet/pods/57dfd470-9c35-404f-b967-14a9103adeea/volumes"
Oct 30 21:49:47.351996 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:47.351979    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="62000861-c573-4dba-8bca-c6f962057f5b" path="/var/lib/kubelet/pods/62000861-c573-4dba-8bca-c6f962057f5b/volumes"
Oct 30 21:49:47.352469 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:47.352453    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="63f89cda-14de-416c-9263-00ffdb8c9ec9" path="/var/lib/kubelet/pods/63f89cda-14de-416c-9263-00ffdb8c9ec9/volumes"
Oct 30 21:49:47.352923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:47.352906    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="74856e06-9b6f-425d-90f8-e973d0649f90" path="/var/lib/kubelet/pods/74856e06-9b6f-425d-90f8-e973d0649f90/volumes"
Oct 30 21:49:47.473480 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:47.473442    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-000f64-765dc8c9bf-lxnrx"]
Oct 30 21:49:47.473739 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:47.473711    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-000f64-765dc8c9bf-lxnrx" podUID="96a36ed7-d92e-4748-96b8-42367bcb1b48" containerName="maas-api" containerID="cri-o://a8ba673e641b1a3a5d0e747639761d8b75fe535f9dd360aedb2bd10a6fe05297" gracePeriod=30
Oct 30 21:49:47.823778 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:47.823540    5032 generic.go:356] "Generic (PLEG): container finished" podID="96a36ed7-d92e-4748-96b8-42367bcb1b48" containerID="a8ba673e641b1a3a5d0e747639761d8b75fe535f9dd360aedb2bd10a6fe05297" exitCode=0
Oct 30 21:49:47.823778 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:47.823645    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-000f64-765dc8c9bf-lxnrx" event={"ID":"96a36ed7-d92e-4748-96b8-42367bcb1b48","Type":"ContainerDied","Data":"a8ba673e641b1a3a5d0e747639761d8b75fe535f9dd360aedb2bd10a6fe05297"}
Oct 30 21:49:48.085653 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:48.080814    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-000f64-765dc8c9bf-lxnrx"
Oct 30 21:49:48.099897 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:48.099866    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/96a36ed7-d92e-4748-96b8-42367bcb1b48-kube-api-access-zvrpl\" (UniqueName: \"kubernetes.io/projected/96a36ed7-d92e-4748-96b8-42367bcb1b48-kube-api-access-zvrpl\") pod \"96a36ed7-d92e-4748-96b8-42367bcb1b48\" (UID: \"96a36ed7-d92e-4748-96b8-42367bcb1b48\") "
Oct 30 21:49:48.100511 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:48.100486    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/96a36ed7-d92e-4748-96b8-42367bcb1b48-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/96a36ed7-d92e-4748-96b8-42367bcb1b48-openshift-service-ca\") pod \"96a36ed7-d92e-4748-96b8-42367bcb1b48\" (UID: \"96a36ed7-d92e-4748-96b8-42367bcb1b48\") "
Oct 30 21:49:48.101224 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:48.101033    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96a36ed7-d92e-4748-96b8-42367bcb1b48-openshift-service-ca" pod "96a36ed7-d92e-4748-96b8-42367bcb1b48" (UID: "96a36ed7-d92e-4748-96b8-42367bcb1b48"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:49:48.102015 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:48.101991    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/96a36ed7-d92e-4748-96b8-42367bcb1b48-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96a36ed7-d92e-4748-96b8-42367bcb1b48-maas-api-tls\") pod \"96a36ed7-d92e-4748-96b8-42367bcb1b48\" (UID: \"96a36ed7-d92e-4748-96b8-42367bcb1b48\") "
Oct 30 21:49:48.102759 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:48.102724    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/96a36ed7-d92e-4748-96b8-42367bcb1b48-metrics-certs\" (UniqueName: \"kubernetes.io/secret/96a36ed7-d92e-4748-96b8-42367bcb1b48-metrics-certs\") pod \"96a36ed7-d92e-4748-96b8-42367bcb1b48\" (UID: \"96a36ed7-d92e-4748-96b8-42367bcb1b48\") "
Oct 30 21:49:48.104922 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:48.104898    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96a36ed7-d92e-4748-96b8-42367bcb1b48-maas-api-tls" pod "96a36ed7-d92e-4748-96b8-42367bcb1b48" (UID: "96a36ed7-d92e-4748-96b8-42367bcb1b48"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:49:48.106099 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:48.104933    5032 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/96a36ed7-d92e-4748-96b8-42367bcb1b48-openshift-service-ca\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:48.108001 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:48.107975    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96a36ed7-d92e-4748-96b8-42367bcb1b48-kube-api-access-zvrpl" pod "96a36ed7-d92e-4748-96b8-42367bcb1b48" (UID: "96a36ed7-d92e-4748-96b8-42367bcb1b48"). InnerVolumeSpecName "kube-api-access-zvrpl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:49:48.109168 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:48.109111    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96a36ed7-d92e-4748-96b8-42367bcb1b48-metrics-certs" pod "96a36ed7-d92e-4748-96b8-42367bcb1b48" (UID: "96a36ed7-d92e-4748-96b8-42367bcb1b48"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:49:48.159809 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:48.159762    5032 patch_prober.go:29] interesting pod/e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused" start-of-body=
Oct 30 21:49:48.160065 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:48.160038    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb" podUID="629450ab-662b-43c5-b93d-e264b1ce2034" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused"
Oct 30 21:49:48.268878 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:48.268848    5032 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96a36ed7-d92e-4748-96b8-42367bcb1b48-maas-api-tls\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:48.269049 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:48.269038    5032 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96a36ed7-d92e-4748-96b8-42367bcb1b48-metrics-certs\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:48.269107 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:48.269098    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zvrpl\" (UniqueName: \"kubernetes.io/projected/96a36ed7-d92e-4748-96b8-42367bcb1b48-kube-api-access-zvrpl\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:48.382489 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:49:48.382453    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 21:49:48.850034 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:48.849643    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-000f64-85b9f75579-vc4bc"
Oct 30 21:49:49.123353 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:49.121934    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d3bbb2d6-2cff-40ad-90fa-7e232d26a860-kube-api-access-45np9\" (UniqueName: \"kubernetes.io/projected/d3bbb2d6-2cff-40ad-90fa-7e232d26a860-kube-api-access-45np9\") pod \"d3bbb2d6-2cff-40ad-90fa-7e232d26a860\" (UID: \"d3bbb2d6-2cff-40ad-90fa-7e232d26a860\") "
Oct 30 21:49:49.123353 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:49.122036    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d3bbb2d6-2cff-40ad-90fa-7e232d26a860-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d3bbb2d6-2cff-40ad-90fa-7e232d26a860-plugins-config-volume\") pod \"d3bbb2d6-2cff-40ad-90fa-7e232d26a860\" (UID: \"d3bbb2d6-2cff-40ad-90fa-7e232d26a860\") "
Oct 30 21:49:49.123976 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:49.123934    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3bbb2d6-2cff-40ad-90fa-7e232d26a860-plugins-config-volume" pod "d3bbb2d6-2cff-40ad-90fa-7e232d26a860" (UID: "d3bbb2d6-2cff-40ad-90fa-7e232d26a860"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:49:49.129852 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:49.129423    5032 generic.go:356] "Generic (PLEG): container finished" podID="c9e3ad2b-93c6-46fd-8b0c-87e24852081e" containerID="e23310c51897580dcafb822b29983854a16210e0688ddc6da4a082275bc747df" exitCode=0
Oct 30 21:49:49.130143 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:49.130054    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-000f64-6576d48f75-5rdtz" event={"ID":"c9e3ad2b-93c6-46fd-8b0c-87e24852081e","Type":"ContainerDied","Data":"e23310c51897580dcafb822b29983854a16210e0688ddc6da4a082275bc747df"}
Oct 30 21:49:49.130242 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:49.130179    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3bbb2d6-2cff-40ad-90fa-7e232d26a860-kube-api-access-45np9" pod "d3bbb2d6-2cff-40ad-90fa-7e232d26a860" (UID: "d3bbb2d6-2cff-40ad-90fa-7e232d26a860"). InnerVolumeSpecName "kube-api-access-45np9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:49:49.137417 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:49.132784    5032 generic.go:356] "Generic (PLEG): container finished" podID="d3bbb2d6-2cff-40ad-90fa-7e232d26a860" containerID="b31f5ee41ac60d32dab66e4d10fbb926e533ab4ff94a8d34169a9921248b9d15" exitCode=0
Oct 30 21:49:49.137417 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:49.132885    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-000f64-85b9f75579-vc4bc" event={"ID":"d3bbb2d6-2cff-40ad-90fa-7e232d26a860","Type":"ContainerDied","Data":"b31f5ee41ac60d32dab66e4d10fbb926e533ab4ff94a8d34169a9921248b9d15"}
Oct 30 21:49:49.137417 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:49.132927    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-000f64-85b9f75579-vc4bc" event={"ID":"d3bbb2d6-2cff-40ad-90fa-7e232d26a860","Type":"ContainerDied","Data":"88c9f14c35b9515a951c8525c3c0c4f05ec7e0a549c6b67c1cc62f85853c463f"}
Oct 30 21:49:49.137417 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:49.132958    5032 scope.go:122] "RemoveContainer" containerID="b31f5ee41ac60d32dab66e4d10fbb926e533ab4ff94a8d34169a9921248b9d15"
Oct 30 21:49:49.137417 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:49.133100    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-000f64-85b9f75579-vc4bc"
Oct 30 21:49:49.156618 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:49.156583    5032 patch_prober.go:29] interesting pod/e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused" start-of-body=
Oct 30 21:49:49.156923 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:49.156890    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb" podUID="629450ab-662b-43c5-b93d-e264b1ce2034" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused"
Oct 30 21:49:49.163553 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:49.160070    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-000f64-765dc8c9bf-lxnrx" event={"ID":"96a36ed7-d92e-4748-96b8-42367bcb1b48","Type":"ContainerDied","Data":"3ece370df703018acd4c8f42e46c94c7122e197adece2b2d5043a1f02d8646d6"}
Oct 30 21:49:49.163553 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:49.160187    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-000f64-765dc8c9bf-lxnrx"
Oct 30 21:49:49.168446 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:49.168425    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-000f64-6576d48f75-5rdtz"
Oct 30 21:49:49.185300 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:49.185276    5032 scope.go:122] "RemoveContainer" containerID="b31f5ee41ac60d32dab66e4d10fbb926e533ab4ff94a8d34169a9921248b9d15"
Oct 30 21:49:49.185703 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:49:49.185679    5032 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b31f5ee41ac60d32dab66e4d10fbb926e533ab4ff94a8d34169a9921248b9d15\": container with ID starting with b31f5ee41ac60d32dab66e4d10fbb926e533ab4ff94a8d34169a9921248b9d15 not found: ID does not exist" containerID="b31f5ee41ac60d32dab66e4d10fbb926e533ab4ff94a8d34169a9921248b9d15"
Oct 30 21:49:49.185806 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:49.185715    5032 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b31f5ee41ac60d32dab66e4d10fbb926e533ab4ff94a8d34169a9921248b9d15"} err="failed to get container status \"b31f5ee41ac60d32dab66e4d10fbb926e533ab4ff94a8d34169a9921248b9d15\": rpc error: code = NotFound desc = could not find container \"b31f5ee41ac60d32dab66e4d10fbb926e533ab4ff94a8d34169a9921248b9d15\": container with ID starting with b31f5ee41ac60d32dab66e4d10fbb926e533ab4ff94a8d34169a9921248b9d15 not found: ID does not exist"
Oct 30 21:49:49.185806 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:49.185739    5032 scope.go:122] "RemoveContainer" containerID="a8ba673e641b1a3a5d0e747639761d8b75fe535f9dd360aedb2bd10a6fe05297"
Oct 30 21:49:49.241386 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:49.234611    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-45np9\" (UniqueName: \"kubernetes.io/projected/d3bbb2d6-2cff-40ad-90fa-7e232d26a860-kube-api-access-45np9\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:49.241386 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:49.234652    5032 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d3bbb2d6-2cff-40ad-90fa-7e232d26a860-plugins-config-volume\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:49.245442 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:49.245405    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-000f64-765dc8c9bf-lxnrx"]
Oct 30 21:49:49.250169 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:49.250138    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-000f64-765dc8c9bf-lxnrx"]
Oct 30 21:49:49.282821 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:49.282740    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-000f64-85b9f75579-vc4bc"]
Oct 30 21:49:49.301477 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:49.301443    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-000f64-85b9f75579-vc4bc"]
Oct 30 21:49:49.359001 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:49.358966    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c9e3ad2b-93c6-46fd-8b0c-87e24852081e-kube-api-access-szmvv\" (UniqueName: \"kubernetes.io/projected/c9e3ad2b-93c6-46fd-8b0c-87e24852081e-kube-api-access-szmvv\") pod \"c9e3ad2b-93c6-46fd-8b0c-87e24852081e\" (UID: \"c9e3ad2b-93c6-46fd-8b0c-87e24852081e\") "
Oct 30 21:49:49.359192 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:49.359181    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c9e3ad2b-93c6-46fd-8b0c-87e24852081e-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c9e3ad2b-93c6-46fd-8b0c-87e24852081e-plugins-config-volume\") pod \"c9e3ad2b-93c6-46fd-8b0c-87e24852081e\" (UID: \"c9e3ad2b-93c6-46fd-8b0c-87e24852081e\") "
Oct 30 21:49:49.360209 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:49.360181    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9e3ad2b-93c6-46fd-8b0c-87e24852081e-plugins-config-volume" pod "c9e3ad2b-93c6-46fd-8b0c-87e24852081e" (UID: "c9e3ad2b-93c6-46fd-8b0c-87e24852081e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:49:49.366157 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:49.366132    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9e3ad2b-93c6-46fd-8b0c-87e24852081e-kube-api-access-szmvv" pod "c9e3ad2b-93c6-46fd-8b0c-87e24852081e" (UID: "c9e3ad2b-93c6-46fd-8b0c-87e24852081e"). InnerVolumeSpecName "kube-api-access-szmvv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:49:49.370435 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:49.370414    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="96a36ed7-d92e-4748-96b8-42367bcb1b48" path="/var/lib/kubelet/pods/96a36ed7-d92e-4748-96b8-42367bcb1b48/volumes"
Oct 30 21:49:49.371023 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:49.370996    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d3bbb2d6-2cff-40ad-90fa-7e232d26a860" path="/var/lib/kubelet/pods/d3bbb2d6-2cff-40ad-90fa-7e232d26a860/volumes"
Oct 30 21:49:49.461235 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:49.461165    5032 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c9e3ad2b-93c6-46fd-8b0c-87e24852081e-plugins-config-volume\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:49.461235 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:49.461193    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-szmvv\" (UniqueName: \"kubernetes.io/projected/c9e3ad2b-93c6-46fd-8b0c-87e24852081e-kube-api-access-szmvv\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:50.151896 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:50.151864    5032 patch_prober.go:29] interesting pod/e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused" start-of-body=
Oct 30 21:49:50.151896 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:50.151924    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb" podUID="629450ab-662b-43c5-b93d-e264b1ce2034" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused"
Oct 30 21:49:50.172106 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:50.172078    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-000f64-6576d48f75-5rdtz"
Oct 30 21:49:50.172276 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:50.172103    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-000f64-6576d48f75-5rdtz" event={"ID":"c9e3ad2b-93c6-46fd-8b0c-87e24852081e","Type":"ContainerDied","Data":"4747ecc63ff2222acfb576b0c4a2972692df92bac39f96997695717b015256b1"}
Oct 30 21:49:50.172276 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:50.172169    5032 scope.go:122] "RemoveContainer" containerID="e23310c51897580dcafb822b29983854a16210e0688ddc6da4a082275bc747df"
Oct 30 21:49:50.193707 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:50.193679    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-000f64-6576d48f75-5rdtz"]
Oct 30 21:49:50.198031 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:50.198005    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-000f64-6576d48f75-5rdtz"]
Oct 30 21:49:51.151599 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:51.151564    5032 patch_prober.go:29] interesting pod/e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused" start-of-body=
Oct 30 21:49:51.152042 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:51.151621    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb" podUID="629450ab-662b-43c5-b93d-e264b1ce2034" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused"
Oct 30 21:49:51.350848 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:51.350809    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c9e3ad2b-93c6-46fd-8b0c-87e24852081e" path="/var/lib/kubelet/pods/c9e3ad2b-93c6-46fd-8b0c-87e24852081e/volumes"
Oct 30 21:49:52.155785 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:52.155752    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb"
Oct 30 21:49:52.156208 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:52.156186    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb"
Oct 30 21:49:52.157019 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:52.156998    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb"
Oct 30 21:49:55.144492 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:55.144443    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f608f7-7f85c8fcc-lts5r"]
Oct 30 21:49:55.146272 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:55.146241    5032 cpu_manager.go:414] "containerMap: removing container" podUID="222c96b4-9c61-49c0-a3ed-450ea7d17b67" containerName="istio-proxy"
Oct 30 21:49:55.146272 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:55.146272    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="222c96b4-9c61-49c0-a3ed-450ea7d17b67" containerName="istio-proxy"
Oct 30 21:49:55.146272 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:55.146311    5032 cpu_manager.go:414] "containerMap: removing container" podUID="96a36ed7-d92e-4748-96b8-42367bcb1b48" containerName="maas-api"
Oct 30 21:49:55.146272 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:55.146319    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="96a36ed7-d92e-4748-96b8-42367bcb1b48" containerName="maas-api"
Oct 30 21:49:55.146689 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:55.146339    5032 cpu_manager.go:414] "containerMap: removing container" podUID="62000861-c573-4dba-8bca-c6f962057f5b" containerName="istio-proxy"
Oct 30 21:49:55.146689 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:55.146348    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="62000861-c573-4dba-8bca-c6f962057f5b" containerName="istio-proxy"
Oct 30 21:49:55.146689 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:55.146381    5032 cpu_manager.go:414] "containerMap: removing container" podUID="57dfd470-9c35-404f-b967-14a9103adeea" containerName="istio-proxy"
Oct 30 21:49:55.146689 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:55.146390    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="57dfd470-9c35-404f-b967-14a9103adeea" containerName="istio-proxy"
Oct 30 21:49:55.146689 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:55.146438    5032 cpu_manager.go:414] "containerMap: removing container" podUID="63f89cda-14de-416c-9263-00ffdb8c9ec9" containerName="istio-proxy"
Oct 30 21:49:55.146689 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:55.146446    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="63f89cda-14de-416c-9263-00ffdb8c9ec9" containerName="istio-proxy"
Oct 30 21:49:55.146689 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:55.146513    5032 cpu_manager.go:414] "containerMap: removing container" podUID="74856e06-9b6f-425d-90f8-e973d0649f90" containerName="istio-proxy"
Oct 30 21:49:55.146689 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:55.146521    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="74856e06-9b6f-425d-90f8-e973d0649f90" containerName="istio-proxy"
Oct 30 21:49:55.146689 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:55.146581    5032 cpu_manager.go:414] "containerMap: removing container" podUID="c9e3ad2b-93c6-46fd-8b0c-87e24852081e" containerName="payload-processing"
Oct 30 21:49:55.146689 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:55.146590    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c9e3ad2b-93c6-46fd-8b0c-87e24852081e" containerName="payload-processing"
Oct 30 21:49:55.146689 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:55.146601    5032 cpu_manager.go:414] "containerMap: removing container" podUID="3b92d898-765d-445d-9056-9f256030162e" containerName="istio-proxy"
Oct 30 21:49:55.146689 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:55.146609    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3b92d898-765d-445d-9056-9f256030162e" containerName="istio-proxy"
Oct 30 21:49:55.146689 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:55.146621    5032 cpu_manager.go:414] "containerMap: removing container" podUID="d3bbb2d6-2cff-40ad-90fa-7e232d26a860" containerName="payload-pre-processing"
Oct 30 21:49:55.146689 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:55.146630    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d3bbb2d6-2cff-40ad-90fa-7e232d26a860" containerName="payload-pre-processing"
Oct 30 21:49:55.147106 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:55.146937    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="96a36ed7-d92e-4748-96b8-42367bcb1b48" containerName="maas-api"
Oct 30 21:49:55.147106 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:55.146961    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="62000861-c573-4dba-8bca-c6f962057f5b" containerName="istio-proxy"
Oct 30 21:49:55.147106 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:55.146979    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="57dfd470-9c35-404f-b967-14a9103adeea" containerName="istio-proxy"
Oct 30 21:49:55.147106 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:55.146991    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="63f89cda-14de-416c-9263-00ffdb8c9ec9" containerName="istio-proxy"
Oct 30 21:49:55.147106 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:55.147014    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="74856e06-9b6f-425d-90f8-e973d0649f90" containerName="istio-proxy"
Oct 30 21:49:55.147106 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:55.147032    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="c9e3ad2b-93c6-46fd-8b0c-87e24852081e" containerName="payload-processing"
Oct 30 21:49:55.147106 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:55.147042    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="3b92d898-765d-445d-9056-9f256030162e" containerName="istio-proxy"
Oct 30 21:49:55.147106 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:55.147056    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="d3bbb2d6-2cff-40ad-90fa-7e232d26a860" containerName="payload-pre-processing"
Oct 30 21:49:55.147106 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:55.147077    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="222c96b4-9c61-49c0-a3ed-450ea7d17b67" containerName="istio-proxy"
Oct 30 21:49:55.153165 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:55.153135    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f608f7-7f85c8fcc-lts5r"
Oct 30 21:49:55.154115 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:55.154086    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f608f7-7f85c8fcc-lts5r"]
Oct 30 21:49:55.158645 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:55.158621    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-f608f7-serving-cert\""
Oct 30 21:49:55.262819 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:55.262774    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpdf4\" (UniqueName: \"kubernetes.io/projected/fd9a1e32-2b8b-4f2c-bcfc-5b7390d9a4a2-kube-api-access-jpdf4\") pod \"maas-api-e2e-api-b-f608f7-7f85c8fcc-lts5r\" (UID: \"fd9a1e32-2b8b-4f2c-bcfc-5b7390d9a4a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f608f7-7f85c8fcc-lts5r"
Oct 30 21:49:55.263034 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:55.262998    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd9a1e32-2b8b-4f2c-bcfc-5b7390d9a4a2-openshift-service-ca\") pod \"maas-api-e2e-api-b-f608f7-7f85c8fcc-lts5r\" (UID: \"fd9a1e32-2b8b-4f2c-bcfc-5b7390d9a4a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f608f7-7f85c8fcc-lts5r"
Oct 30 21:49:55.263103 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:55.263054    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd9a1e32-2b8b-4f2c-bcfc-5b7390d9a4a2-maas-api-tls\") pod \"maas-api-e2e-api-b-f608f7-7f85c8fcc-lts5r\" (UID: \"fd9a1e32-2b8b-4f2c-bcfc-5b7390d9a4a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f608f7-7f85c8fcc-lts5r"
Oct 30 21:49:55.263183 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:55.263149    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd9a1e32-2b8b-4f2c-bcfc-5b7390d9a4a2-metrics-certs\") pod \"maas-api-e2e-api-b-f608f7-7f85c8fcc-lts5r\" (UID: \"fd9a1e32-2b8b-4f2c-bcfc-5b7390d9a4a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f608f7-7f85c8fcc-lts5r"
Oct 30 21:49:55.366941 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:55.365181    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd9a1e32-2b8b-4f2c-bcfc-5b7390d9a4a2-openshift-service-ca\") pod \"maas-api-e2e-api-b-f608f7-7f85c8fcc-lts5r\" (UID: \"fd9a1e32-2b8b-4f2c-bcfc-5b7390d9a4a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f608f7-7f85c8fcc-lts5r"
Oct 30 21:49:55.366941 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:55.365247    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd9a1e32-2b8b-4f2c-bcfc-5b7390d9a4a2-maas-api-tls\") pod \"maas-api-e2e-api-b-f608f7-7f85c8fcc-lts5r\" (UID: \"fd9a1e32-2b8b-4f2c-bcfc-5b7390d9a4a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f608f7-7f85c8fcc-lts5r"
Oct 30 21:49:55.393417 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:55.375205    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd9a1e32-2b8b-4f2c-bcfc-5b7390d9a4a2-openshift-service-ca\") pod \"maas-api-e2e-api-b-f608f7-7f85c8fcc-lts5r\" (UID: \"fd9a1e32-2b8b-4f2c-bcfc-5b7390d9a4a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f608f7-7f85c8fcc-lts5r"
Oct 30 21:49:55.393417 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:55.375301    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd9a1e32-2b8b-4f2c-bcfc-5b7390d9a4a2-metrics-certs\") pod \"maas-api-e2e-api-b-f608f7-7f85c8fcc-lts5r\" (UID: \"fd9a1e32-2b8b-4f2c-bcfc-5b7390d9a4a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f608f7-7f85c8fcc-lts5r"
Oct 30 21:49:55.393417 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:55.375753    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpdf4\" (UniqueName: \"kubernetes.io/projected/fd9a1e32-2b8b-4f2c-bcfc-5b7390d9a4a2-kube-api-access-jpdf4\") pod \"maas-api-e2e-api-b-f608f7-7f85c8fcc-lts5r\" (UID: \"fd9a1e32-2b8b-4f2c-bcfc-5b7390d9a4a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f608f7-7f85c8fcc-lts5r"
Oct 30 21:49:55.393417 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:55.380114    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd9a1e32-2b8b-4f2c-bcfc-5b7390d9a4a2-metrics-certs\") pod \"maas-api-e2e-api-b-f608f7-7f85c8fcc-lts5r\" (UID: \"fd9a1e32-2b8b-4f2c-bcfc-5b7390d9a4a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f608f7-7f85c8fcc-lts5r"
Oct 30 21:49:55.393417 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:55.392996    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpdf4\" (UniqueName: \"kubernetes.io/projected/fd9a1e32-2b8b-4f2c-bcfc-5b7390d9a4a2-kube-api-access-jpdf4\") pod \"maas-api-e2e-api-b-f608f7-7f85c8fcc-lts5r\" (UID: \"fd9a1e32-2b8b-4f2c-bcfc-5b7390d9a4a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f608f7-7f85c8fcc-lts5r"
Oct 30 21:49:55.398932 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:55.398846    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd9a1e32-2b8b-4f2c-bcfc-5b7390d9a4a2-maas-api-tls\") pod \"maas-api-e2e-api-b-f608f7-7f85c8fcc-lts5r\" (UID: \"fd9a1e32-2b8b-4f2c-bcfc-5b7390d9a4a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f608f7-7f85c8fcc-lts5r"
Oct 30 21:49:55.483410 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:55.476902    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f608f7-7f85c8fcc-lts5r"
Oct 30 21:49:55.835558 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:49:55.835522    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd9a1e32_2b8b_4f2c_bcfc_5b7390d9a4a2.slice/crio-b9efee46401415630b854ded7171e613293f1b8f8f6bb50adb6d8501eccf00f8 WatchSource:0}: Error finding container b9efee46401415630b854ded7171e613293f1b8f8f6bb50adb6d8501eccf00f8: Status 404 returned error can't find the container with id b9efee46401415630b854ded7171e613293f1b8f8f6bb50adb6d8501eccf00f8
Oct 30 21:49:55.841102 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:55.840500    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f608f7-7f85c8fcc-lts5r"]
Oct 30 21:49:56.239719 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:56.239679    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f608f7-7f85c8fcc-lts5r" event={"ID":"fd9a1e32-2b8b-4f2c-bcfc-5b7390d9a4a2","Type":"ContainerStarted","Data":"b9efee46401415630b854ded7171e613293f1b8f8f6bb50adb6d8501eccf00f8"}
Oct 30 21:49:57.036721 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:57.036686    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h"]
Oct 30 21:49:57.037573 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:57.037545    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp"]
Oct 30 21:49:57.038550 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:57.038515    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp" podUID="86937e06-85cc-48d5-9935-9ae922c3518c" containerName="istio-proxy" containerID="cri-o://bbd3e0f1b0c6a756b516f0adcb34e4c1de51ead6a715f5678a60ef4ded05fe40" gracePeriod=30
Oct 30 21:49:57.040879 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:57.039331    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h" podUID="fb5d58b6-89f9-4735-ba24-e4165dcb4c81" containerName="istio-proxy" containerID="cri-o://1c5a110507f29ec0cbeab84476c21dd0413d3278f930d7aa409b5b74f51449ab" gracePeriod=30
Oct 30 21:49:57.267794 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:57.267746    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f608f7-7f85c8fcc-lts5r" event={"ID":"fd9a1e32-2b8b-4f2c-bcfc-5b7390d9a4a2","Type":"ContainerStarted","Data":"7d2acb90d44968ba6a0044fb2dd11ff2be1e9cb0b1dd79119fe0b145c5ffc0f6"}
Oct 30 21:49:57.270291 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:57.270267    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f608f7-7f85c8fcc-lts5r"
Oct 30 21:49:58.829788 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:58.829730    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f608f7-7f85c8fcc-lts5r" podStartSLOduration=3.410142936 podStartE2EDuration="3.829709656s" totalImagesPullingTime="419.56672ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:49:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:49:57.312884702 +0000 UTC m=+1310.114697168" watchObservedRunningTime="2026-10-30 21:49:58.829709656 +0000 UTC m=+1311.631522121"
Oct 30 21:49:58.832328 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:58.832298    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t"]
Oct 30 21:49:58.840723 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:58.840692    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t"
Oct 30 21:49:58.850820 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:58.850784    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t"]
Oct 30 21:49:58.980322 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:58.980288    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d633840-2f60-4a22-9492-b53908c54f33-workload-socket\") pod \"e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t\" (UID: \"3d633840-2f60-4a22-9492-b53908c54f33\") " pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t"
Oct 30 21:49:58.980582 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:58.980333    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d633840-2f60-4a22-9492-b53908c54f33-istio-data\") pod \"e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t\" (UID: \"3d633840-2f60-4a22-9492-b53908c54f33\") " pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t"
Oct 30 21:49:58.980582 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:58.980539    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d633840-2f60-4a22-9492-b53908c54f33-istiod-ca-cert\") pod \"e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t\" (UID: \"3d633840-2f60-4a22-9492-b53908c54f33\") " pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t"
Oct 30 21:49:58.980582 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:58.980565    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d633840-2f60-4a22-9492-b53908c54f33-credential-socket\") pod \"e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t\" (UID: \"3d633840-2f60-4a22-9492-b53908c54f33\") " pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t"
Oct 30 21:49:58.980582 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:58.980581    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d633840-2f60-4a22-9492-b53908c54f33-workload-certs\") pod \"e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t\" (UID: \"3d633840-2f60-4a22-9492-b53908c54f33\") " pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t"
Oct 30 21:49:58.980727 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:58.980702    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d633840-2f60-4a22-9492-b53908c54f33-istio-token\") pod \"e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t\" (UID: \"3d633840-2f60-4a22-9492-b53908c54f33\") " pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t"
Oct 30 21:49:58.980845 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:58.980822    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpgbj\" (UniqueName: \"kubernetes.io/projected/3d633840-2f60-4a22-9492-b53908c54f33-kube-api-access-xpgbj\") pod \"e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t\" (UID: \"3d633840-2f60-4a22-9492-b53908c54f33\") " pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t"
Oct 30 21:49:58.980926 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:58.980912    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d633840-2f60-4a22-9492-b53908c54f33-istio-envoy\") pod \"e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t\" (UID: \"3d633840-2f60-4a22-9492-b53908c54f33\") " pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t"
Oct 30 21:49:58.980961 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:58.980935    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d633840-2f60-4a22-9492-b53908c54f33-istio-podinfo\") pod \"e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t\" (UID: \"3d633840-2f60-4a22-9492-b53908c54f33\") " pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t"
Oct 30 21:49:59.082919 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:59.082825    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpgbj\" (UniqueName: \"kubernetes.io/projected/3d633840-2f60-4a22-9492-b53908c54f33-kube-api-access-xpgbj\") pod \"e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t\" (UID: \"3d633840-2f60-4a22-9492-b53908c54f33\") " pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t"
Oct 30 21:49:59.083109 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:59.082919    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d633840-2f60-4a22-9492-b53908c54f33-istio-envoy\") pod \"e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t\" (UID: \"3d633840-2f60-4a22-9492-b53908c54f33\") " pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t"
Oct 30 21:49:59.083109 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:59.082940    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d633840-2f60-4a22-9492-b53908c54f33-istio-podinfo\") pod \"e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t\" (UID: \"3d633840-2f60-4a22-9492-b53908c54f33\") " pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t"
Oct 30 21:49:59.083109 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:59.082964    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d633840-2f60-4a22-9492-b53908c54f33-workload-socket\") pod \"e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t\" (UID: \"3d633840-2f60-4a22-9492-b53908c54f33\") " pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t"
Oct 30 21:49:59.083109 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:59.082983    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d633840-2f60-4a22-9492-b53908c54f33-istio-data\") pod \"e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t\" (UID: \"3d633840-2f60-4a22-9492-b53908c54f33\") " pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t"
Oct 30 21:49:59.083324 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:59.083112    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d633840-2f60-4a22-9492-b53908c54f33-istiod-ca-cert\") pod \"e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t\" (UID: \"3d633840-2f60-4a22-9492-b53908c54f33\") " pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t"
Oct 30 21:49:59.083324 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:59.083149    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d633840-2f60-4a22-9492-b53908c54f33-credential-socket\") pod \"e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t\" (UID: \"3d633840-2f60-4a22-9492-b53908c54f33\") " pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t"
Oct 30 21:49:59.083324 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:59.083173    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d633840-2f60-4a22-9492-b53908c54f33-workload-certs\") pod \"e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t\" (UID: \"3d633840-2f60-4a22-9492-b53908c54f33\") " pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t"
Oct 30 21:49:59.083324 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:59.083233    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d633840-2f60-4a22-9492-b53908c54f33-istio-token\") pod \"e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t\" (UID: \"3d633840-2f60-4a22-9492-b53908c54f33\") " pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t"
Oct 30 21:49:59.083589 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:59.083569    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d633840-2f60-4a22-9492-b53908c54f33-workload-socket\") pod \"e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t\" (UID: \"3d633840-2f60-4a22-9492-b53908c54f33\") " pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t"
Oct 30 21:49:59.083646 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:59.083591    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d633840-2f60-4a22-9492-b53908c54f33-istio-data\") pod \"e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t\" (UID: \"3d633840-2f60-4a22-9492-b53908c54f33\") " pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t"
Oct 30 21:49:59.083741 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:59.083721    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d633840-2f60-4a22-9492-b53908c54f33-credential-socket\") pod \"e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t\" (UID: \"3d633840-2f60-4a22-9492-b53908c54f33\") " pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t"
Oct 30 21:49:59.083865 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:59.083846    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d633840-2f60-4a22-9492-b53908c54f33-workload-certs\") pod \"e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t\" (UID: \"3d633840-2f60-4a22-9492-b53908c54f33\") " pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t"
Oct 30 21:49:59.083936 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:59.083918    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d633840-2f60-4a22-9492-b53908c54f33-istiod-ca-cert\") pod \"e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t\" (UID: \"3d633840-2f60-4a22-9492-b53908c54f33\") " pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t"
Oct 30 21:49:59.086108 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:59.086081    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d633840-2f60-4a22-9492-b53908c54f33-istio-envoy\") pod \"e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t\" (UID: \"3d633840-2f60-4a22-9492-b53908c54f33\") " pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t"
Oct 30 21:49:59.086260 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:59.086240    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d633840-2f60-4a22-9492-b53908c54f33-istio-podinfo\") pod \"e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t\" (UID: \"3d633840-2f60-4a22-9492-b53908c54f33\") " pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t"
Oct 30 21:49:59.098076 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:59.098046    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d633840-2f60-4a22-9492-b53908c54f33-istio-token\") pod \"e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t\" (UID: \"3d633840-2f60-4a22-9492-b53908c54f33\") " pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t"
Oct 30 21:49:59.098225 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:59.098200    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpgbj\" (UniqueName: \"kubernetes.io/projected/3d633840-2f60-4a22-9492-b53908c54f33-kube-api-access-xpgbj\") pod \"e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t\" (UID: \"3d633840-2f60-4a22-9492-b53908c54f33\") " pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t"
Oct 30 21:49:59.171795 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:59.171755    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t"
Oct 30 21:49:59.473398 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:59.473349    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t"]
Oct 30 21:49:59.473594 ip-10-0-129-195 kubenswrapper[5032]: W1030 21:49:59.473564    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d633840_2f60_4a22_9492_b53908c54f33.slice/crio-bc07d473f73583cc6cb6dfe3135c705a7d31e4102bf3665eba4f1e95bc339aa6 WatchSource:0}: Error finding container bc07d473f73583cc6cb6dfe3135c705a7d31e4102bf3665eba4f1e95bc339aa6: Status 404 returned error can't find the container with id bc07d473f73583cc6cb6dfe3135c705a7d31e4102bf3665eba4f1e95bc339aa6
Oct 30 21:49:59.477173 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:59.477130    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:49:59.477265 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:59.477227    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:49:59.477321 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:49:59.477269    5032 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:50:00.322462 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:00.318566    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t" event={"ID":"3d633840-2f60-4a22-9492-b53908c54f33","Type":"ContainerStarted","Data":"14827636a9d93fb0847b3c9a5bda7d79282a1e37ecbd87be16d217c96641d6c3"}
Oct 30 21:50:00.322462 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:00.318636    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t" event={"ID":"3d633840-2f60-4a22-9492-b53908c54f33","Type":"ContainerStarted","Data":"bc07d473f73583cc6cb6dfe3135c705a7d31e4102bf3665eba4f1e95bc339aa6"}
Oct 30 21:50:00.375989 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:00.375927    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t" podStartSLOduration=2.375909521 podStartE2EDuration="2.375909521s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:49:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:50:00.364243572 +0000 UTC m=+1313.166056036" watchObservedRunningTime="2026-10-30 21:50:00.375909521 +0000 UTC m=+1313.177721984"
Oct 30 21:50:01.171931 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:01.171899    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t"
Oct 30 21:50:01.174073 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:01.174044    5032 patch_prober.go:29] interesting pod/e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused" start-of-body=
Oct 30 21:50:01.174188 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:01.174100    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t" podUID="3d633840-2f60-4a22-9492-b53908c54f33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused"
Oct 30 21:50:02.172721 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:02.172659    5032 patch_prober.go:29] interesting pod/e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused" start-of-body=
Oct 30 21:50:02.173104 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:02.172750    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t" podUID="3d633840-2f60-4a22-9492-b53908c54f33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused"
Oct 30 21:50:02.374185 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:02.374072    5032 generic.go:356] "Generic (PLEG): container finished" podID="fb5d58b6-89f9-4735-ba24-e4165dcb4c81" containerID="1c5a110507f29ec0cbeab84476c21dd0413d3278f930d7aa409b5b74f51449ab" exitCode=0
Oct 30 21:50:02.374316 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:02.374185    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h" event={"ID":"fb5d58b6-89f9-4735-ba24-e4165dcb4c81","Type":"ContainerDied","Data":"1c5a110507f29ec0cbeab84476c21dd0413d3278f930d7aa409b5b74f51449ab"}
Oct 30 21:50:02.376515 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:02.376485    5032 generic.go:356] "Generic (PLEG): container finished" podID="86937e06-85cc-48d5-9935-9ae922c3518c" containerID="bbd3e0f1b0c6a756b516f0adcb34e4c1de51ead6a715f5678a60ef4ded05fe40" exitCode=0
Oct 30 21:50:02.376798 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:02.376765    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp" event={"ID":"86937e06-85cc-48d5-9935-9ae922c3518c","Type":"ContainerDied","Data":"bbd3e0f1b0c6a756b516f0adcb34e4c1de51ead6a715f5678a60ef4ded05fe40"}
Oct 30 21:50:02.399115 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:02.399093    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp"
Oct 30 21:50:02.554592 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:02.554560    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h"
Oct 30 21:50:02.580633 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:02.580600    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/86937e06-85cc-48d5-9935-9ae922c3518c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86937e06-85cc-48d5-9935-9ae922c3518c-istio-podinfo\") pod \"86937e06-85cc-48d5-9935-9ae922c3518c\" (UID: \"86937e06-85cc-48d5-9935-9ae922c3518c\") "
Oct 30 21:50:02.580795 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:02.580655    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/86937e06-85cc-48d5-9935-9ae922c3518c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86937e06-85cc-48d5-9935-9ae922c3518c-istio-data\") pod \"86937e06-85cc-48d5-9935-9ae922c3518c\" (UID: \"86937e06-85cc-48d5-9935-9ae922c3518c\") "
Oct 30 21:50:02.580795 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:02.580699    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/86937e06-85cc-48d5-9935-9ae922c3518c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86937e06-85cc-48d5-9935-9ae922c3518c-workload-socket\") pod \"86937e06-85cc-48d5-9935-9ae922c3518c\" (UID: \"86937e06-85cc-48d5-9935-9ae922c3518c\") "
Oct 30 21:50:02.580795 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:02.580734    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/86937e06-85cc-48d5-9935-9ae922c3518c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86937e06-85cc-48d5-9935-9ae922c3518c-workload-certs\") pod \"86937e06-85cc-48d5-9935-9ae922c3518c\" (UID: \"86937e06-85cc-48d5-9935-9ae922c3518c\") "
Oct 30 21:50:02.580956 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:02.580797    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/86937e06-85cc-48d5-9935-9ae922c3518c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86937e06-85cc-48d5-9935-9ae922c3518c-istio-envoy\") pod \"86937e06-85cc-48d5-9935-9ae922c3518c\" (UID: \"86937e06-85cc-48d5-9935-9ae922c3518c\") "
Oct 30 21:50:02.580956 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:02.580849    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/86937e06-85cc-48d5-9935-9ae922c3518c-kube-api-access-xnjfc\" (UniqueName: \"kubernetes.io/projected/86937e06-85cc-48d5-9935-9ae922c3518c-kube-api-access-xnjfc\") pod \"86937e06-85cc-48d5-9935-9ae922c3518c\" (UID: \"86937e06-85cc-48d5-9935-9ae922c3518c\") "
Oct 30 21:50:02.580956 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:02.580913    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/86937e06-85cc-48d5-9935-9ae922c3518c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86937e06-85cc-48d5-9935-9ae922c3518c-istiod-ca-cert\") pod \"86937e06-85cc-48d5-9935-9ae922c3518c\" (UID: \"86937e06-85cc-48d5-9935-9ae922c3518c\") "
Oct 30 21:50:02.580956 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:02.580948    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/86937e06-85cc-48d5-9935-9ae922c3518c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86937e06-85cc-48d5-9935-9ae922c3518c-credential-socket\") pod \"86937e06-85cc-48d5-9935-9ae922c3518c\" (UID: \"86937e06-85cc-48d5-9935-9ae922c3518c\") "
Oct 30 21:50:02.581981 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:02.581048    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/86937e06-85cc-48d5-9935-9ae922c3518c-istio-token\" (UniqueName: \"kubernetes.io/projected/86937e06-85cc-48d5-9935-9ae922c3518c-istio-token\") pod \"86937e06-85cc-48d5-9935-9ae922c3518c\" (UID: \"86937e06-85cc-48d5-9935-9ae922c3518c\") "
Oct 30 21:50:02.581981 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:02.581178    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86937e06-85cc-48d5-9935-9ae922c3518c-workload-certs" pod "86937e06-85cc-48d5-9935-9ae922c3518c" (UID: "86937e06-85cc-48d5-9935-9ae922c3518c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:50:02.581981 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:02.581674    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86937e06-85cc-48d5-9935-9ae922c3518c-istio-data" pod "86937e06-85cc-48d5-9935-9ae922c3518c" (UID: "86937e06-85cc-48d5-9935-9ae922c3518c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:50:02.581981 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:02.581749    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86937e06-85cc-48d5-9935-9ae922c3518c-istiod-ca-cert" pod "86937e06-85cc-48d5-9935-9ae922c3518c" (UID: "86937e06-85cc-48d5-9935-9ae922c3518c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:50:02.581981 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:02.581850    5032 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86937e06-85cc-48d5-9935-9ae922c3518c-workload-certs\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:02.581981 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:02.581869    5032 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86937e06-85cc-48d5-9935-9ae922c3518c-istio-data\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:02.582309 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:02.582220    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86937e06-85cc-48d5-9935-9ae922c3518c-credential-socket" pod "86937e06-85cc-48d5-9935-9ae922c3518c" (UID: "86937e06-85cc-48d5-9935-9ae922c3518c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:50:02.582461 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:02.580985    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86937e06-85cc-48d5-9935-9ae922c3518c-workload-socket" pod "86937e06-85cc-48d5-9935-9ae922c3518c" (UID: "86937e06-85cc-48d5-9935-9ae922c3518c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:50:02.585574 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:02.585549    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86937e06-85cc-48d5-9935-9ae922c3518c-istio-token" pod "86937e06-85cc-48d5-9935-9ae922c3518c" (UID: "86937e06-85cc-48d5-9935-9ae922c3518c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:50:02.585875 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:02.585841    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/86937e06-85cc-48d5-9935-9ae922c3518c-istio-podinfo" pod "86937e06-85cc-48d5-9935-9ae922c3518c" (UID: "86937e06-85cc-48d5-9935-9ae922c3518c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:50:02.586262 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:02.586234    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86937e06-85cc-48d5-9935-9ae922c3518c-istio-envoy" pod "86937e06-85cc-48d5-9935-9ae922c3518c" (UID: "86937e06-85cc-48d5-9935-9ae922c3518c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:50:02.587412 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:02.587348    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86937e06-85cc-48d5-9935-9ae922c3518c-kube-api-access-xnjfc" pod "86937e06-85cc-48d5-9935-9ae922c3518c" (UID: "86937e06-85cc-48d5-9935-9ae922c3518c"). InnerVolumeSpecName "kube-api-access-xnjfc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:50:02.682810 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:02.682730    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fb5d58b6-89f9-4735-ba24-e4165dcb4c81-kube-api-access-wqqfd\" (UniqueName: \"kubernetes.io/projected/fb5d58b6-89f9-4735-ba24-e4165dcb4c81-kube-api-access-wqqfd\") pod \"fb5d58b6-89f9-4735-ba24-e4165dcb4c81\" (UID: \"fb5d58b6-89f9-4735-ba24-e4165dcb4c81\") "
Oct 30 21:50:02.683021 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:02.682816    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fb5d58b6-89f9-4735-ba24-e4165dcb4c81-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb5d58b6-89f9-4735-ba24-e4165dcb4c81-workload-certs\") pod \"fb5d58b6-89f9-4735-ba24-e4165dcb4c81\" (UID: \"fb5d58b6-89f9-4735-ba24-e4165dcb4c81\") "
Oct 30 21:50:02.683021 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:02.682855    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/fb5d58b6-89f9-4735-ba24-e4165dcb4c81-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb5d58b6-89f9-4735-ba24-e4165dcb4c81-istio-podinfo\") pod \"fb5d58b6-89f9-4735-ba24-e4165dcb4c81\" (UID: \"fb5d58b6-89f9-4735-ba24-e4165dcb4c81\") "
Oct 30 21:50:02.683021 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:02.683009    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fb5d58b6-89f9-4735-ba24-e4165dcb4c81-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb5d58b6-89f9-4735-ba24-e4165dcb4c81-istio-data\") pod \"fb5d58b6-89f9-4735-ba24-e4165dcb4c81\" (UID: \"fb5d58b6-89f9-4735-ba24-e4165dcb4c81\") "
Oct 30 21:50:02.683172 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:02.683097    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fb5d58b6-89f9-4735-ba24-e4165dcb4c81-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb5d58b6-89f9-4735-ba24-e4165dcb4c81-istio-envoy\") pod \"fb5d58b6-89f9-4735-ba24-e4165dcb4c81\" (UID: \"fb5d58b6-89f9-4735-ba24-e4165dcb4c81\") "
Oct 30 21:50:02.683172 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:02.683122    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fb5d58b6-89f9-4735-ba24-e4165dcb4c81-istio-token\" (UniqueName: \"kubernetes.io/projected/fb5d58b6-89f9-4735-ba24-e4165dcb4c81-istio-token\") pod \"fb5d58b6-89f9-4735-ba24-e4165dcb4c81\" (UID: \"fb5d58b6-89f9-4735-ba24-e4165dcb4c81\") "
Oct 30 21:50:02.683289 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:02.683168    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb5d58b6-89f9-4735-ba24-e4165dcb4c81-workload-certs" pod "fb5d58b6-89f9-4735-ba24-e4165dcb4c81" (UID: "fb5d58b6-89f9-4735-ba24-e4165dcb4c81"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:50:02.683289 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:02.683196    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fb5d58b6-89f9-4735-ba24-e4165dcb4c81-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb5d58b6-89f9-4735-ba24-e4165dcb4c81-credential-socket\") pod \"fb5d58b6-89f9-4735-ba24-e4165dcb4c81\" (UID: \"fb5d58b6-89f9-4735-ba24-e4165dcb4c81\") "
Oct 30 21:50:02.683450 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:02.683309    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fb5d58b6-89f9-4735-ba24-e4165dcb4c81-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb5d58b6-89f9-4735-ba24-e4165dcb4c81-istiod-ca-cert\") pod \"fb5d58b6-89f9-4735-ba24-e4165dcb4c81\" (UID: \"fb5d58b6-89f9-4735-ba24-e4165dcb4c81\") "
Oct 30 21:50:02.683450 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:02.683313    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb5d58b6-89f9-4735-ba24-e4165dcb4c81-istio-data" pod "fb5d58b6-89f9-4735-ba24-e4165dcb4c81" (UID: "fb5d58b6-89f9-4735-ba24-e4165dcb4c81"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:50:02.683450 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:02.683341    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fb5d58b6-89f9-4735-ba24-e4165dcb4c81-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb5d58b6-89f9-4735-ba24-e4165dcb4c81-workload-socket\") pod \"fb5d58b6-89f9-4735-ba24-e4165dcb4c81\" (UID: \"fb5d58b6-89f9-4735-ba24-e4165dcb4c81\") "
Oct 30 21:50:02.683607 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:02.683522    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb5d58b6-89f9-4735-ba24-e4165dcb4c81-credential-socket" pod "fb5d58b6-89f9-4735-ba24-e4165dcb4c81" (UID: "fb5d58b6-89f9-4735-ba24-e4165dcb4c81"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:50:02.683729 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:02.683697    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb5d58b6-89f9-4735-ba24-e4165dcb4c81-istiod-ca-cert" pod "fb5d58b6-89f9-4735-ba24-e4165dcb4c81" (UID: "fb5d58b6-89f9-4735-ba24-e4165dcb4c81"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:50:02.683844 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:02.683822    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb5d58b6-89f9-4735-ba24-e4165dcb4c81-workload-socket" pod "fb5d58b6-89f9-4735-ba24-e4165dcb4c81" (UID: "fb5d58b6-89f9-4735-ba24-e4165dcb4c81"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:50:02.684277 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:02.684256    5032 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86937e06-85cc-48d5-9935-9ae922c3518c-istiod-ca-cert\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:02.684405 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:02.684279    5032 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86937e06-85cc-48d5-9935-9ae922c3518c-credential-socket\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:02.684405 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:02.684290    5032 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb5d58b6-89f9-4735-ba24-e4165dcb4c81-credential-socket\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:02.684405 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:02.684299    5032 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86937e06-85cc-48d5-9935-9ae922c3518c-istio-token\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:02.684405 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:02.684308    5032 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb5d58b6-89f9-4735-ba24-e4165dcb4c81-istiod-ca-cert\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:02.684405 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:02.684321    5032 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb5d58b6-89f9-4735-ba24-e4165dcb4c81-workload-socket\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:02.684405 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:02.684336    5032 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb5d58b6-89f9-4735-ba24-e4165dcb4c81-workload-certs\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:02.684405 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:02.684348    5032 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86937e06-85cc-48d5-9935-9ae922c3518c-istio-podinfo\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:02.684405 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:02.684372    5032 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86937e06-85cc-48d5-9935-9ae922c3518c-workload-socket\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:02.684405 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:02.684385    5032 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86937e06-85cc-48d5-9935-9ae922c3518c-istio-envoy\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:02.684405 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:02.684398    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xnjfc\" (UniqueName: \"kubernetes.io/projected/86937e06-85cc-48d5-9935-9ae922c3518c-kube-api-access-xnjfc\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:02.684728 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:02.684412    5032 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb5d58b6-89f9-4735-ba24-e4165dcb4c81-istio-data\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:02.685396 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:02.685338    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fb5d58b6-89f9-4735-ba24-e4165dcb4c81-istio-podinfo" pod "fb5d58b6-89f9-4735-ba24-e4165dcb4c81" (UID: "fb5d58b6-89f9-4735-ba24-e4165dcb4c81"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:50:02.685790 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:02.685774    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb5d58b6-89f9-4735-ba24-e4165dcb4c81-istio-token" pod "fb5d58b6-89f9-4735-ba24-e4165dcb4c81" (UID: "fb5d58b6-89f9-4735-ba24-e4165dcb4c81"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:50:02.685842 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:02.685819    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb5d58b6-89f9-4735-ba24-e4165dcb4c81-kube-api-access-wqqfd" pod "fb5d58b6-89f9-4735-ba24-e4165dcb4c81" (UID: "fb5d58b6-89f9-4735-ba24-e4165dcb4c81"). InnerVolumeSpecName "kube-api-access-wqqfd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:50:02.686135 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:02.686116    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb5d58b6-89f9-4735-ba24-e4165dcb4c81-istio-envoy" pod "fb5d58b6-89f9-4735-ba24-e4165dcb4c81" (UID: "fb5d58b6-89f9-4735-ba24-e4165dcb4c81"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:50:02.785603 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:02.785568    5032 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb5d58b6-89f9-4735-ba24-e4165dcb4c81-istio-podinfo\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:02.785603 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:02.785600    5032 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb5d58b6-89f9-4735-ba24-e4165dcb4c81-istio-envoy\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:02.785816 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:02.785613    5032 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb5d58b6-89f9-4735-ba24-e4165dcb4c81-istio-token\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:02.785816 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:02.785625    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wqqfd\" (UniqueName: \"kubernetes.io/projected/fb5d58b6-89f9-4735-ba24-e4165dcb4c81-kube-api-access-wqqfd\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:03.172953 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:03.172920    5032 patch_prober.go:29] interesting pod/e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused" start-of-body=
Oct 30 21:50:03.173353 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:03.172986    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t" podUID="3d633840-2f60-4a22-9492-b53908c54f33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused"
Oct 30 21:50:03.394254 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:03.394224    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h"
Oct 30 21:50:03.395343 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:03.395305    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h" event={"ID":"fb5d58b6-89f9-4735-ba24-e4165dcb4c81","Type":"ContainerDied","Data":"6f69c058ac9be551378f6159da0710dba8cab0b2e9594f93f845d57cf476ca34"}
Oct 30 21:50:03.395709 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:03.395689    5032 scope.go:122] "RemoveContainer" containerID="1c5a110507f29ec0cbeab84476c21dd0413d3278f930d7aa409b5b74f51449ab"
Oct 30 21:50:03.398147 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:03.398119    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp" event={"ID":"86937e06-85cc-48d5-9935-9ae922c3518c","Type":"ContainerDied","Data":"34c23ade960d9c6231d94ca2cd5bab37d0be190bf53b0a5da0b282bf7fb66590"}
Oct 30 21:50:03.398239 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:03.398209    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp"
Oct 30 21:50:03.443608 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:03.441608    5032 scope.go:122] "RemoveContainer" containerID="bbd3e0f1b0c6a756b516f0adcb34e4c1de51ead6a715f5678a60ef4ded05fe40"
Oct 30 21:50:03.454075 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:03.454036    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h"]
Oct 30 21:50:03.459502 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:03.459473    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-zc84h"]
Oct 30 21:50:03.471001 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:03.470962    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp"]
Oct 30 21:50:03.476677 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:03.476645    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-000f64-openshift-default-679c88c5b9-4wdnp"]
Oct 30 21:50:04.172881 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:04.172848    5032 patch_prober.go:29] interesting pod/e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused" start-of-body=
Oct 30 21:50:04.173099 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:04.172899    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t" podUID="3d633840-2f60-4a22-9492-b53908c54f33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused"
Oct 30 21:50:04.288807 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:04.288780    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f608f7-7f85c8fcc-lts5r"
Oct 30 21:50:05.173010 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:05.172967    5032 patch_prober.go:29] interesting pod/e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused" start-of-body=
Oct 30 21:50:05.173190 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:05.173035    5032 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t" podUID="3d633840-2f60-4a22-9492-b53908c54f33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused"
Oct 30 21:50:05.350270 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:05.350233    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="86937e06-85cc-48d5-9935-9ae922c3518c" path="/var/lib/kubelet/pods/86937e06-85cc-48d5-9935-9ae922c3518c/volumes"
Oct 30 21:50:05.350905 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:05.350889    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fb5d58b6-89f9-4735-ba24-e4165dcb4c81" path="/var/lib/kubelet/pods/fb5d58b6-89f9-4735-ba24-e4165dcb4c81/volumes"
Oct 30 21:50:06.177237 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:06.177205    5032 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t"
Oct 30 21:50:06.177749 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:06.177730    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t"
Oct 30 21:50:06.178546 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:06.178524    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t"
Oct 30 21:50:08.937040 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:08.936980    5032 scope.go:122] "RemoveContainer" containerID="51dc5925f7d3db9b4cf18e7e05f841ea126c46e10ceb6d77d137207f2c52ae78"
Oct 30 21:50:08.955504 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:08.955472    5032 scope.go:122] "RemoveContainer" containerID="aa9393190b852941e794ff7c91f89d633c495233afd09ade66eb9f4afe9393b0"
Oct 30 21:50:12.043080 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:12.043023    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb"]
Oct 30 21:50:12.043598 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:12.043269    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb" podUID="629450ab-662b-43c5-b93d-e264b1ce2034" containerName="istio-proxy" containerID="cri-o://9b6b8ea7cbe715277dfd7789482c5bc09d4ec4ffb1e88addfebb451618561677" gracePeriod=30
Oct 30 21:50:12.048575 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:12.048545    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t"]
Oct 30 21:50:12.048939 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:12.048905    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t" podUID="3d633840-2f60-4a22-9492-b53908c54f33" containerName="istio-proxy" containerID="cri-o://14827636a9d93fb0847b3c9a5bda7d79282a1e37ecbd87be16d217c96641d6c3" gracePeriod=30
Oct 30 21:50:17.423284 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.423256    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t"
Oct 30 21:50:17.478112 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.478071    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb"
Oct 30 21:50:17.494780 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.494750    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3d633840-2f60-4a22-9492-b53908c54f33-istio-token\" (UniqueName: \"kubernetes.io/projected/3d633840-2f60-4a22-9492-b53908c54f33-istio-token\") pod \"3d633840-2f60-4a22-9492-b53908c54f33\" (UID: \"3d633840-2f60-4a22-9492-b53908c54f33\") "
Oct 30 21:50:17.494951 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.494866    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d633840-2f60-4a22-9492-b53908c54f33-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d633840-2f60-4a22-9492-b53908c54f33-credential-socket\") pod \"3d633840-2f60-4a22-9492-b53908c54f33\" (UID: \"3d633840-2f60-4a22-9492-b53908c54f33\") "
Oct 30 21:50:17.494951 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.494943    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3d633840-2f60-4a22-9492-b53908c54f33-kube-api-access-xpgbj\" (UniqueName: \"kubernetes.io/projected/3d633840-2f60-4a22-9492-b53908c54f33-kube-api-access-xpgbj\") pod \"3d633840-2f60-4a22-9492-b53908c54f33\" (UID: \"3d633840-2f60-4a22-9492-b53908c54f33\") "
Oct 30 21:50:17.495099 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.494987    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d633840-2f60-4a22-9492-b53908c54f33-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d633840-2f60-4a22-9492-b53908c54f33-istio-data\") pod \"3d633840-2f60-4a22-9492-b53908c54f33\" (UID: \"3d633840-2f60-4a22-9492-b53908c54f33\") "
Oct 30 21:50:17.495099 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.495027    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d633840-2f60-4a22-9492-b53908c54f33-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d633840-2f60-4a22-9492-b53908c54f33-istio-envoy\") pod \"3d633840-2f60-4a22-9492-b53908c54f33\" (UID: \"3d633840-2f60-4a22-9492-b53908c54f33\") "
Oct 30 21:50:17.495205 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.495127    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d633840-2f60-4a22-9492-b53908c54f33-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d633840-2f60-4a22-9492-b53908c54f33-workload-socket\") pod \"3d633840-2f60-4a22-9492-b53908c54f33\" (UID: \"3d633840-2f60-4a22-9492-b53908c54f33\") "
Oct 30 21:50:17.495205 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.495187    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d633840-2f60-4a22-9492-b53908c54f33-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d633840-2f60-4a22-9492-b53908c54f33-workload-certs\") pod \"3d633840-2f60-4a22-9492-b53908c54f33\" (UID: \"3d633840-2f60-4a22-9492-b53908c54f33\") "
Oct 30 21:50:17.495303 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.495223    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3d633840-2f60-4a22-9492-b53908c54f33-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d633840-2f60-4a22-9492-b53908c54f33-istiod-ca-cert\") pod \"3d633840-2f60-4a22-9492-b53908c54f33\" (UID: \"3d633840-2f60-4a22-9492-b53908c54f33\") "
Oct 30 21:50:17.495303 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.495226    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d633840-2f60-4a22-9492-b53908c54f33-credential-socket" pod "3d633840-2f60-4a22-9492-b53908c54f33" (UID: "3d633840-2f60-4a22-9492-b53908c54f33"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:50:17.495459 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.495439    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3d633840-2f60-4a22-9492-b53908c54f33-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d633840-2f60-4a22-9492-b53908c54f33-istio-podinfo\") pod \"3d633840-2f60-4a22-9492-b53908c54f33\" (UID: \"3d633840-2f60-4a22-9492-b53908c54f33\") "
Oct 30 21:50:17.495753 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.495474    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d633840-2f60-4a22-9492-b53908c54f33-istio-data" pod "3d633840-2f60-4a22-9492-b53908c54f33" (UID: "3d633840-2f60-4a22-9492-b53908c54f33"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:50:17.495848 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.495622    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d633840-2f60-4a22-9492-b53908c54f33-workload-certs" pod "3d633840-2f60-4a22-9492-b53908c54f33" (UID: "3d633840-2f60-4a22-9492-b53908c54f33"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:50:17.495848 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.495706    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d633840-2f60-4a22-9492-b53908c54f33-workload-socket" pod "3d633840-2f60-4a22-9492-b53908c54f33" (UID: "3d633840-2f60-4a22-9492-b53908c54f33"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:50:17.495848 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.495825    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d633840-2f60-4a22-9492-b53908c54f33-istiod-ca-cert" pod "3d633840-2f60-4a22-9492-b53908c54f33" (UID: "3d633840-2f60-4a22-9492-b53908c54f33"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:50:17.496659 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.496638    5032 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d633840-2f60-4a22-9492-b53908c54f33-workload-certs\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:17.496781 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.496765    5032 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d633840-2f60-4a22-9492-b53908c54f33-istiod-ca-cert\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:17.496870 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.496857    5032 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d633840-2f60-4a22-9492-b53908c54f33-credential-socket\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:17.496969 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.496956    5032 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d633840-2f60-4a22-9492-b53908c54f33-istio-data\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:17.497053 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.497041    5032 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d633840-2f60-4a22-9492-b53908c54f33-workload-socket\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:17.497627 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.497602    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d633840-2f60-4a22-9492-b53908c54f33-istio-token" pod "3d633840-2f60-4a22-9492-b53908c54f33" (UID: "3d633840-2f60-4a22-9492-b53908c54f33"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:50:17.498114 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.498091    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3d633840-2f60-4a22-9492-b53908c54f33-istio-podinfo" pod "3d633840-2f60-4a22-9492-b53908c54f33" (UID: "3d633840-2f60-4a22-9492-b53908c54f33"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:50:17.498455 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.498434    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d633840-2f60-4a22-9492-b53908c54f33-kube-api-access-xpgbj" pod "3d633840-2f60-4a22-9492-b53908c54f33" (UID: "3d633840-2f60-4a22-9492-b53908c54f33"). InnerVolumeSpecName "kube-api-access-xpgbj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:50:17.498839 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.498822    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d633840-2f60-4a22-9492-b53908c54f33-istio-envoy" pod "3d633840-2f60-4a22-9492-b53908c54f33" (UID: "3d633840-2f60-4a22-9492-b53908c54f33"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:50:17.543154 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.543116    5032 generic.go:356] "Generic (PLEG): container finished" podID="3d633840-2f60-4a22-9492-b53908c54f33" containerID="14827636a9d93fb0847b3c9a5bda7d79282a1e37ecbd87be16d217c96641d6c3" exitCode=0
Oct 30 21:50:17.543326 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.543176    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t"
Oct 30 21:50:17.543326 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.543199    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t" event={"ID":"3d633840-2f60-4a22-9492-b53908c54f33","Type":"ContainerDied","Data":"14827636a9d93fb0847b3c9a5bda7d79282a1e37ecbd87be16d217c96641d6c3"}
Oct 30 21:50:17.543326 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.543277    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t" event={"ID":"3d633840-2f60-4a22-9492-b53908c54f33","Type":"ContainerDied","Data":"bc07d473f73583cc6cb6dfe3135c705a7d31e4102bf3665eba4f1e95bc339aa6"}
Oct 30 21:50:17.543326 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.543310    5032 scope.go:122] "RemoveContainer" containerID="14827636a9d93fb0847b3c9a5bda7d79282a1e37ecbd87be16d217c96641d6c3"
Oct 30 21:50:17.546028 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.545957    5032 generic.go:356] "Generic (PLEG): container finished" podID="629450ab-662b-43c5-b93d-e264b1ce2034" containerID="9b6b8ea7cbe715277dfd7789482c5bc09d4ec4ffb1e88addfebb451618561677" exitCode=0
Oct 30 21:50:17.546161 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.546065    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb" event={"ID":"629450ab-662b-43c5-b93d-e264b1ce2034","Type":"ContainerDied","Data":"9b6b8ea7cbe715277dfd7789482c5bc09d4ec4ffb1e88addfebb451618561677"}
Oct 30 21:50:17.546161 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.546110    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb" event={"ID":"629450ab-662b-43c5-b93d-e264b1ce2034","Type":"ContainerDied","Data":"2a156f8f68a3ab4cf5e5f14d4f10773edf39e9b8f23df4841a72f01cbf0ec3b4"}
Oct 30 21:50:17.546296 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.546179    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb"
Oct 30 21:50:17.560214 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.560189    5032 scope.go:122] "RemoveContainer" containerID="14827636a9d93fb0847b3c9a5bda7d79282a1e37ecbd87be16d217c96641d6c3"
Oct 30 21:50:17.561277 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:50:17.561247    5032 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14827636a9d93fb0847b3c9a5bda7d79282a1e37ecbd87be16d217c96641d6c3\": container with ID starting with 14827636a9d93fb0847b3c9a5bda7d79282a1e37ecbd87be16d217c96641d6c3 not found: ID does not exist" containerID="14827636a9d93fb0847b3c9a5bda7d79282a1e37ecbd87be16d217c96641d6c3"
Oct 30 21:50:17.561430 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.561288    5032 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14827636a9d93fb0847b3c9a5bda7d79282a1e37ecbd87be16d217c96641d6c3"} err="failed to get container status \"14827636a9d93fb0847b3c9a5bda7d79282a1e37ecbd87be16d217c96641d6c3\": rpc error: code = NotFound desc = could not find container \"14827636a9d93fb0847b3c9a5bda7d79282a1e37ecbd87be16d217c96641d6c3\": container with ID starting with 14827636a9d93fb0847b3c9a5bda7d79282a1e37ecbd87be16d217c96641d6c3 not found: ID does not exist"
Oct 30 21:50:17.561430 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.561311    5032 scope.go:122] "RemoveContainer" containerID="9b6b8ea7cbe715277dfd7789482c5bc09d4ec4ffb1e88addfebb451618561677"
Oct 30 21:50:17.575916 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.575881    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t"]
Oct 30 21:50:17.576072 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.576052    5032 scope.go:122] "RemoveContainer" containerID="9b6b8ea7cbe715277dfd7789482c5bc09d4ec4ffb1e88addfebb451618561677"
Oct 30 21:50:17.579119 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:50:17.579084    5032 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b6b8ea7cbe715277dfd7789482c5bc09d4ec4ffb1e88addfebb451618561677\": container with ID starting with 9b6b8ea7cbe715277dfd7789482c5bc09d4ec4ffb1e88addfebb451618561677 not found: ID does not exist" containerID="9b6b8ea7cbe715277dfd7789482c5bc09d4ec4ffb1e88addfebb451618561677"
Oct 30 21:50:17.579228 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.579131    5032 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b6b8ea7cbe715277dfd7789482c5bc09d4ec4ffb1e88addfebb451618561677"} err="failed to get container status \"9b6b8ea7cbe715277dfd7789482c5bc09d4ec4ffb1e88addfebb451618561677\": rpc error: code = NotFound desc = could not find container \"9b6b8ea7cbe715277dfd7789482c5bc09d4ec4ffb1e88addfebb451618561677\": container with ID starting with 9b6b8ea7cbe715277dfd7789482c5bc09d4ec4ffb1e88addfebb451618561677 not found: ID does not exist"
Oct 30 21:50:17.582906 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.582875    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-bt58t"]
Oct 30 21:50:17.598201 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.598171    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/629450ab-662b-43c5-b93d-e264b1ce2034-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/629450ab-662b-43c5-b93d-e264b1ce2034-istio-envoy\") pod \"629450ab-662b-43c5-b93d-e264b1ce2034\" (UID: \"629450ab-662b-43c5-b93d-e264b1ce2034\") "
Oct 30 21:50:17.598499 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.598345    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/629450ab-662b-43c5-b93d-e264b1ce2034-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/629450ab-662b-43c5-b93d-e264b1ce2034-workload-certs\") pod \"629450ab-662b-43c5-b93d-e264b1ce2034\" (UID: \"629450ab-662b-43c5-b93d-e264b1ce2034\") "
Oct 30 21:50:17.598499 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.598444    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/629450ab-662b-43c5-b93d-e264b1ce2034-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/629450ab-662b-43c5-b93d-e264b1ce2034-workload-socket\") pod \"629450ab-662b-43c5-b93d-e264b1ce2034\" (UID: \"629450ab-662b-43c5-b93d-e264b1ce2034\") "
Oct 30 21:50:17.598499 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.598481    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/629450ab-662b-43c5-b93d-e264b1ce2034-kube-api-access-m2gmb\" (UniqueName: \"kubernetes.io/projected/629450ab-662b-43c5-b93d-e264b1ce2034-kube-api-access-m2gmb\") pod \"629450ab-662b-43c5-b93d-e264b1ce2034\" (UID: \"629450ab-662b-43c5-b93d-e264b1ce2034\") "
Oct 30 21:50:17.598788 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.598522    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/629450ab-662b-43c5-b93d-e264b1ce2034-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/629450ab-662b-43c5-b93d-e264b1ce2034-istio-podinfo\") pod \"629450ab-662b-43c5-b93d-e264b1ce2034\" (UID: \"629450ab-662b-43c5-b93d-e264b1ce2034\") "
Oct 30 21:50:17.598788 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.598571    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/629450ab-662b-43c5-b93d-e264b1ce2034-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/629450ab-662b-43c5-b93d-e264b1ce2034-istiod-ca-cert\") pod \"629450ab-662b-43c5-b93d-e264b1ce2034\" (UID: \"629450ab-662b-43c5-b93d-e264b1ce2034\") "
Oct 30 21:50:17.598788 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.598695    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/629450ab-662b-43c5-b93d-e264b1ce2034-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/629450ab-662b-43c5-b93d-e264b1ce2034-credential-socket\") pod \"629450ab-662b-43c5-b93d-e264b1ce2034\" (UID: \"629450ab-662b-43c5-b93d-e264b1ce2034\") "
Oct 30 21:50:17.598788 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.598749    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/629450ab-662b-43c5-b93d-e264b1ce2034-istio-token\" (UniqueName: \"kubernetes.io/projected/629450ab-662b-43c5-b93d-e264b1ce2034-istio-token\") pod \"629450ab-662b-43c5-b93d-e264b1ce2034\" (UID: \"629450ab-662b-43c5-b93d-e264b1ce2034\") "
Oct 30 21:50:17.599051 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.598909    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/629450ab-662b-43c5-b93d-e264b1ce2034-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/629450ab-662b-43c5-b93d-e264b1ce2034-istio-data\") pod \"629450ab-662b-43c5-b93d-e264b1ce2034\" (UID: \"629450ab-662b-43c5-b93d-e264b1ce2034\") "
Oct 30 21:50:17.599619 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.599577    5032 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d633840-2f60-4a22-9492-b53908c54f33-istio-envoy\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:17.599619 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.599605    5032 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d633840-2f60-4a22-9492-b53908c54f33-istio-podinfo\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:17.599796 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.599622    5032 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d633840-2f60-4a22-9492-b53908c54f33-istio-token\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:17.599796 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.599636    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xpgbj\" (UniqueName: \"kubernetes.io/projected/3d633840-2f60-4a22-9492-b53908c54f33-kube-api-access-xpgbj\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:17.599796 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.599696    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/629450ab-662b-43c5-b93d-e264b1ce2034-credential-socket" pod "629450ab-662b-43c5-b93d-e264b1ce2034" (UID: "629450ab-662b-43c5-b93d-e264b1ce2034"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:50:17.599971 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.599869    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/629450ab-662b-43c5-b93d-e264b1ce2034-istio-data" pod "629450ab-662b-43c5-b93d-e264b1ce2034" (UID: "629450ab-662b-43c5-b93d-e264b1ce2034"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:50:17.600035 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.600017    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/629450ab-662b-43c5-b93d-e264b1ce2034-istiod-ca-cert" pod "629450ab-662b-43c5-b93d-e264b1ce2034" (UID: "629450ab-662b-43c5-b93d-e264b1ce2034"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:50:17.600281 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.600255    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/629450ab-662b-43c5-b93d-e264b1ce2034-workload-socket" pod "629450ab-662b-43c5-b93d-e264b1ce2034" (UID: "629450ab-662b-43c5-b93d-e264b1ce2034"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:50:17.600541 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.600521    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/629450ab-662b-43c5-b93d-e264b1ce2034-workload-certs" pod "629450ab-662b-43c5-b93d-e264b1ce2034" (UID: "629450ab-662b-43c5-b93d-e264b1ce2034"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:50:17.602471 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.602443    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/629450ab-662b-43c5-b93d-e264b1ce2034-istio-podinfo" pod "629450ab-662b-43c5-b93d-e264b1ce2034" (UID: "629450ab-662b-43c5-b93d-e264b1ce2034"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:50:17.602579 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.602522    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/629450ab-662b-43c5-b93d-e264b1ce2034-istio-envoy" pod "629450ab-662b-43c5-b93d-e264b1ce2034" (UID: "629450ab-662b-43c5-b93d-e264b1ce2034"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:50:17.602786 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.602758    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/629450ab-662b-43c5-b93d-e264b1ce2034-kube-api-access-m2gmb" pod "629450ab-662b-43c5-b93d-e264b1ce2034" (UID: "629450ab-662b-43c5-b93d-e264b1ce2034"). InnerVolumeSpecName "kube-api-access-m2gmb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:50:17.603244 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.603220    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/629450ab-662b-43c5-b93d-e264b1ce2034-istio-token" pod "629450ab-662b-43c5-b93d-e264b1ce2034" (UID: "629450ab-662b-43c5-b93d-e264b1ce2034"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:50:17.701606 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.701574    5032 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/629450ab-662b-43c5-b93d-e264b1ce2034-workload-certs\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:17.701606 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.701603    5032 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/629450ab-662b-43c5-b93d-e264b1ce2034-workload-socket\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:17.701904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.701617    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m2gmb\" (UniqueName: \"kubernetes.io/projected/629450ab-662b-43c5-b93d-e264b1ce2034-kube-api-access-m2gmb\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:17.701904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.701629    5032 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/629450ab-662b-43c5-b93d-e264b1ce2034-istio-podinfo\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:17.701904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.701639    5032 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/629450ab-662b-43c5-b93d-e264b1ce2034-istiod-ca-cert\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:17.701904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.701651    5032 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/629450ab-662b-43c5-b93d-e264b1ce2034-credential-socket\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:17.701904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.701665    5032 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/629450ab-662b-43c5-b93d-e264b1ce2034-istio-token\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:17.701904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.701674    5032 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/629450ab-662b-43c5-b93d-e264b1ce2034-istio-data\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:17.701904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.701682    5032 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/629450ab-662b-43c5-b93d-e264b1ce2034-istio-envoy\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:17.872514 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.872475    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb"]
Oct 30 21:50:17.887325 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:17.887292    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-08aafeac-openshift-default-6cb947dcbf-pjjfb"]
Oct 30 21:50:18.346349 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:50:18.346317    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 21:50:19.366138 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:19.366099    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3d633840-2f60-4a22-9492-b53908c54f33" path="/var/lib/kubelet/pods/3d633840-2f60-4a22-9492-b53908c54f33/volumes"
Oct 30 21:50:19.366711 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:19.366693    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="629450ab-662b-43c5-b93d-e264b1ce2034" path="/var/lib/kubelet/pods/629450ab-662b-43c5-b93d-e264b1ce2034/volumes"
Oct 30 21:50:22.252203 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:22.252161    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 21:50:26.540883 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:26.540846    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-1f725d-7678fd7cb5-ntchg"]
Oct 30 21:50:26.541960 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:26.541846    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-1f725d-7678fd7cb5-ntchg" podUID="b68282ae-6782-4b8f-9686-73089f54ff78" containerName="maas-api" containerID="cri-o://1d830815087efc871482b672a5263c6704a207bd47f1d95dcf7c22da5edc8014" gracePeriod=30
Oct 30 21:50:26.915451 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:26.915237    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-1f725d-7678fd7cb5-ntchg"
Oct 30 21:50:27.014436 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:27.007609    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b68282ae-6782-4b8f-9686-73089f54ff78-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b68282ae-6782-4b8f-9686-73089f54ff78-maas-api-tls\") pod \"b68282ae-6782-4b8f-9686-73089f54ff78\" (UID: \"b68282ae-6782-4b8f-9686-73089f54ff78\") "
Oct 30 21:50:27.014436 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:27.013216    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b68282ae-6782-4b8f-9686-73089f54ff78-kube-api-access-5m89f\" (UniqueName: \"kubernetes.io/projected/b68282ae-6782-4b8f-9686-73089f54ff78-kube-api-access-5m89f\") pod \"b68282ae-6782-4b8f-9686-73089f54ff78\" (UID: \"b68282ae-6782-4b8f-9686-73089f54ff78\") "
Oct 30 21:50:27.014436 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:27.013434    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b68282ae-6782-4b8f-9686-73089f54ff78-metrics-certs\" (UniqueName: \"kubernetes.io/secret/b68282ae-6782-4b8f-9686-73089f54ff78-metrics-certs\") pod \"b68282ae-6782-4b8f-9686-73089f54ff78\" (UID: \"b68282ae-6782-4b8f-9686-73089f54ff78\") "
Oct 30 21:50:27.014436 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:27.013564    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b68282ae-6782-4b8f-9686-73089f54ff78-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b68282ae-6782-4b8f-9686-73089f54ff78-openshift-service-ca\") pod \"b68282ae-6782-4b8f-9686-73089f54ff78\" (UID: \"b68282ae-6782-4b8f-9686-73089f54ff78\") "
Oct 30 21:50:27.015035 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:27.010866    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b68282ae-6782-4b8f-9686-73089f54ff78-maas-api-tls" pod "b68282ae-6782-4b8f-9686-73089f54ff78" (UID: "b68282ae-6782-4b8f-9686-73089f54ff78"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:50:27.015128 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:27.014634    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b68282ae-6782-4b8f-9686-73089f54ff78-openshift-service-ca" pod "b68282ae-6782-4b8f-9686-73089f54ff78" (UID: "b68282ae-6782-4b8f-9686-73089f54ff78"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:50:27.019068 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:27.018442    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b68282ae-6782-4b8f-9686-73089f54ff78-kube-api-access-5m89f" pod "b68282ae-6782-4b8f-9686-73089f54ff78" (UID: "b68282ae-6782-4b8f-9686-73089f54ff78"). InnerVolumeSpecName "kube-api-access-5m89f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:50:27.029560 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:27.029510    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b68282ae-6782-4b8f-9686-73089f54ff78-metrics-certs" pod "b68282ae-6782-4b8f-9686-73089f54ff78" (UID: "b68282ae-6782-4b8f-9686-73089f54ff78"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:50:27.115897 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:27.115798    5032 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b68282ae-6782-4b8f-9686-73089f54ff78-openshift-service-ca\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:27.115897 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:27.115834    5032 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b68282ae-6782-4b8f-9686-73089f54ff78-maas-api-tls\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:27.115897 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:27.115847    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5m89f\" (UniqueName: \"kubernetes.io/projected/b68282ae-6782-4b8f-9686-73089f54ff78-kube-api-access-5m89f\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:27.115897 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:27.115857    5032 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b68282ae-6782-4b8f-9686-73089f54ff78-metrics-certs\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:27.625967 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:27.625934    5032 generic.go:356] "Generic (PLEG): container finished" podID="b68282ae-6782-4b8f-9686-73089f54ff78" containerID="1d830815087efc871482b672a5263c6704a207bd47f1d95dcf7c22da5edc8014" exitCode=0
Oct 30 21:50:27.626406 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:27.625980    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-1f725d-7678fd7cb5-ntchg"
Oct 30 21:50:27.626406 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:27.626024    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-1f725d-7678fd7cb5-ntchg" event={"ID":"b68282ae-6782-4b8f-9686-73089f54ff78","Type":"ContainerDied","Data":"1d830815087efc871482b672a5263c6704a207bd47f1d95dcf7c22da5edc8014"}
Oct 30 21:50:27.626406 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:27.626081    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-1f725d-7678fd7cb5-ntchg" event={"ID":"b68282ae-6782-4b8f-9686-73089f54ff78","Type":"ContainerDied","Data":"c562c534d8bb7d222fa7884dd5e91aa25c85202c7941e0506d712282b2d19699"}
Oct 30 21:50:27.626406 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:27.626109    5032 scope.go:122] "RemoveContainer" containerID="1d830815087efc871482b672a5263c6704a207bd47f1d95dcf7c22da5edc8014"
Oct 30 21:50:27.637466 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:27.637447    5032 scope.go:122] "RemoveContainer" containerID="1d830815087efc871482b672a5263c6704a207bd47f1d95dcf7c22da5edc8014"
Oct 30 21:50:27.637787 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:50:27.637765    5032 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d830815087efc871482b672a5263c6704a207bd47f1d95dcf7c22da5edc8014\": container with ID starting with 1d830815087efc871482b672a5263c6704a207bd47f1d95dcf7c22da5edc8014 not found: ID does not exist" containerID="1d830815087efc871482b672a5263c6704a207bd47f1d95dcf7c22da5edc8014"
Oct 30 21:50:27.637865 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:27.637794    5032 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d830815087efc871482b672a5263c6704a207bd47f1d95dcf7c22da5edc8014"} err="failed to get container status \"1d830815087efc871482b672a5263c6704a207bd47f1d95dcf7c22da5edc8014\": rpc error: code = NotFound desc = could not find container \"1d830815087efc871482b672a5263c6704a207bd47f1d95dcf7c22da5edc8014\": container with ID starting with 1d830815087efc871482b672a5263c6704a207bd47f1d95dcf7c22da5edc8014 not found: ID does not exist"
Oct 30 21:50:27.650439 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:27.650411    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-1f725d-7678fd7cb5-ntchg"]
Oct 30 21:50:27.654310 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:27.654285    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-1f725d-7678fd7cb5-ntchg"]
Oct 30 21:50:29.347497 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:29.347467    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b68282ae-6782-4b8f-9686-73089f54ff78" path="/var/lib/kubelet/pods/b68282ae-6782-4b8f-9686-73089f54ff78/volumes"
Oct 30 21:50:41.504040 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:41.504000    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-2b36c1-799756d5b8-2flb4"]
Oct 30 21:50:41.505140 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:41.504261    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-2b36c1-799756d5b8-2flb4" podUID="3eecfc0e-8c6b-4d24-b14b-242e6e52829d" containerName="maas-api" containerID="cri-o://3c959a66d56f631820780ffa8da16c97d6a9a69641312e653cabe92b738fb100" gracePeriod=30
Oct 30 21:50:41.767301 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:41.767047    5032 generic.go:356] "Generic (PLEG): container finished" podID="3eecfc0e-8c6b-4d24-b14b-242e6e52829d" containerID="3c959a66d56f631820780ffa8da16c97d6a9a69641312e653cabe92b738fb100" exitCode=0
Oct 30 21:50:41.767537 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:41.767304    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-2b36c1-799756d5b8-2flb4" event={"ID":"3eecfc0e-8c6b-4d24-b14b-242e6e52829d","Type":"ContainerDied","Data":"3c959a66d56f631820780ffa8da16c97d6a9a69641312e653cabe92b738fb100"}
Oct 30 21:50:41.863181 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:41.856611    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-2b36c1-799756d5b8-2flb4"
Oct 30 21:50:42.008055 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:42.008022    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3eecfc0e-8c6b-4d24-b14b-242e6e52829d-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3eecfc0e-8c6b-4d24-b14b-242e6e52829d-openshift-service-ca\") pod \"3eecfc0e-8c6b-4d24-b14b-242e6e52829d\" (UID: \"3eecfc0e-8c6b-4d24-b14b-242e6e52829d\") "
Oct 30 21:50:42.008255 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:42.008232    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3eecfc0e-8c6b-4d24-b14b-242e6e52829d-kube-api-access-hk9gt\" (UniqueName: \"kubernetes.io/projected/3eecfc0e-8c6b-4d24-b14b-242e6e52829d-kube-api-access-hk9gt\") pod \"3eecfc0e-8c6b-4d24-b14b-242e6e52829d\" (UID: \"3eecfc0e-8c6b-4d24-b14b-242e6e52829d\") "
Oct 30 21:50:42.008322 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:42.008280    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3eecfc0e-8c6b-4d24-b14b-242e6e52829d-metrics-certs\" (UniqueName: \"kubernetes.io/secret/3eecfc0e-8c6b-4d24-b14b-242e6e52829d-metrics-certs\") pod \"3eecfc0e-8c6b-4d24-b14b-242e6e52829d\" (UID: \"3eecfc0e-8c6b-4d24-b14b-242e6e52829d\") "
Oct 30 21:50:42.008322 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:42.008313    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3eecfc0e-8c6b-4d24-b14b-242e6e52829d-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3eecfc0e-8c6b-4d24-b14b-242e6e52829d-maas-api-tls\") pod \"3eecfc0e-8c6b-4d24-b14b-242e6e52829d\" (UID: \"3eecfc0e-8c6b-4d24-b14b-242e6e52829d\") "
Oct 30 21:50:42.012578 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:42.012516    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3eecfc0e-8c6b-4d24-b14b-242e6e52829d-kube-api-access-hk9gt" pod "3eecfc0e-8c6b-4d24-b14b-242e6e52829d" (UID: "3eecfc0e-8c6b-4d24-b14b-242e6e52829d"). InnerVolumeSpecName "kube-api-access-hk9gt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:50:42.014595 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:42.014568    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3eecfc0e-8c6b-4d24-b14b-242e6e52829d-metrics-certs" pod "3eecfc0e-8c6b-4d24-b14b-242e6e52829d" (UID: "3eecfc0e-8c6b-4d24-b14b-242e6e52829d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:50:42.014757 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:42.014677    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3eecfc0e-8c6b-4d24-b14b-242e6e52829d-openshift-service-ca" pod "3eecfc0e-8c6b-4d24-b14b-242e6e52829d" (UID: "3eecfc0e-8c6b-4d24-b14b-242e6e52829d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:50:42.050297 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:42.048865    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3eecfc0e-8c6b-4d24-b14b-242e6e52829d-maas-api-tls" pod "3eecfc0e-8c6b-4d24-b14b-242e6e52829d" (UID: "3eecfc0e-8c6b-4d24-b14b-242e6e52829d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:50:42.111527 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:42.111494    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hk9gt\" (UniqueName: \"kubernetes.io/projected/3eecfc0e-8c6b-4d24-b14b-242e6e52829d-kube-api-access-hk9gt\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:42.111527 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:42.111524    5032 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3eecfc0e-8c6b-4d24-b14b-242e6e52829d-metrics-certs\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:42.111527 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:42.111535    5032 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3eecfc0e-8c6b-4d24-b14b-242e6e52829d-maas-api-tls\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:42.111760 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:42.111543    5032 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3eecfc0e-8c6b-4d24-b14b-242e6e52829d-openshift-service-ca\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:42.777147 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:42.777115    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-2b36c1-799756d5b8-2flb4"
Oct 30 21:50:42.777685 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:42.777149    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-2b36c1-799756d5b8-2flb4" event={"ID":"3eecfc0e-8c6b-4d24-b14b-242e6e52829d","Type":"ContainerDied","Data":"3b498a23a040429da0be75bdfd226034ad85a073c0a3591d32d3a7d300d7122a"}
Oct 30 21:50:42.777685 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:42.777209    5032 scope.go:122] "RemoveContainer" containerID="3c959a66d56f631820780ffa8da16c97d6a9a69641312e653cabe92b738fb100"
Oct 30 21:50:42.804432 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:42.804404    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-2b36c1-799756d5b8-2flb4"]
Oct 30 21:50:42.809697 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:42.809671    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-2b36c1-799756d5b8-2flb4"]
Oct 30 21:50:43.347977 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:43.347943    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3eecfc0e-8c6b-4d24-b14b-242e6e52829d" path="/var/lib/kubelet/pods/3eecfc0e-8c6b-4d24-b14b-242e6e52829d/volumes"
Oct 30 21:50:44.807785 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:44.807750    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-2b36c1-799756d5b8-2flb4" podUID="3eecfc0e-8c6b-4d24-b14b-242e6e52829d" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.105:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:50:48.347375 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:50:48.347324    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 21:50:54.920657 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:54.920619    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fbf428-59dbc6d9d7-tndfr"]
Oct 30 21:50:54.921495 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:54.920885    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fbf428-59dbc6d9d7-tndfr" podUID="4c6d3630-8f1a-4298-ab10-d5619869e909" containerName="maas-api" containerID="cri-o://04caebcc2bdb9e2a30094e841b5c2f07d0a482c6bc093561730f4a50f4bc4dac" gracePeriod=30
Oct 30 21:50:55.260470 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:55.260439    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fbf428-59dbc6d9d7-tndfr"
Oct 30 21:50:55.326889 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:55.325514    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4c6d3630-8f1a-4298-ab10-d5619869e909-kube-api-access-7mwr9\" (UniqueName: \"kubernetes.io/projected/4c6d3630-8f1a-4298-ab10-d5619869e909-kube-api-access-7mwr9\") pod \"4c6d3630-8f1a-4298-ab10-d5619869e909\" (UID: \"4c6d3630-8f1a-4298-ab10-d5619869e909\") "
Oct 30 21:50:55.326889 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:55.325743    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4c6d3630-8f1a-4298-ab10-d5619869e909-metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c6d3630-8f1a-4298-ab10-d5619869e909-metrics-certs\") pod \"4c6d3630-8f1a-4298-ab10-d5619869e909\" (UID: \"4c6d3630-8f1a-4298-ab10-d5619869e909\") "
Oct 30 21:50:55.326889 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:55.325790    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4c6d3630-8f1a-4298-ab10-d5619869e909-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c6d3630-8f1a-4298-ab10-d5619869e909-openshift-service-ca\") pod \"4c6d3630-8f1a-4298-ab10-d5619869e909\" (UID: \"4c6d3630-8f1a-4298-ab10-d5619869e909\") "
Oct 30 21:50:55.326889 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:55.325833    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4c6d3630-8f1a-4298-ab10-d5619869e909-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c6d3630-8f1a-4298-ab10-d5619869e909-maas-api-tls\") pod \"4c6d3630-8f1a-4298-ab10-d5619869e909\" (UID: \"4c6d3630-8f1a-4298-ab10-d5619869e909\") "
Oct 30 21:50:55.328265 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:55.328233    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c6d3630-8f1a-4298-ab10-d5619869e909-openshift-service-ca" pod "4c6d3630-8f1a-4298-ab10-d5619869e909" (UID: "4c6d3630-8f1a-4298-ab10-d5619869e909"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:50:55.330946 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:55.330893    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c6d3630-8f1a-4298-ab10-d5619869e909-metrics-certs" pod "4c6d3630-8f1a-4298-ab10-d5619869e909" (UID: "4c6d3630-8f1a-4298-ab10-d5619869e909"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:50:55.331047 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:55.330906    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c6d3630-8f1a-4298-ab10-d5619869e909-kube-api-access-7mwr9" pod "4c6d3630-8f1a-4298-ab10-d5619869e909" (UID: "4c6d3630-8f1a-4298-ab10-d5619869e909"). InnerVolumeSpecName "kube-api-access-7mwr9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:50:55.333502 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:55.333464    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c6d3630-8f1a-4298-ab10-d5619869e909-maas-api-tls" pod "4c6d3630-8f1a-4298-ab10-d5619869e909" (UID: "4c6d3630-8f1a-4298-ab10-d5619869e909"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:50:55.428286 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:55.428245    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7mwr9\" (UniqueName: \"kubernetes.io/projected/4c6d3630-8f1a-4298-ab10-d5619869e909-kube-api-access-7mwr9\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:55.428553 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:55.428520    5032 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c6d3630-8f1a-4298-ab10-d5619869e909-metrics-certs\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:55.428553 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:55.428549    5032 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c6d3630-8f1a-4298-ab10-d5619869e909-openshift-service-ca\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:55.428781 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:55.428564    5032 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c6d3630-8f1a-4298-ab10-d5619869e909-maas-api-tls\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:55.922936 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:55.922901    5032 generic.go:356] "Generic (PLEG): container finished" podID="4c6d3630-8f1a-4298-ab10-d5619869e909" containerID="04caebcc2bdb9e2a30094e841b5c2f07d0a482c6bc093561730f4a50f4bc4dac" exitCode=0
Oct 30 21:50:55.923423 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:55.922951    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fbf428-59dbc6d9d7-tndfr"
Oct 30 21:50:55.923423 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:55.922991    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fbf428-59dbc6d9d7-tndfr" event={"ID":"4c6d3630-8f1a-4298-ab10-d5619869e909","Type":"ContainerDied","Data":"04caebcc2bdb9e2a30094e841b5c2f07d0a482c6bc093561730f4a50f4bc4dac"}
Oct 30 21:50:55.923423 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:55.923050    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fbf428-59dbc6d9d7-tndfr" event={"ID":"4c6d3630-8f1a-4298-ab10-d5619869e909","Type":"ContainerDied","Data":"b0b8c5a9628608e3596b6e64da0be381e9fd60c3d63802491f16fe0c24bc43cb"}
Oct 30 21:50:55.923423 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:55.923074    5032 scope.go:122] "RemoveContainer" containerID="04caebcc2bdb9e2a30094e841b5c2f07d0a482c6bc093561730f4a50f4bc4dac"
Oct 30 21:50:55.934333 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:55.934309    5032 scope.go:122] "RemoveContainer" containerID="04caebcc2bdb9e2a30094e841b5c2f07d0a482c6bc093561730f4a50f4bc4dac"
Oct 30 21:50:55.934684 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:50:55.934658    5032 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04caebcc2bdb9e2a30094e841b5c2f07d0a482c6bc093561730f4a50f4bc4dac\": container with ID starting with 04caebcc2bdb9e2a30094e841b5c2f07d0a482c6bc093561730f4a50f4bc4dac not found: ID does not exist" containerID="04caebcc2bdb9e2a30094e841b5c2f07d0a482c6bc093561730f4a50f4bc4dac"
Oct 30 21:50:55.934776 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:55.934698    5032 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04caebcc2bdb9e2a30094e841b5c2f07d0a482c6bc093561730f4a50f4bc4dac"} err="failed to get container status \"04caebcc2bdb9e2a30094e841b5c2f07d0a482c6bc093561730f4a50f4bc4dac\": rpc error: code = NotFound desc = could not find container \"04caebcc2bdb9e2a30094e841b5c2f07d0a482c6bc093561730f4a50f4bc4dac\": container with ID starting with 04caebcc2bdb9e2a30094e841b5c2f07d0a482c6bc093561730f4a50f4bc4dac not found: ID does not exist"
Oct 30 21:50:55.945975 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:55.945946    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fbf428-59dbc6d9d7-tndfr"]
Oct 30 21:50:55.949416 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:55.949385    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fbf428-59dbc6d9d7-tndfr"]
Oct 30 21:50:57.348956 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:50:57.348923    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4c6d3630-8f1a-4298-ab10-d5619869e909" path="/var/lib/kubelet/pods/4c6d3630-8f1a-4298-ab10-d5619869e909/volumes"
Oct 30 21:51:06.055300 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:06.055266    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-aad4dd-679948c8f9-bwf4l"]
Oct 30 21:51:06.055760 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:06.055533    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-aad4dd-679948c8f9-bwf4l" podUID="2d543f80-60d7-4c58-ae3d-667f2e577282" containerName="maas-api" containerID="cri-o://34290ab3c027e8fe6ae94a2720c387c62af229e217cc7adad72af0748cb713f1" gracePeriod=30
Oct 30 21:51:06.433857 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:06.433826    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-aad4dd-679948c8f9-bwf4l"
Oct 30 21:51:06.565494 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:06.565461    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2d543f80-60d7-4c58-ae3d-667f2e577282-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2d543f80-60d7-4c58-ae3d-667f2e577282-maas-api-tls\") pod \"2d543f80-60d7-4c58-ae3d-667f2e577282\" (UID: \"2d543f80-60d7-4c58-ae3d-667f2e577282\") "
Oct 30 21:51:06.565700 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:06.565523    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2d543f80-60d7-4c58-ae3d-667f2e577282-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d543f80-60d7-4c58-ae3d-667f2e577282-openshift-service-ca\") pod \"2d543f80-60d7-4c58-ae3d-667f2e577282\" (UID: \"2d543f80-60d7-4c58-ae3d-667f2e577282\") "
Oct 30 21:51:06.565700 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:06.565610    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2d543f80-60d7-4c58-ae3d-667f2e577282-metrics-certs\" (UniqueName: \"kubernetes.io/secret/2d543f80-60d7-4c58-ae3d-667f2e577282-metrics-certs\") pod \"2d543f80-60d7-4c58-ae3d-667f2e577282\" (UID: \"2d543f80-60d7-4c58-ae3d-667f2e577282\") "
Oct 30 21:51:06.565700 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:06.565679    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2d543f80-60d7-4c58-ae3d-667f2e577282-kube-api-access-bbbjz\" (UniqueName: \"kubernetes.io/projected/2d543f80-60d7-4c58-ae3d-667f2e577282-kube-api-access-bbbjz\") pod \"2d543f80-60d7-4c58-ae3d-667f2e577282\" (UID: \"2d543f80-60d7-4c58-ae3d-667f2e577282\") "
Oct 30 21:51:06.565978 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:06.565952    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d543f80-60d7-4c58-ae3d-667f2e577282-openshift-service-ca" pod "2d543f80-60d7-4c58-ae3d-667f2e577282" (UID: "2d543f80-60d7-4c58-ae3d-667f2e577282"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:51:06.566489 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:06.566466    5032 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d543f80-60d7-4c58-ae3d-667f2e577282-openshift-service-ca\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:06.568356 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:06.568332    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d543f80-60d7-4c58-ae3d-667f2e577282-kube-api-access-bbbjz" pod "2d543f80-60d7-4c58-ae3d-667f2e577282" (UID: "2d543f80-60d7-4c58-ae3d-667f2e577282"). InnerVolumeSpecName "kube-api-access-bbbjz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:51:06.570504 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:06.570422    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d543f80-60d7-4c58-ae3d-667f2e577282-maas-api-tls" pod "2d543f80-60d7-4c58-ae3d-667f2e577282" (UID: "2d543f80-60d7-4c58-ae3d-667f2e577282"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:51:06.570591 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:06.570537    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d543f80-60d7-4c58-ae3d-667f2e577282-metrics-certs" pod "2d543f80-60d7-4c58-ae3d-667f2e577282" (UID: "2d543f80-60d7-4c58-ae3d-667f2e577282"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:51:06.667804 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:06.667719    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bbbjz\" (UniqueName: \"kubernetes.io/projected/2d543f80-60d7-4c58-ae3d-667f2e577282-kube-api-access-bbbjz\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:06.667804 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:06.667753    5032 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2d543f80-60d7-4c58-ae3d-667f2e577282-maas-api-tls\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:06.667804 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:06.667762    5032 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2d543f80-60d7-4c58-ae3d-667f2e577282-metrics-certs\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:07.015750 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:07.015712    5032 generic.go:356] "Generic (PLEG): container finished" podID="2d543f80-60d7-4c58-ae3d-667f2e577282" containerID="34290ab3c027e8fe6ae94a2720c387c62af229e217cc7adad72af0748cb713f1" exitCode=0
Oct 30 21:51:07.015935 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:07.015790    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-aad4dd-679948c8f9-bwf4l"
Oct 30 21:51:07.015935 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:07.015795    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-aad4dd-679948c8f9-bwf4l" event={"ID":"2d543f80-60d7-4c58-ae3d-667f2e577282","Type":"ContainerDied","Data":"34290ab3c027e8fe6ae94a2720c387c62af229e217cc7adad72af0748cb713f1"}
Oct 30 21:51:07.015935 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:07.015871    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-aad4dd-679948c8f9-bwf4l" event={"ID":"2d543f80-60d7-4c58-ae3d-667f2e577282","Type":"ContainerDied","Data":"eaca0eab88d721808069f449465b07901c521f699ca62e3b26790a4220e28dd3"}
Oct 30 21:51:07.015935 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:07.015903    5032 scope.go:122] "RemoveContainer" containerID="34290ab3c027e8fe6ae94a2720c387c62af229e217cc7adad72af0748cb713f1"
Oct 30 21:51:07.029802 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:07.029455    5032 scope.go:122] "RemoveContainer" containerID="34290ab3c027e8fe6ae94a2720c387c62af229e217cc7adad72af0748cb713f1"
Oct 30 21:51:07.029802 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:51:07.029779    5032 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34290ab3c027e8fe6ae94a2720c387c62af229e217cc7adad72af0748cb713f1\": container with ID starting with 34290ab3c027e8fe6ae94a2720c387c62af229e217cc7adad72af0748cb713f1 not found: ID does not exist" containerID="34290ab3c027e8fe6ae94a2720c387c62af229e217cc7adad72af0748cb713f1"
Oct 30 21:51:07.029952 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:07.029823    5032 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34290ab3c027e8fe6ae94a2720c387c62af229e217cc7adad72af0748cb713f1"} err="failed to get container status \"34290ab3c027e8fe6ae94a2720c387c62af229e217cc7adad72af0748cb713f1\": rpc error: code = NotFound desc = could not find container \"34290ab3c027e8fe6ae94a2720c387c62af229e217cc7adad72af0748cb713f1\": container with ID starting with 34290ab3c027e8fe6ae94a2720c387c62af229e217cc7adad72af0748cb713f1 not found: ID does not exist"
Oct 30 21:51:07.046002 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:07.045971    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-aad4dd-679948c8f9-bwf4l"]
Oct 30 21:51:07.075665 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:07.073680    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-aad4dd-679948c8f9-bwf4l"]
Oct 30 21:51:07.347354 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:07.347280    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2d543f80-60d7-4c58-ae3d-667f2e577282" path="/var/lib/kubelet/pods/2d543f80-60d7-4c58-ae3d-667f2e577282/volumes"
Oct 30 21:51:09.209785 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:09.209742    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx"]
Oct 30 21:51:10.956097 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:10.956026    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f608f7-7f85c8fcc-lts5r"]
Oct 30 21:51:10.957260 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:10.957227    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f608f7-7f85c8fcc-lts5r" podUID="fd9a1e32-2b8b-4f2c-bcfc-5b7390d9a4a2" containerName="maas-api" containerID="cri-o://7d2acb90d44968ba6a0044fb2dd11ff2be1e9cb0b1dd79119fe0b145c5ffc0f6" gracePeriod=30
Oct 30 21:51:11.296342 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:11.296300    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f608f7-7f85c8fcc-lts5r"
Oct 30 21:51:11.337247 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:11.337212    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fd9a1e32-2b8b-4f2c-bcfc-5b7390d9a4a2-kube-api-access-jpdf4\" (UniqueName: \"kubernetes.io/projected/fd9a1e32-2b8b-4f2c-bcfc-5b7390d9a4a2-kube-api-access-jpdf4\") pod \"fd9a1e32-2b8b-4f2c-bcfc-5b7390d9a4a2\" (UID: \"fd9a1e32-2b8b-4f2c-bcfc-5b7390d9a4a2\") "
Oct 30 21:51:11.337600 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:11.337579    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fd9a1e32-2b8b-4f2c-bcfc-5b7390d9a4a2-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd9a1e32-2b8b-4f2c-bcfc-5b7390d9a4a2-maas-api-tls\") pod \"fd9a1e32-2b8b-4f2c-bcfc-5b7390d9a4a2\" (UID: \"fd9a1e32-2b8b-4f2c-bcfc-5b7390d9a4a2\") "
Oct 30 21:51:11.337842 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:11.337825    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fd9a1e32-2b8b-4f2c-bcfc-5b7390d9a4a2-metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd9a1e32-2b8b-4f2c-bcfc-5b7390d9a4a2-metrics-certs\") pod \"fd9a1e32-2b8b-4f2c-bcfc-5b7390d9a4a2\" (UID: \"fd9a1e32-2b8b-4f2c-bcfc-5b7390d9a4a2\") "
Oct 30 21:51:11.337969 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:11.337956    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fd9a1e32-2b8b-4f2c-bcfc-5b7390d9a4a2-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd9a1e32-2b8b-4f2c-bcfc-5b7390d9a4a2-openshift-service-ca\") pod \"fd9a1e32-2b8b-4f2c-bcfc-5b7390d9a4a2\" (UID: \"fd9a1e32-2b8b-4f2c-bcfc-5b7390d9a4a2\") "
Oct 30 21:51:11.343872 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:11.343836    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd9a1e32-2b8b-4f2c-bcfc-5b7390d9a4a2-openshift-service-ca" pod "fd9a1e32-2b8b-4f2c-bcfc-5b7390d9a4a2" (UID: "fd9a1e32-2b8b-4f2c-bcfc-5b7390d9a4a2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:51:11.355333 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:11.355286    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd9a1e32-2b8b-4f2c-bcfc-5b7390d9a4a2-kube-api-access-jpdf4" pod "fd9a1e32-2b8b-4f2c-bcfc-5b7390d9a4a2" (UID: "fd9a1e32-2b8b-4f2c-bcfc-5b7390d9a4a2"). InnerVolumeSpecName "kube-api-access-jpdf4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:51:11.355333 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:11.355291    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd9a1e32-2b8b-4f2c-bcfc-5b7390d9a4a2-metrics-certs" pod "fd9a1e32-2b8b-4f2c-bcfc-5b7390d9a4a2" (UID: "fd9a1e32-2b8b-4f2c-bcfc-5b7390d9a4a2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:51:11.355804 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:11.355753    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd9a1e32-2b8b-4f2c-bcfc-5b7390d9a4a2-maas-api-tls" pod "fd9a1e32-2b8b-4f2c-bcfc-5b7390d9a4a2" (UID: "fd9a1e32-2b8b-4f2c-bcfc-5b7390d9a4a2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:51:11.440305 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:11.440273    5032 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd9a1e32-2b8b-4f2c-bcfc-5b7390d9a4a2-metrics-certs\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:11.440305 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:11.440300    5032 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd9a1e32-2b8b-4f2c-bcfc-5b7390d9a4a2-openshift-service-ca\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:11.440305 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:11.440312    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jpdf4\" (UniqueName: \"kubernetes.io/projected/fd9a1e32-2b8b-4f2c-bcfc-5b7390d9a4a2-kube-api-access-jpdf4\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:11.440630 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:11.440326    5032 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd9a1e32-2b8b-4f2c-bcfc-5b7390d9a4a2-maas-api-tls\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:12.055072 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:12.055033    5032 generic.go:356] "Generic (PLEG): container finished" podID="fd9a1e32-2b8b-4f2c-bcfc-5b7390d9a4a2" containerID="7d2acb90d44968ba6a0044fb2dd11ff2be1e9cb0b1dd79119fe0b145c5ffc0f6" exitCode=0
Oct 30 21:51:12.055543 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:12.055093    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f608f7-7f85c8fcc-lts5r"
Oct 30 21:51:12.055543 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:12.055125    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f608f7-7f85c8fcc-lts5r" event={"ID":"fd9a1e32-2b8b-4f2c-bcfc-5b7390d9a4a2","Type":"ContainerDied","Data":"7d2acb90d44968ba6a0044fb2dd11ff2be1e9cb0b1dd79119fe0b145c5ffc0f6"}
Oct 30 21:51:12.055543 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:12.055197    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f608f7-7f85c8fcc-lts5r" event={"ID":"fd9a1e32-2b8b-4f2c-bcfc-5b7390d9a4a2","Type":"ContainerDied","Data":"b9efee46401415630b854ded7171e613293f1b8f8f6bb50adb6d8501eccf00f8"}
Oct 30 21:51:12.055543 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:12.055229    5032 scope.go:122] "RemoveContainer" containerID="7d2acb90d44968ba6a0044fb2dd11ff2be1e9cb0b1dd79119fe0b145c5ffc0f6"
Oct 30 21:51:12.067853 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:12.067832    5032 scope.go:122] "RemoveContainer" containerID="7d2acb90d44968ba6a0044fb2dd11ff2be1e9cb0b1dd79119fe0b145c5ffc0f6"
Oct 30 21:51:12.068224 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:51:12.068188    5032 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d2acb90d44968ba6a0044fb2dd11ff2be1e9cb0b1dd79119fe0b145c5ffc0f6\": container with ID starting with 7d2acb90d44968ba6a0044fb2dd11ff2be1e9cb0b1dd79119fe0b145c5ffc0f6 not found: ID does not exist" containerID="7d2acb90d44968ba6a0044fb2dd11ff2be1e9cb0b1dd79119fe0b145c5ffc0f6"
Oct 30 21:51:12.068317 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:12.068226    5032 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d2acb90d44968ba6a0044fb2dd11ff2be1e9cb0b1dd79119fe0b145c5ffc0f6"} err="failed to get container status \"7d2acb90d44968ba6a0044fb2dd11ff2be1e9cb0b1dd79119fe0b145c5ffc0f6\": rpc error: code = NotFound desc = could not find container \"7d2acb90d44968ba6a0044fb2dd11ff2be1e9cb0b1dd79119fe0b145c5ffc0f6\": container with ID starting with 7d2acb90d44968ba6a0044fb2dd11ff2be1e9cb0b1dd79119fe0b145c5ffc0f6 not found: ID does not exist"
Oct 30 21:51:12.082530 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:12.082499    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f608f7-7f85c8fcc-lts5r"]
Oct 30 21:51:12.088220 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:12.088188    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f608f7-7f85c8fcc-lts5r"]
Oct 30 21:51:13.347635 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:13.347594    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fd9a1e32-2b8b-4f2c-bcfc-5b7390d9a4a2" path="/var/lib/kubelet/pods/fd9a1e32-2b8b-4f2c-bcfc-5b7390d9a4a2/volumes"
Oct 30 21:51:14.461306 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:14.461269    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 21:51:14.574382 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:14.557901    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-db0b54-758d689765-f75zr"]
Oct 30 21:51:18.351222 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:51:18.351141    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 21:51:19.558250 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:19.558198    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w2-db0b54-758d689765-f75zr" podUID="00d3249f-1824-4ba1-8c18-6f6ab1f7ff92" containerName="payload-processing" containerID="cri-o://ca16fad36ce0915273285b3cb403453d153d91475661a2941daba98cbb240035" gracePeriod=25
Oct 30 21:51:21.225609 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:21.225569    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw"]
Oct 30 21:51:24.232145 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:24.232102    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx" podUID="34fdf57d-86f6-474f-a4f6-811e5063c405" containerName="main" containerID="cri-o://5f265d992ebd991fccd84b93efd73f1d4b3191b2687f726ef34555318f98777b" gracePeriod=45
Oct 30 21:51:24.531593 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:24.531562    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx"
Oct 30 21:51:24.620908 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:24.620876    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/34fdf57d-86f6-474f-a4f6-811e5063c405-home\" (UniqueName: \"kubernetes.io/empty-dir/34fdf57d-86f6-474f-a4f6-811e5063c405-home\") pod \"34fdf57d-86f6-474f-a4f6-811e5063c405\" (UID: \"34fdf57d-86f6-474f-a4f6-811e5063c405\") "
Oct 30 21:51:24.621089 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:24.620922    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/34fdf57d-86f6-474f-a4f6-811e5063c405-tls-certs\" (UniqueName: \"kubernetes.io/secret/34fdf57d-86f6-474f-a4f6-811e5063c405-tls-certs\") pod \"34fdf57d-86f6-474f-a4f6-811e5063c405\" (UID: \"34fdf57d-86f6-474f-a4f6-811e5063c405\") "
Oct 30 21:51:24.621089 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:24.620959    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/34fdf57d-86f6-474f-a4f6-811e5063c405-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/34fdf57d-86f6-474f-a4f6-811e5063c405-model-cache\") pod \"34fdf57d-86f6-474f-a4f6-811e5063c405\" (UID: \"34fdf57d-86f6-474f-a4f6-811e5063c405\") "
Oct 30 21:51:24.621089 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:24.621045    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/34fdf57d-86f6-474f-a4f6-811e5063c405-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34fdf57d-86f6-474f-a4f6-811e5063c405-tmp-dir\") pod \"34fdf57d-86f6-474f-a4f6-811e5063c405\" (UID: \"34fdf57d-86f6-474f-a4f6-811e5063c405\") "
Oct 30 21:51:24.621089 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:24.621079    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/34fdf57d-86f6-474f-a4f6-811e5063c405-kube-api-access-qtm86\" (UniqueName: \"kubernetes.io/projected/34fdf57d-86f6-474f-a4f6-811e5063c405-kube-api-access-qtm86\") pod \"34fdf57d-86f6-474f-a4f6-811e5063c405\" (UID: \"34fdf57d-86f6-474f-a4f6-811e5063c405\") "
Oct 30 21:51:24.621407 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:24.621185    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/34fdf57d-86f6-474f-a4f6-811e5063c405-dshm\" (UniqueName: \"kubernetes.io/empty-dir/34fdf57d-86f6-474f-a4f6-811e5063c405-dshm\") pod \"34fdf57d-86f6-474f-a4f6-811e5063c405\" (UID: \"34fdf57d-86f6-474f-a4f6-811e5063c405\") "
Oct 30 21:51:24.621407 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:24.621207    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34fdf57d-86f6-474f-a4f6-811e5063c405-home" pod "34fdf57d-86f6-474f-a4f6-811e5063c405" (UID: "34fdf57d-86f6-474f-a4f6-811e5063c405"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:24.621975 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:24.621950    5032 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/34fdf57d-86f6-474f-a4f6-811e5063c405-home\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:24.622120 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:24.622094    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34fdf57d-86f6-474f-a4f6-811e5063c405-tmp-dir" pod "34fdf57d-86f6-474f-a4f6-811e5063c405" (UID: "34fdf57d-86f6-474f-a4f6-811e5063c405"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:24.622195 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:24.622163    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34fdf57d-86f6-474f-a4f6-811e5063c405-model-cache" pod "34fdf57d-86f6-474f-a4f6-811e5063c405" (UID: "34fdf57d-86f6-474f-a4f6-811e5063c405"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:24.625775 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:24.625750    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34fdf57d-86f6-474f-a4f6-811e5063c405-dshm" pod "34fdf57d-86f6-474f-a4f6-811e5063c405" (UID: "34fdf57d-86f6-474f-a4f6-811e5063c405"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:24.625932 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:24.625914    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34fdf57d-86f6-474f-a4f6-811e5063c405-kube-api-access-qtm86" pod "34fdf57d-86f6-474f-a4f6-811e5063c405" (UID: "34fdf57d-86f6-474f-a4f6-811e5063c405"). InnerVolumeSpecName "kube-api-access-qtm86". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:51:24.625999 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:24.625934    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/34fdf57d-86f6-474f-a4f6-811e5063c405-tls-certs" pod "34fdf57d-86f6-474f-a4f6-811e5063c405" (UID: "34fdf57d-86f6-474f-a4f6-811e5063c405"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:51:24.723628 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:24.723594    5032 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/34fdf57d-86f6-474f-a4f6-811e5063c405-tls-certs\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:24.723628 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:24.723621    5032 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/34fdf57d-86f6-474f-a4f6-811e5063c405-model-cache\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:24.723628 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:24.723631    5032 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34fdf57d-86f6-474f-a4f6-811e5063c405-tmp-dir\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:24.723963 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:24.723641    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qtm86\" (UniqueName: \"kubernetes.io/projected/34fdf57d-86f6-474f-a4f6-811e5063c405-kube-api-access-qtm86\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:24.723963 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:24.723650    5032 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/34fdf57d-86f6-474f-a4f6-811e5063c405-dshm\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:25.169340 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:25.169302    5032 generic.go:356] "Generic (PLEG): container finished" podID="34fdf57d-86f6-474f-a4f6-811e5063c405" containerID="5f265d992ebd991fccd84b93efd73f1d4b3191b2687f726ef34555318f98777b" exitCode=0
Oct 30 21:51:25.169554 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:25.169349    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx"
Oct 30 21:51:25.169554 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:25.169394    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx" event={"ID":"34fdf57d-86f6-474f-a4f6-811e5063c405","Type":"ContainerDied","Data":"5f265d992ebd991fccd84b93efd73f1d4b3191b2687f726ef34555318f98777b"}
Oct 30 21:51:25.169554 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:25.169454    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx" event={"ID":"34fdf57d-86f6-474f-a4f6-811e5063c405","Type":"ContainerDied","Data":"ce49f6e2fa90182255a031091f80456f47f97505c675d5476e8d9a23a5a81892"}
Oct 30 21:51:25.169554 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:25.169478    5032 scope.go:122] "RemoveContainer" containerID="5f265d992ebd991fccd84b93efd73f1d4b3191b2687f726ef34555318f98777b"
Oct 30 21:51:25.182284 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:25.182259    5032 scope.go:122] "RemoveContainer" containerID="5f265d992ebd991fccd84b93efd73f1d4b3191b2687f726ef34555318f98777b"
Oct 30 21:51:25.182612 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:51:25.182592    5032 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f265d992ebd991fccd84b93efd73f1d4b3191b2687f726ef34555318f98777b\": container with ID starting with 5f265d992ebd991fccd84b93efd73f1d4b3191b2687f726ef34555318f98777b not found: ID does not exist" containerID="5f265d992ebd991fccd84b93efd73f1d4b3191b2687f726ef34555318f98777b"
Oct 30 21:51:25.182675 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:25.182622    5032 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f265d992ebd991fccd84b93efd73f1d4b3191b2687f726ef34555318f98777b"} err="failed to get container status \"5f265d992ebd991fccd84b93efd73f1d4b3191b2687f726ef34555318f98777b\": rpc error: code = NotFound desc = could not find container \"5f265d992ebd991fccd84b93efd73f1d4b3191b2687f726ef34555318f98777b\": container with ID starting with 5f265d992ebd991fccd84b93efd73f1d4b3191b2687f726ef34555318f98777b not found: ID does not exist"
Oct 30 21:51:25.198269 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:25.198241    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx"]
Oct 30 21:51:25.202462 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:25.202435    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-db0b54/premium-w2-db0b54-kserve-84695b566c-4p6rx"]
Oct 30 21:51:25.354950 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:25.354909    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="34fdf57d-86f6-474f-a4f6-811e5063c405" path="/var/lib/kubelet/pods/34fdf57d-86f6-474f-a4f6-811e5063c405/volumes"
Oct 30 21:51:26.346600 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:26.346561    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc"]
Oct 30 21:51:26.351810 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:26.351779    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg"]
Oct 30 21:51:26.405041 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:26.405006    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w2-db0b54-758d689765-f75zr" podUID="00d3249f-1824-4ba1-8c18-6f6ab1f7ff92" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:51:26.546373 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:26.546334    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-0fa864-5d85948857-tw7k6"]
Oct 30 21:51:26.567187 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:26.567126    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-0fa864-76b48d9455-wxt6j"]
Oct 30 21:51:30.483689 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:30.483654    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-0c62ab-5d54447cf4-6c8wt"]
Oct 30 21:51:30.484548 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:30.484514    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-0c62ab-5d54447cf4-6c8wt" podUID="6ce45345-9ad2-4e54-b19b-ffa9a73db1be" containerName="maas-api" containerID="cri-o://9d8fcddf56ea2414d0625b1298728e94385f773b114ac206d8144c5b05976526" gracePeriod=30
Oct 30 21:51:30.500078 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:30.500042    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-0c62ab-5d54447cf4-6c8wt" podUID="6ce45345-9ad2-4e54-b19b-ffa9a73db1be" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.109:8443/health\": dial tcp 10.132.0.109:8443: connect: connection refused"
Oct 30 21:51:30.899394 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:30.898603    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-0c62ab-5d54447cf4-6c8wt"
Oct 30 21:51:31.059288 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:31.059207    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6ce45345-9ad2-4e54-b19b-ffa9a73db1be-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ce45345-9ad2-4e54-b19b-ffa9a73db1be-openshift-service-ca\") pod \"6ce45345-9ad2-4e54-b19b-ffa9a73db1be\" (UID: \"6ce45345-9ad2-4e54-b19b-ffa9a73db1be\") "
Oct 30 21:51:31.059476 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:31.059306    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6ce45345-9ad2-4e54-b19b-ffa9a73db1be-kube-api-access-9jpf4\" (UniqueName: \"kubernetes.io/projected/6ce45345-9ad2-4e54-b19b-ffa9a73db1be-kube-api-access-9jpf4\") pod \"6ce45345-9ad2-4e54-b19b-ffa9a73db1be\" (UID: \"6ce45345-9ad2-4e54-b19b-ffa9a73db1be\") "
Oct 30 21:51:31.059476 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:31.059341    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6ce45345-9ad2-4e54-b19b-ffa9a73db1be-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ce45345-9ad2-4e54-b19b-ffa9a73db1be-maas-api-tls\") pod \"6ce45345-9ad2-4e54-b19b-ffa9a73db1be\" (UID: \"6ce45345-9ad2-4e54-b19b-ffa9a73db1be\") "
Oct 30 21:51:31.059582 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:31.059484    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6ce45345-9ad2-4e54-b19b-ffa9a73db1be-metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ce45345-9ad2-4e54-b19b-ffa9a73db1be-metrics-certs\") pod \"6ce45345-9ad2-4e54-b19b-ffa9a73db1be\" (UID: \"6ce45345-9ad2-4e54-b19b-ffa9a73db1be\") "
Oct 30 21:51:31.059641 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:31.059586    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ce45345-9ad2-4e54-b19b-ffa9a73db1be-openshift-service-ca" pod "6ce45345-9ad2-4e54-b19b-ffa9a73db1be" (UID: "6ce45345-9ad2-4e54-b19b-ffa9a73db1be"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:51:31.059890 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:31.059872    5032 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ce45345-9ad2-4e54-b19b-ffa9a73db1be-openshift-service-ca\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:31.061650 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:31.061620    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ce45345-9ad2-4e54-b19b-ffa9a73db1be-metrics-certs" pod "6ce45345-9ad2-4e54-b19b-ffa9a73db1be" (UID: "6ce45345-9ad2-4e54-b19b-ffa9a73db1be"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:51:31.061752 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:31.061716    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ce45345-9ad2-4e54-b19b-ffa9a73db1be-maas-api-tls" pod "6ce45345-9ad2-4e54-b19b-ffa9a73db1be" (UID: "6ce45345-9ad2-4e54-b19b-ffa9a73db1be"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:51:31.061796 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:31.061765    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ce45345-9ad2-4e54-b19b-ffa9a73db1be-kube-api-access-9jpf4" pod "6ce45345-9ad2-4e54-b19b-ffa9a73db1be" (UID: "6ce45345-9ad2-4e54-b19b-ffa9a73db1be"). InnerVolumeSpecName "kube-api-access-9jpf4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:51:31.161614 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:31.161574    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9jpf4\" (UniqueName: \"kubernetes.io/projected/6ce45345-9ad2-4e54-b19b-ffa9a73db1be-kube-api-access-9jpf4\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:31.161614 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:31.161607    5032 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ce45345-9ad2-4e54-b19b-ffa9a73db1be-maas-api-tls\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:31.161614 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:31.161621    5032 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ce45345-9ad2-4e54-b19b-ffa9a73db1be-metrics-certs\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:31.227454 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:31.227331    5032 generic.go:356] "Generic (PLEG): container finished" podID="6ce45345-9ad2-4e54-b19b-ffa9a73db1be" containerID="9d8fcddf56ea2414d0625b1298728e94385f773b114ac206d8144c5b05976526" exitCode=0
Oct 30 21:51:31.227454 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:31.227414    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-0c62ab-5d54447cf4-6c8wt" event={"ID":"6ce45345-9ad2-4e54-b19b-ffa9a73db1be","Type":"ContainerDied","Data":"9d8fcddf56ea2414d0625b1298728e94385f773b114ac206d8144c5b05976526"}
Oct 30 21:51:31.227668 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:31.227468    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-0c62ab-5d54447cf4-6c8wt" event={"ID":"6ce45345-9ad2-4e54-b19b-ffa9a73db1be","Type":"ContainerDied","Data":"4dc350615a651ec827bba2752cf4e8cc3dd74faa073d5f9d069afb45dbf8dca2"}
Oct 30 21:51:31.227668 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:31.227475    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-0c62ab-5d54447cf4-6c8wt"
Oct 30 21:51:31.227668 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:31.227498    5032 scope.go:122] "RemoveContainer" containerID="9d8fcddf56ea2414d0625b1298728e94385f773b114ac206d8144c5b05976526"
Oct 30 21:51:31.253125 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:31.251910    5032 scope.go:122] "RemoveContainer" containerID="9d8fcddf56ea2414d0625b1298728e94385f773b114ac206d8144c5b05976526"
Oct 30 21:51:31.253125 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:51:31.252542    5032 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d8fcddf56ea2414d0625b1298728e94385f773b114ac206d8144c5b05976526\": container with ID starting with 9d8fcddf56ea2414d0625b1298728e94385f773b114ac206d8144c5b05976526 not found: ID does not exist" containerID="9d8fcddf56ea2414d0625b1298728e94385f773b114ac206d8144c5b05976526"
Oct 30 21:51:31.253125 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:31.252580    5032 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d8fcddf56ea2414d0625b1298728e94385f773b114ac206d8144c5b05976526"} err="failed to get container status \"9d8fcddf56ea2414d0625b1298728e94385f773b114ac206d8144c5b05976526\": rpc error: code = NotFound desc = could not find container \"9d8fcddf56ea2414d0625b1298728e94385f773b114ac206d8144c5b05976526\": container with ID starting with 9d8fcddf56ea2414d0625b1298728e94385f773b114ac206d8144c5b05976526 not found: ID does not exist"
Oct 30 21:51:31.260323 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:31.260294    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-0c62ab-5d54447cf4-6c8wt"]
Oct 30 21:51:31.263749 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:31.263723    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-0c62ab-5d54447cf4-6c8wt"]
Oct 30 21:51:31.356927 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:31.356833    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6ce45345-9ad2-4e54-b19b-ffa9a73db1be" path="/var/lib/kubelet/pods/6ce45345-9ad2-4e54-b19b-ffa9a73db1be/volumes"
Oct 30 21:51:31.547276 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:31.547232    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w4-0fa864-5d85948857-tw7k6" podUID="76de8529-43f7-4285-864b-f8242fca2cae" containerName="payload-processing" containerID="cri-o://8a53fc244ecaf6746c1a1d24f9cec8a7e839613c0bb15552cb0610bb93f1b446" gracePeriod=25
Oct 30 21:51:31.567946 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:31.567903    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-0fa864-76b48d9455-wxt6j" podUID="18d702a7-00ad-4a6b-8de5-806e5900cd7d" containerName="payload-pre-processing" containerID="cri-o://0d899832b76b126f7ee364e0454d922673062cbfb2def07182a739f32bf85d4f" gracePeriod=25
Oct 30 21:51:31.600128 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:31.600093    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-1d8034-6bbd97dcf7-l2gd8"]
Oct 30 21:51:31.623243 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:31.623160    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-1d8034-5b96976fc5-hsr78"]
Oct 30 21:51:36.251807 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:36.251744    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw" podUID="914f7544-6ef8-4496-b98d-b7ebc4123cda" containerName="main" containerID="cri-o://a76aa49cf87523727b92511ed06e0157f5e68250327deea402ed3da6905be90e" gracePeriod=45
Oct 30 21:51:36.401479 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:36.401438    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w4-0fa864-5d85948857-tw7k6" podUID="76de8529-43f7-4285-864b-f8242fca2cae" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:51:36.401683 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:36.401489    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w2-db0b54-758d689765-f75zr" podUID="00d3249f-1824-4ba1-8c18-6f6ab1f7ff92" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:51:36.401769 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:36.401678    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-0fa864-76b48d9455-wxt6j" podUID="18d702a7-00ad-4a6b-8de5-806e5900cd7d" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:51:36.552886 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:36.552493    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw"
Oct 30 21:51:36.601793 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:36.601734    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w3-1d8034-6bbd97dcf7-l2gd8" podUID="e25d6f68-6e47-449f-b02f-03fff2b2bb30" containerName="payload-processing" containerID="cri-o://5e56a198bf2559217d743ea619f914b5b821cf25512ea712631b76860cd9dd7e" gracePeriod=25
Oct 30 21:51:36.623900 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:36.623820    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-1d8034-5b96976fc5-hsr78" podUID="27aef427-6b1e-4845-b8d9-a83cba81b0e0" containerName="payload-pre-processing" containerID="cri-o://4a6b5c624625d96186bc32eb917cd7d6554b6357f49b5e34177e00d4855580ff" gracePeriod=25
Oct 30 21:51:36.669008 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:36.668973    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/914f7544-6ef8-4496-b98d-b7ebc4123cda-tls-certs\" (UniqueName: \"kubernetes.io/secret/914f7544-6ef8-4496-b98d-b7ebc4123cda-tls-certs\") pod \"914f7544-6ef8-4496-b98d-b7ebc4123cda\" (UID: \"914f7544-6ef8-4496-b98d-b7ebc4123cda\") "
Oct 30 21:51:36.669190 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:36.669034    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/914f7544-6ef8-4496-b98d-b7ebc4123cda-dshm\" (UniqueName: \"kubernetes.io/empty-dir/914f7544-6ef8-4496-b98d-b7ebc4123cda-dshm\") pod \"914f7544-6ef8-4496-b98d-b7ebc4123cda\" (UID: \"914f7544-6ef8-4496-b98d-b7ebc4123cda\") "
Oct 30 21:51:36.669190 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:36.669109    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/914f7544-6ef8-4496-b98d-b7ebc4123cda-home\" (UniqueName: \"kubernetes.io/empty-dir/914f7544-6ef8-4496-b98d-b7ebc4123cda-home\") pod \"914f7544-6ef8-4496-b98d-b7ebc4123cda\" (UID: \"914f7544-6ef8-4496-b98d-b7ebc4123cda\") "
Oct 30 21:51:36.669447 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:36.669379    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/914f7544-6ef8-4496-b98d-b7ebc4123cda-home" pod "914f7544-6ef8-4496-b98d-b7ebc4123cda" (UID: "914f7544-6ef8-4496-b98d-b7ebc4123cda"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:36.669759 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:36.669730    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/914f7544-6ef8-4496-b98d-b7ebc4123cda-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/914f7544-6ef8-4496-b98d-b7ebc4123cda-tmp-dir\") pod \"914f7544-6ef8-4496-b98d-b7ebc4123cda\" (UID: \"914f7544-6ef8-4496-b98d-b7ebc4123cda\") "
Oct 30 21:51:36.670160 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:36.670048    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/914f7544-6ef8-4496-b98d-b7ebc4123cda-tmp-dir" pod "914f7544-6ef8-4496-b98d-b7ebc4123cda" (UID: "914f7544-6ef8-4496-b98d-b7ebc4123cda"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:36.670322 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:36.670306    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/914f7544-6ef8-4496-b98d-b7ebc4123cda-kube-api-access-9hsrd\" (UniqueName: \"kubernetes.io/projected/914f7544-6ef8-4496-b98d-b7ebc4123cda-kube-api-access-9hsrd\") pod \"914f7544-6ef8-4496-b98d-b7ebc4123cda\" (UID: \"914f7544-6ef8-4496-b98d-b7ebc4123cda\") "
Oct 30 21:51:36.671406 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:36.670400    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/914f7544-6ef8-4496-b98d-b7ebc4123cda-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/914f7544-6ef8-4496-b98d-b7ebc4123cda-model-cache\") pod \"914f7544-6ef8-4496-b98d-b7ebc4123cda\" (UID: \"914f7544-6ef8-4496-b98d-b7ebc4123cda\") "
Oct 30 21:51:36.671406 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:36.670728    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/914f7544-6ef8-4496-b98d-b7ebc4123cda-model-cache" pod "914f7544-6ef8-4496-b98d-b7ebc4123cda" (UID: "914f7544-6ef8-4496-b98d-b7ebc4123cda"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:36.671406 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:36.671022    5032 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/914f7544-6ef8-4496-b98d-b7ebc4123cda-model-cache\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:36.671406 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:36.671035    5032 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/914f7544-6ef8-4496-b98d-b7ebc4123cda-home\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:36.671406 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:36.671060    5032 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/914f7544-6ef8-4496-b98d-b7ebc4123cda-tmp-dir\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:36.674074 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:36.674051    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/914f7544-6ef8-4496-b98d-b7ebc4123cda-dshm" pod "914f7544-6ef8-4496-b98d-b7ebc4123cda" (UID: "914f7544-6ef8-4496-b98d-b7ebc4123cda"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:36.676406 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:36.676338    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/914f7544-6ef8-4496-b98d-b7ebc4123cda-tls-certs" pod "914f7544-6ef8-4496-b98d-b7ebc4123cda" (UID: "914f7544-6ef8-4496-b98d-b7ebc4123cda"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:51:36.677431 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:36.677406    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/914f7544-6ef8-4496-b98d-b7ebc4123cda-kube-api-access-9hsrd" pod "914f7544-6ef8-4496-b98d-b7ebc4123cda" (UID: "914f7544-6ef8-4496-b98d-b7ebc4123cda"). InnerVolumeSpecName "kube-api-access-9hsrd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:51:36.772388 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:36.772282    5032 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/914f7544-6ef8-4496-b98d-b7ebc4123cda-tls-certs\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:36.772388 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:36.772323    5032 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/914f7544-6ef8-4496-b98d-b7ebc4123cda-dshm\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:36.772388 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:36.772339    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9hsrd\" (UniqueName: \"kubernetes.io/projected/914f7544-6ef8-4496-b98d-b7ebc4123cda-kube-api-access-9hsrd\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:37.279304 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:37.279271    5032 generic.go:356] "Generic (PLEG): container finished" podID="914f7544-6ef8-4496-b98d-b7ebc4123cda" containerID="a76aa49cf87523727b92511ed06e0157f5e68250327deea402ed3da6905be90e" exitCode=0
Oct 30 21:51:37.279857 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:37.279313    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw"
Oct 30 21:51:37.279857 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:37.279346    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw" event={"ID":"914f7544-6ef8-4496-b98d-b7ebc4123cda","Type":"ContainerDied","Data":"a76aa49cf87523727b92511ed06e0157f5e68250327deea402ed3da6905be90e"}
Oct 30 21:51:37.279857 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:37.279397    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw" event={"ID":"914f7544-6ef8-4496-b98d-b7ebc4123cda","Type":"ContainerDied","Data":"3bf1d930d5f2c23358af0342c0fd8205b60e23ab25f6251002d4947cb989ec81"}
Oct 30 21:51:37.279857 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:37.279414    5032 scope.go:122] "RemoveContainer" containerID="a76aa49cf87523727b92511ed06e0157f5e68250327deea402ed3da6905be90e"
Oct 30 21:51:37.292457 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:37.292436    5032 scope.go:122] "RemoveContainer" containerID="a76aa49cf87523727b92511ed06e0157f5e68250327deea402ed3da6905be90e"
Oct 30 21:51:37.292792 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:51:37.292774    5032 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a76aa49cf87523727b92511ed06e0157f5e68250327deea402ed3da6905be90e\": container with ID starting with a76aa49cf87523727b92511ed06e0157f5e68250327deea402ed3da6905be90e not found: ID does not exist" containerID="a76aa49cf87523727b92511ed06e0157f5e68250327deea402ed3da6905be90e"
Oct 30 21:51:37.292857 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:37.292799    5032 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a76aa49cf87523727b92511ed06e0157f5e68250327deea402ed3da6905be90e"} err="failed to get container status \"a76aa49cf87523727b92511ed06e0157f5e68250327deea402ed3da6905be90e\": rpc error: code = NotFound desc = could not find container \"a76aa49cf87523727b92511ed06e0157f5e68250327deea402ed3da6905be90e\": container with ID starting with a76aa49cf87523727b92511ed06e0157f5e68250327deea402ed3da6905be90e not found: ID does not exist"
Oct 30 21:51:37.310458 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:37.310425    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw"]
Oct 30 21:51:37.316511 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:37.316486    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-0fa864/distinct-2-w4-0fa864-kserve-5d6b444487-l4ljw"]
Oct 30 21:51:37.348297 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:37.348266    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="914f7544-6ef8-4496-b98d-b7ebc4123cda" path="/var/lib/kubelet/pods/914f7544-6ef8-4496-b98d-b7ebc4123cda/volumes"
Oct 30 21:51:37.411986 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:37.411950    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w3-1d8034-6bbd97dcf7-l2gd8" podUID="e25d6f68-6e47-449f-b02f-03fff2b2bb30" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:51:38.556689 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:38.556653    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-0fa864-6bc77cf54d-2zd6r"]
Oct 30 21:51:38.557682 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:38.556907    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-0fa864-6bc77cf54d-2zd6r" podUID="b19b4f72-13ba-4a3f-b3a8-73bc3c4c1d4b" containerName="maas-api" containerID="cri-o://dee681b536a7a792f6d1ed3d190ec48d01d610ee18b3f6b64155a9fbf6e78368" gracePeriod=30
Oct 30 21:51:38.961825 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:38.961785    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-0fa864-6bc77cf54d-2zd6r"
Oct 30 21:51:39.050187 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:39.050153    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b19b4f72-13ba-4a3f-b3a8-73bc3c4c1d4b-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b19b4f72-13ba-4a3f-b3a8-73bc3c4c1d4b-openshift-service-ca\") pod \"b19b4f72-13ba-4a3f-b3a8-73bc3c4c1d4b\" (UID: \"b19b4f72-13ba-4a3f-b3a8-73bc3c4c1d4b\") "
Oct 30 21:51:39.050355 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:39.050245    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b19b4f72-13ba-4a3f-b3a8-73bc3c4c1d4b-kube-api-access-g8pqq\" (UniqueName: \"kubernetes.io/projected/b19b4f72-13ba-4a3f-b3a8-73bc3c4c1d4b-kube-api-access-g8pqq\") pod \"b19b4f72-13ba-4a3f-b3a8-73bc3c4c1d4b\" (UID: \"b19b4f72-13ba-4a3f-b3a8-73bc3c4c1d4b\") "
Oct 30 21:51:39.050459 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:39.050440    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b19b4f72-13ba-4a3f-b3a8-73bc3c4c1d4b-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b19b4f72-13ba-4a3f-b3a8-73bc3c4c1d4b-maas-api-tls\") pod \"b19b4f72-13ba-4a3f-b3a8-73bc3c4c1d4b\" (UID: \"b19b4f72-13ba-4a3f-b3a8-73bc3c4c1d4b\") "
Oct 30 21:51:39.050584 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:39.050564    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b19b4f72-13ba-4a3f-b3a8-73bc3c4c1d4b-openshift-service-ca" pod "b19b4f72-13ba-4a3f-b3a8-73bc3c4c1d4b" (UID: "b19b4f72-13ba-4a3f-b3a8-73bc3c4c1d4b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:51:39.050651 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:39.050587    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b19b4f72-13ba-4a3f-b3a8-73bc3c4c1d4b-metrics-certs\" (UniqueName: \"kubernetes.io/secret/b19b4f72-13ba-4a3f-b3a8-73bc3c4c1d4b-metrics-certs\") pod \"b19b4f72-13ba-4a3f-b3a8-73bc3c4c1d4b\" (UID: \"b19b4f72-13ba-4a3f-b3a8-73bc3c4c1d4b\") "
Oct 30 21:51:39.051203 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:39.051179    5032 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b19b4f72-13ba-4a3f-b3a8-73bc3c4c1d4b-openshift-service-ca\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:39.052586 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:39.052559    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b19b4f72-13ba-4a3f-b3a8-73bc3c4c1d4b-maas-api-tls" pod "b19b4f72-13ba-4a3f-b3a8-73bc3c4c1d4b" (UID: "b19b4f72-13ba-4a3f-b3a8-73bc3c4c1d4b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:51:39.052992 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:39.052972    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b19b4f72-13ba-4a3f-b3a8-73bc3c4c1d4b-kube-api-access-g8pqq" pod "b19b4f72-13ba-4a3f-b3a8-73bc3c4c1d4b" (UID: "b19b4f72-13ba-4a3f-b3a8-73bc3c4c1d4b"). InnerVolumeSpecName "kube-api-access-g8pqq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:51:39.053065 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:39.053012    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b19b4f72-13ba-4a3f-b3a8-73bc3c4c1d4b-metrics-certs" pod "b19b4f72-13ba-4a3f-b3a8-73bc3c4c1d4b" (UID: "b19b4f72-13ba-4a3f-b3a8-73bc3c4c1d4b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:51:39.152881 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:39.152847    5032 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b19b4f72-13ba-4a3f-b3a8-73bc3c4c1d4b-maas-api-tls\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:39.152881 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:39.152879    5032 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b19b4f72-13ba-4a3f-b3a8-73bc3c4c1d4b-metrics-certs\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:39.153094 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:39.152891    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g8pqq\" (UniqueName: \"kubernetes.io/projected/b19b4f72-13ba-4a3f-b3a8-73bc3c4c1d4b-kube-api-access-g8pqq\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:39.296824 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:39.296787    5032 generic.go:356] "Generic (PLEG): container finished" podID="b19b4f72-13ba-4a3f-b3a8-73bc3c4c1d4b" containerID="dee681b536a7a792f6d1ed3d190ec48d01d610ee18b3f6b64155a9fbf6e78368" exitCode=0
Oct 30 21:51:39.297004 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:39.296835    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-0fa864-6bc77cf54d-2zd6r"
Oct 30 21:51:39.297004 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:39.296843    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-0fa864-6bc77cf54d-2zd6r" event={"ID":"b19b4f72-13ba-4a3f-b3a8-73bc3c4c1d4b","Type":"ContainerDied","Data":"dee681b536a7a792f6d1ed3d190ec48d01d610ee18b3f6b64155a9fbf6e78368"}
Oct 30 21:51:39.297004 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:39.296892    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-0fa864-6bc77cf54d-2zd6r" event={"ID":"b19b4f72-13ba-4a3f-b3a8-73bc3c4c1d4b","Type":"ContainerDied","Data":"d1b85680cc7da8155f8374da38786c2084dd27dfc30aa088f0a372ef545ea31f"}
Oct 30 21:51:39.297004 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:39.296916    5032 scope.go:122] "RemoveContainer" containerID="dee681b536a7a792f6d1ed3d190ec48d01d610ee18b3f6b64155a9fbf6e78368"
Oct 30 21:51:39.309010 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:39.308980    5032 scope.go:122] "RemoveContainer" containerID="dee681b536a7a792f6d1ed3d190ec48d01d610ee18b3f6b64155a9fbf6e78368"
Oct 30 21:51:39.309345 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:51:39.309318    5032 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dee681b536a7a792f6d1ed3d190ec48d01d610ee18b3f6b64155a9fbf6e78368\": container with ID starting with dee681b536a7a792f6d1ed3d190ec48d01d610ee18b3f6b64155a9fbf6e78368 not found: ID does not exist" containerID="dee681b536a7a792f6d1ed3d190ec48d01d610ee18b3f6b64155a9fbf6e78368"
Oct 30 21:51:39.309455 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:39.309356    5032 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dee681b536a7a792f6d1ed3d190ec48d01d610ee18b3f6b64155a9fbf6e78368"} err="failed to get container status \"dee681b536a7a792f6d1ed3d190ec48d01d610ee18b3f6b64155a9fbf6e78368\": rpc error: code = NotFound desc = could not find container \"dee681b536a7a792f6d1ed3d190ec48d01d610ee18b3f6b64155a9fbf6e78368\": container with ID starting with dee681b536a7a792f6d1ed3d190ec48d01d610ee18b3f6b64155a9fbf6e78368 not found: ID does not exist"
Oct 30 21:51:39.352412 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:39.340057    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-0fa864-6bc77cf54d-2zd6r"]
Oct 30 21:51:39.370602 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:39.370568    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-0fa864-6bc77cf54d-2zd6r"]
Oct 30 21:51:39.864752 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:39.864723    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-db0b54-758d689765-f75zr"
Oct 30 21:51:39.980174 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:39.980144    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/00d3249f-1824-4ba1-8c18-6f6ab1f7ff92-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00d3249f-1824-4ba1-8c18-6f6ab1f7ff92-plugins-config-volume\") pod \"00d3249f-1824-4ba1-8c18-6f6ab1f7ff92\" (UID: \"00d3249f-1824-4ba1-8c18-6f6ab1f7ff92\") "
Oct 30 21:51:39.980350 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:39.980235    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/00d3249f-1824-4ba1-8c18-6f6ab1f7ff92-kube-api-access-jzfrv\" (UniqueName: \"kubernetes.io/projected/00d3249f-1824-4ba1-8c18-6f6ab1f7ff92-kube-api-access-jzfrv\") pod \"00d3249f-1824-4ba1-8c18-6f6ab1f7ff92\" (UID: \"00d3249f-1824-4ba1-8c18-6f6ab1f7ff92\") "
Oct 30 21:51:39.980602 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:39.980578    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00d3249f-1824-4ba1-8c18-6f6ab1f7ff92-plugins-config-volume" pod "00d3249f-1824-4ba1-8c18-6f6ab1f7ff92" (UID: "00d3249f-1824-4ba1-8c18-6f6ab1f7ff92"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:51:39.980938 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:39.980922    5032 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00d3249f-1824-4ba1-8c18-6f6ab1f7ff92-plugins-config-volume\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:39.982440 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:39.982415    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00d3249f-1824-4ba1-8c18-6f6ab1f7ff92-kube-api-access-jzfrv" pod "00d3249f-1824-4ba1-8c18-6f6ab1f7ff92" (UID: "00d3249f-1824-4ba1-8c18-6f6ab1f7ff92"). InnerVolumeSpecName "kube-api-access-jzfrv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:51:40.082553 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:40.082469    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jzfrv\" (UniqueName: \"kubernetes.io/projected/00d3249f-1824-4ba1-8c18-6f6ab1f7ff92-kube-api-access-jzfrv\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:40.306738 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:40.306706    5032 generic.go:356] "Generic (PLEG): container finished" podID="00d3249f-1824-4ba1-8c18-6f6ab1f7ff92" containerID="ca16fad36ce0915273285b3cb403453d153d91475661a2941daba98cbb240035" exitCode=0
Oct 30 21:51:40.308110 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:40.306884    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-db0b54-758d689765-f75zr" event={"ID":"00d3249f-1824-4ba1-8c18-6f6ab1f7ff92","Type":"ContainerDied","Data":"ca16fad36ce0915273285b3cb403453d153d91475661a2941daba98cbb240035"}
Oct 30 21:51:40.308110 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:40.306945    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-db0b54-758d689765-f75zr" event={"ID":"00d3249f-1824-4ba1-8c18-6f6ab1f7ff92","Type":"ContainerDied","Data":"f8c381b8d7f4b62de6bf349613b8db4c67903d9296900cacf420028e2c68ac53"}
Oct 30 21:51:40.308110 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:40.306981    5032 scope.go:122] "RemoveContainer" containerID="ca16fad36ce0915273285b3cb403453d153d91475661a2941daba98cbb240035"
Oct 30 21:51:40.308110 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:40.306990    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-db0b54-758d689765-f75zr"
Oct 30 21:51:40.332671 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:40.331416    5032 scope.go:122] "RemoveContainer" containerID="ca16fad36ce0915273285b3cb403453d153d91475661a2941daba98cbb240035"
Oct 30 21:51:40.332931 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:51:40.332903    5032 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca16fad36ce0915273285b3cb403453d153d91475661a2941daba98cbb240035\": container with ID starting with ca16fad36ce0915273285b3cb403453d153d91475661a2941daba98cbb240035 not found: ID does not exist" containerID="ca16fad36ce0915273285b3cb403453d153d91475661a2941daba98cbb240035"
Oct 30 21:51:40.333080 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:40.332939    5032 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca16fad36ce0915273285b3cb403453d153d91475661a2941daba98cbb240035"} err="failed to get container status \"ca16fad36ce0915273285b3cb403453d153d91475661a2941daba98cbb240035\": rpc error: code = NotFound desc = could not find container \"ca16fad36ce0915273285b3cb403453d153d91475661a2941daba98cbb240035\": container with ID starting with ca16fad36ce0915273285b3cb403453d153d91475661a2941daba98cbb240035 not found: ID does not exist"
Oct 30 21:51:40.353632 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:40.353599    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf"]
Oct 30 21:51:40.354380 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:40.354315    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf" podUID="c0ca92b2-a242-47f5-9cc6-ee04860b5692" containerName="istio-proxy" containerID="cri-o://b2b97753a9fba276652fbba6d1514744ba74cfd3f791c70e1e8b7d7ec5c60cee" gracePeriod=30
Oct 30 21:51:40.366685 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:40.366644    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-db0b54-758d689765-f75zr"]
Oct 30 21:51:40.370128 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:40.370100    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm"]
Oct 30 21:51:40.370317 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:40.370293    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm" podUID="6b99e2de-e8a0-4566-b14c-dd627c74edf3" containerName="istio-proxy" containerID="cri-o://215dd4edb5877e9e26232807f565e61d0266b5ee2227ae963f89fae8a84effb6" gracePeriod=30
Oct 30 21:51:40.377800 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:40.377773    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-db0b54-758d689765-f75zr"]
Oct 30 21:51:41.349220 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:41.349173    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="00d3249f-1824-4ba1-8c18-6f6ab1f7ff92" path="/var/lib/kubelet/pods/00d3249f-1824-4ba1-8c18-6f6ab1f7ff92/volumes"
Oct 30 21:51:41.350495 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:41.349988    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b19b4f72-13ba-4a3f-b3a8-73bc3c4c1d4b" path="/var/lib/kubelet/pods/b19b4f72-13ba-4a3f-b3a8-73bc3c4c1d4b/volumes"
Oct 30 21:51:41.370071 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:41.370037    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc" podUID="23b97dbb-8488-4c44-a810-882ea18ed919" containerName="main" containerID="cri-o://55fec9098ae03c756d258df3c3d2d75a5db97bfdeaad74957831d75d1af4449b" gracePeriod=45
Oct 30 21:51:41.376965 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:41.376936    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg" podUID="2d0ea058-b6ca-4181-8823-2b719883e545" containerName="main" containerID="cri-o://6797a8c03a4ba0206dd9a003cee3991819a6ab7cce24fc5acfb065cc7557ca8a" gracePeriod=45
Oct 30 21:51:41.742696 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:41.742663    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc"
Oct 30 21:51:41.746481 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:41.746454    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg"
Oct 30 21:51:41.803973 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:41.803935    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2d0ea058-b6ca-4181-8823-2b719883e545-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d0ea058-b6ca-4181-8823-2b719883e545-tmp-dir\") pod \"2d0ea058-b6ca-4181-8823-2b719883e545\" (UID: \"2d0ea058-b6ca-4181-8823-2b719883e545\") "
Oct 30 21:51:41.804148 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:41.803988    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/23b97dbb-8488-4c44-a810-882ea18ed919-home\" (UniqueName: \"kubernetes.io/empty-dir/23b97dbb-8488-4c44-a810-882ea18ed919-home\") pod \"23b97dbb-8488-4c44-a810-882ea18ed919\" (UID: \"23b97dbb-8488-4c44-a810-882ea18ed919\") "
Oct 30 21:51:41.804148 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:41.804082    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2d0ea058-b6ca-4181-8823-2b719883e545-kube-api-access-t9zz8\" (UniqueName: \"kubernetes.io/projected/2d0ea058-b6ca-4181-8823-2b719883e545-kube-api-access-t9zz8\") pod \"2d0ea058-b6ca-4181-8823-2b719883e545\" (UID: \"2d0ea058-b6ca-4181-8823-2b719883e545\") "
Oct 30 21:51:41.804290 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:41.804163    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2d0ea058-b6ca-4181-8823-2b719883e545-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d0ea058-b6ca-4181-8823-2b719883e545-model-cache\") pod \"2d0ea058-b6ca-4181-8823-2b719883e545\" (UID: \"2d0ea058-b6ca-4181-8823-2b719883e545\") "
Oct 30 21:51:41.804290 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:41.804207    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/23b97dbb-8488-4c44-a810-882ea18ed919-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/23b97dbb-8488-4c44-a810-882ea18ed919-model-cache\") pod \"23b97dbb-8488-4c44-a810-882ea18ed919\" (UID: \"23b97dbb-8488-4c44-a810-882ea18ed919\") "
Oct 30 21:51:41.804290 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:41.804257    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2d0ea058-b6ca-4181-8823-2b719883e545-dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d0ea058-b6ca-4181-8823-2b719883e545-dshm\") pod \"2d0ea058-b6ca-4181-8823-2b719883e545\" (UID: \"2d0ea058-b6ca-4181-8823-2b719883e545\") "
Oct 30 21:51:41.804290 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:41.804286    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2d0ea058-b6ca-4181-8823-2b719883e545-home\" (UniqueName: \"kubernetes.io/empty-dir/2d0ea058-b6ca-4181-8823-2b719883e545-home\") pod \"2d0ea058-b6ca-4181-8823-2b719883e545\" (UID: \"2d0ea058-b6ca-4181-8823-2b719883e545\") "
Oct 30 21:51:41.804519 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:41.804395    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/23b97dbb-8488-4c44-a810-882ea18ed919-kube-api-access-bzpfw\" (UniqueName: \"kubernetes.io/projected/23b97dbb-8488-4c44-a810-882ea18ed919-kube-api-access-bzpfw\") pod \"23b97dbb-8488-4c44-a810-882ea18ed919\" (UID: \"23b97dbb-8488-4c44-a810-882ea18ed919\") "
Oct 30 21:51:41.804519 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:41.804459    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2d0ea058-b6ca-4181-8823-2b719883e545-tls-certs\" (UniqueName: \"kubernetes.io/secret/2d0ea058-b6ca-4181-8823-2b719883e545-tls-certs\") pod \"2d0ea058-b6ca-4181-8823-2b719883e545\" (UID: \"2d0ea058-b6ca-4181-8823-2b719883e545\") "
Oct 30 21:51:41.804644 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:41.804547    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/23b97dbb-8488-4c44-a810-882ea18ed919-dshm\" (UniqueName: \"kubernetes.io/empty-dir/23b97dbb-8488-4c44-a810-882ea18ed919-dshm\") pod \"23b97dbb-8488-4c44-a810-882ea18ed919\" (UID: \"23b97dbb-8488-4c44-a810-882ea18ed919\") "
Oct 30 21:51:41.804644 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:41.804604    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/23b97dbb-8488-4c44-a810-882ea18ed919-tls-certs\" (UniqueName: \"kubernetes.io/secret/23b97dbb-8488-4c44-a810-882ea18ed919-tls-certs\") pod \"23b97dbb-8488-4c44-a810-882ea18ed919\" (UID: \"23b97dbb-8488-4c44-a810-882ea18ed919\") "
Oct 30 21:51:41.804644 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:41.804633    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/23b97dbb-8488-4c44-a810-882ea18ed919-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/23b97dbb-8488-4c44-a810-882ea18ed919-tmp-dir\") pod \"23b97dbb-8488-4c44-a810-882ea18ed919\" (UID: \"23b97dbb-8488-4c44-a810-882ea18ed919\") "
Oct 30 21:51:41.805417 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:41.805333    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23b97dbb-8488-4c44-a810-882ea18ed919-model-cache" pod "23b97dbb-8488-4c44-a810-882ea18ed919" (UID: "23b97dbb-8488-4c44-a810-882ea18ed919"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:41.805560 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:41.805534    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d0ea058-b6ca-4181-8823-2b719883e545-model-cache" pod "2d0ea058-b6ca-4181-8823-2b719883e545" (UID: "2d0ea058-b6ca-4181-8823-2b719883e545"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:41.805652 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:41.805599    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d0ea058-b6ca-4181-8823-2b719883e545-home" pod "2d0ea058-b6ca-4181-8823-2b719883e545" (UID: "2d0ea058-b6ca-4181-8823-2b719883e545"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:41.805867 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:41.805831    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23b97dbb-8488-4c44-a810-882ea18ed919-tmp-dir" pod "23b97dbb-8488-4c44-a810-882ea18ed919" (UID: "23b97dbb-8488-4c44-a810-882ea18ed919"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:41.806008 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:41.805983    5032 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d0ea058-b6ca-4181-8823-2b719883e545-model-cache\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:41.806189 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:41.806011    5032 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/23b97dbb-8488-4c44-a810-882ea18ed919-model-cache\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:41.806189 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:41.806026    5032 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d0ea058-b6ca-4181-8823-2b719883e545-home\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:41.806189 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:41.806039    5032 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/23b97dbb-8488-4c44-a810-882ea18ed919-tmp-dir\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:41.806189 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:41.806150    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23b97dbb-8488-4c44-a810-882ea18ed919-home" pod "23b97dbb-8488-4c44-a810-882ea18ed919" (UID: "23b97dbb-8488-4c44-a810-882ea18ed919"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:41.806480 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:41.806208    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d0ea058-b6ca-4181-8823-2b719883e545-tmp-dir" pod "2d0ea058-b6ca-4181-8823-2b719883e545" (UID: "2d0ea058-b6ca-4181-8823-2b719883e545"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:41.810289 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:41.810254    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23b97dbb-8488-4c44-a810-882ea18ed919-dshm" pod "23b97dbb-8488-4c44-a810-882ea18ed919" (UID: "23b97dbb-8488-4c44-a810-882ea18ed919"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:41.810528 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:41.810425    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d0ea058-b6ca-4181-8823-2b719883e545-tls-certs" pod "2d0ea058-b6ca-4181-8823-2b719883e545" (UID: "2d0ea058-b6ca-4181-8823-2b719883e545"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:51:41.810528 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:41.810497    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d0ea058-b6ca-4181-8823-2b719883e545-dshm" pod "2d0ea058-b6ca-4181-8823-2b719883e545" (UID: "2d0ea058-b6ca-4181-8823-2b719883e545"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:41.810997 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:41.810965    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d0ea058-b6ca-4181-8823-2b719883e545-kube-api-access-t9zz8" pod "2d0ea058-b6ca-4181-8823-2b719883e545" (UID: "2d0ea058-b6ca-4181-8823-2b719883e545"). InnerVolumeSpecName "kube-api-access-t9zz8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:51:41.812273 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:41.812250    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23b97dbb-8488-4c44-a810-882ea18ed919-kube-api-access-bzpfw" pod "23b97dbb-8488-4c44-a810-882ea18ed919" (UID: "23b97dbb-8488-4c44-a810-882ea18ed919"). InnerVolumeSpecName "kube-api-access-bzpfw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:51:41.812479 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:41.812458    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23b97dbb-8488-4c44-a810-882ea18ed919-tls-certs" pod "23b97dbb-8488-4c44-a810-882ea18ed919" (UID: "23b97dbb-8488-4c44-a810-882ea18ed919"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:51:41.907990 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:41.907950    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bzpfw\" (UniqueName: \"kubernetes.io/projected/23b97dbb-8488-4c44-a810-882ea18ed919-kube-api-access-bzpfw\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:41.907990 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:41.907990    5032 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d0ea058-b6ca-4181-8823-2b719883e545-tls-certs\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:41.907990 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:41.908000    5032 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/23b97dbb-8488-4c44-a810-882ea18ed919-dshm\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:41.908391 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:41.908009    5032 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/23b97dbb-8488-4c44-a810-882ea18ed919-tls-certs\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:41.908391 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:41.908018    5032 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d0ea058-b6ca-4181-8823-2b719883e545-tmp-dir\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:41.908391 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:41.908027    5032 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/23b97dbb-8488-4c44-a810-882ea18ed919-home\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:41.908391 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:41.908035    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t9zz8\" (UniqueName: \"kubernetes.io/projected/2d0ea058-b6ca-4181-8823-2b719883e545-kube-api-access-t9zz8\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:41.908391 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:41.908043    5032 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d0ea058-b6ca-4181-8823-2b719883e545-dshm\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:42.074747 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:42.074713    5032 patch_prober.go:29] interesting pod/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:51:42.074911 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:42.074788    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm" podUID="6b99e2de-e8a0-4566-b14c-dd627c74edf3" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:51:42.335067 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:42.334970    5032 generic.go:356] "Generic (PLEG): container finished" podID="2d0ea058-b6ca-4181-8823-2b719883e545" containerID="6797a8c03a4ba0206dd9a003cee3991819a6ab7cce24fc5acfb065cc7557ca8a" exitCode=0
Oct 30 21:51:42.335067 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:42.335024    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg"
Oct 30 21:51:42.335067 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:42.335045    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg" event={"ID":"2d0ea058-b6ca-4181-8823-2b719883e545","Type":"ContainerDied","Data":"6797a8c03a4ba0206dd9a003cee3991819a6ab7cce24fc5acfb065cc7557ca8a"}
Oct 30 21:51:42.335347 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:42.335098    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg" event={"ID":"2d0ea058-b6ca-4181-8823-2b719883e545","Type":"ContainerDied","Data":"87c54864389501b136ba8b2ea4d7da6a0880791e10ae9f83177efc1abfef2efa"}
Oct 30 21:51:42.335347 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:42.335120    5032 scope.go:122] "RemoveContainer" containerID="6797a8c03a4ba0206dd9a003cee3991819a6ab7cce24fc5acfb065cc7557ca8a"
Oct 30 21:51:42.336805 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:42.336783    5032 generic.go:356] "Generic (PLEG): container finished" podID="23b97dbb-8488-4c44-a810-882ea18ed919" containerID="55fec9098ae03c756d258df3c3d2d75a5db97bfdeaad74957831d75d1af4449b" exitCode=0
Oct 30 21:51:42.337151 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:42.336840    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc"
Oct 30 21:51:42.337151 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:42.336871    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc" event={"ID":"23b97dbb-8488-4c44-a810-882ea18ed919","Type":"ContainerDied","Data":"55fec9098ae03c756d258df3c3d2d75a5db97bfdeaad74957831d75d1af4449b"}
Oct 30 21:51:42.337151 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:42.336916    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc" event={"ID":"23b97dbb-8488-4c44-a810-882ea18ed919","Type":"ContainerDied","Data":"4a379b2784d148084fae9bc331b46d462af255ccb510895bf36543e1f844579f"}
Oct 30 21:51:42.345841 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:42.345827    5032 scope.go:122] "RemoveContainer" containerID="6797a8c03a4ba0206dd9a003cee3991819a6ab7cce24fc5acfb065cc7557ca8a"
Oct 30 21:51:42.346175 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:51:42.346157    5032 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6797a8c03a4ba0206dd9a003cee3991819a6ab7cce24fc5acfb065cc7557ca8a\": container with ID starting with 6797a8c03a4ba0206dd9a003cee3991819a6ab7cce24fc5acfb065cc7557ca8a not found: ID does not exist" containerID="6797a8c03a4ba0206dd9a003cee3991819a6ab7cce24fc5acfb065cc7557ca8a"
Oct 30 21:51:42.346216 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:42.346185    5032 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6797a8c03a4ba0206dd9a003cee3991819a6ab7cce24fc5acfb065cc7557ca8a"} err="failed to get container status \"6797a8c03a4ba0206dd9a003cee3991819a6ab7cce24fc5acfb065cc7557ca8a\": rpc error: code = NotFound desc = could not find container \"6797a8c03a4ba0206dd9a003cee3991819a6ab7cce24fc5acfb065cc7557ca8a\": container with ID starting with 6797a8c03a4ba0206dd9a003cee3991819a6ab7cce24fc5acfb065cc7557ca8a not found: ID does not exist"
Oct 30 21:51:42.346216 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:42.346201    5032 scope.go:122] "RemoveContainer" containerID="55fec9098ae03c756d258df3c3d2d75a5db97bfdeaad74957831d75d1af4449b"
Oct 30 21:51:42.355297 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:42.355277    5032 scope.go:122] "RemoveContainer" containerID="55fec9098ae03c756d258df3c3d2d75a5db97bfdeaad74957831d75d1af4449b"
Oct 30 21:51:42.355633 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:51:42.355575    5032 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55fec9098ae03c756d258df3c3d2d75a5db97bfdeaad74957831d75d1af4449b\": container with ID starting with 55fec9098ae03c756d258df3c3d2d75a5db97bfdeaad74957831d75d1af4449b not found: ID does not exist" containerID="55fec9098ae03c756d258df3c3d2d75a5db97bfdeaad74957831d75d1af4449b"
Oct 30 21:51:42.355633 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:42.355600    5032 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55fec9098ae03c756d258df3c3d2d75a5db97bfdeaad74957831d75d1af4449b"} err="failed to get container status \"55fec9098ae03c756d258df3c3d2d75a5db97bfdeaad74957831d75d1af4449b\": rpc error: code = NotFound desc = could not find container \"55fec9098ae03c756d258df3c3d2d75a5db97bfdeaad74957831d75d1af4449b\": container with ID starting with 55fec9098ae03c756d258df3c3d2d75a5db97bfdeaad74957831d75d1af4449b not found: ID does not exist"
Oct 30 21:51:42.360882 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:42.360859    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg"]
Oct 30 21:51:42.365986 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:42.365964    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-1d8034/distinct-w3-1d8034-kserve-7d4cd46dcb-c8drg"]
Oct 30 21:51:42.379166 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:42.379143    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc"]
Oct 30 21:51:42.387903 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:42.387878    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-1d8034/distinct-2-w3-1d8034-kserve-dd9c9d64c-ddglc"]
Oct 30 21:51:43.347037 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:43.347003    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="23b97dbb-8488-4c44-a810-882ea18ed919" path="/var/lib/kubelet/pods/23b97dbb-8488-4c44-a810-882ea18ed919/volumes"
Oct 30 21:51:43.347567 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:43.347546    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2d0ea058-b6ca-4181-8823-2b719883e545" path="/var/lib/kubelet/pods/2d0ea058-b6ca-4181-8823-2b719883e545/volumes"
Oct 30 21:51:44.881052 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:44.881013    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj"]
Oct 30 21:51:45.698731 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:45.698700    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf"
Oct 30 21:51:45.746867 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:45.746837    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm"
Oct 30 21:51:45.759416 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:45.759382    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c0ca92b2-a242-47f5-9cc6-ee04860b5692-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0ca92b2-a242-47f5-9cc6-ee04860b5692-workload-certs\") pod \"c0ca92b2-a242-47f5-9cc6-ee04860b5692\" (UID: \"c0ca92b2-a242-47f5-9cc6-ee04860b5692\") "
Oct 30 21:51:45.759589 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:45.759449    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c0ca92b2-a242-47f5-9cc6-ee04860b5692-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0ca92b2-a242-47f5-9cc6-ee04860b5692-istio-podinfo\") pod \"c0ca92b2-a242-47f5-9cc6-ee04860b5692\" (UID: \"c0ca92b2-a242-47f5-9cc6-ee04860b5692\") "
Oct 30 21:51:45.759589 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:45.759500    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c0ca92b2-a242-47f5-9cc6-ee04860b5692-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0ca92b2-a242-47f5-9cc6-ee04860b5692-istiod-ca-cert\") pod \"c0ca92b2-a242-47f5-9cc6-ee04860b5692\" (UID: \"c0ca92b2-a242-47f5-9cc6-ee04860b5692\") "
Oct 30 21:51:45.759589 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:45.759533    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c0ca92b2-a242-47f5-9cc6-ee04860b5692-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0ca92b2-a242-47f5-9cc6-ee04860b5692-istio-data\") pod \"c0ca92b2-a242-47f5-9cc6-ee04860b5692\" (UID: \"c0ca92b2-a242-47f5-9cc6-ee04860b5692\") "
Oct 30 21:51:45.759784 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:45.759633    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c0ca92b2-a242-47f5-9cc6-ee04860b5692-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0ca92b2-a242-47f5-9cc6-ee04860b5692-credential-socket\") pod \"c0ca92b2-a242-47f5-9cc6-ee04860b5692\" (UID: \"c0ca92b2-a242-47f5-9cc6-ee04860b5692\") "
Oct 30 21:51:45.759784 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:45.759716    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0ca92b2-a242-47f5-9cc6-ee04860b5692-workload-certs" pod "c0ca92b2-a242-47f5-9cc6-ee04860b5692" (UID: "c0ca92b2-a242-47f5-9cc6-ee04860b5692"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:45.759784 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:45.759744    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c0ca92b2-a242-47f5-9cc6-ee04860b5692-kube-api-access-6lw9n\" (UniqueName: \"kubernetes.io/projected/c0ca92b2-a242-47f5-9cc6-ee04860b5692-kube-api-access-6lw9n\") pod \"c0ca92b2-a242-47f5-9cc6-ee04860b5692\" (UID: \"c0ca92b2-a242-47f5-9cc6-ee04860b5692\") "
Oct 30 21:51:45.759784 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:45.759779    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c0ca92b2-a242-47f5-9cc6-ee04860b5692-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0ca92b2-a242-47f5-9cc6-ee04860b5692-workload-socket\") pod \"c0ca92b2-a242-47f5-9cc6-ee04860b5692\" (UID: \"c0ca92b2-a242-47f5-9cc6-ee04860b5692\") "
Oct 30 21:51:45.760001 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:45.759826    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c0ca92b2-a242-47f5-9cc6-ee04860b5692-istio-token\" (UniqueName: \"kubernetes.io/projected/c0ca92b2-a242-47f5-9cc6-ee04860b5692-istio-token\") pod \"c0ca92b2-a242-47f5-9cc6-ee04860b5692\" (UID: \"c0ca92b2-a242-47f5-9cc6-ee04860b5692\") "
Oct 30 21:51:45.760001 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:45.759932    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c0ca92b2-a242-47f5-9cc6-ee04860b5692-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0ca92b2-a242-47f5-9cc6-ee04860b5692-istio-envoy\") pod \"c0ca92b2-a242-47f5-9cc6-ee04860b5692\" (UID: \"c0ca92b2-a242-47f5-9cc6-ee04860b5692\") "
Oct 30 21:51:45.760134 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:45.760108    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0ca92b2-a242-47f5-9cc6-ee04860b5692-istiod-ca-cert" pod "c0ca92b2-a242-47f5-9cc6-ee04860b5692" (UID: "c0ca92b2-a242-47f5-9cc6-ee04860b5692"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:51:45.760583 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:45.760559    5032 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0ca92b2-a242-47f5-9cc6-ee04860b5692-workload-certs\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:45.760829 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:45.760742    5032 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0ca92b2-a242-47f5-9cc6-ee04860b5692-istiod-ca-cert\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:45.762426 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:45.762346    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0ca92b2-a242-47f5-9cc6-ee04860b5692-workload-socket" pod "c0ca92b2-a242-47f5-9cc6-ee04860b5692" (UID: "c0ca92b2-a242-47f5-9cc6-ee04860b5692"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:45.762616 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:45.762583    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0ca92b2-a242-47f5-9cc6-ee04860b5692-credential-socket" pod "c0ca92b2-a242-47f5-9cc6-ee04860b5692" (UID: "c0ca92b2-a242-47f5-9cc6-ee04860b5692"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:45.762866 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:45.762817    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0ca92b2-a242-47f5-9cc6-ee04860b5692-istio-data" pod "c0ca92b2-a242-47f5-9cc6-ee04860b5692" (UID: "c0ca92b2-a242-47f5-9cc6-ee04860b5692"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:45.764210 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:45.764182    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0ca92b2-a242-47f5-9cc6-ee04860b5692-istio-envoy" pod "c0ca92b2-a242-47f5-9cc6-ee04860b5692" (UID: "c0ca92b2-a242-47f5-9cc6-ee04860b5692"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:45.764836 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:45.764802    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0ca92b2-a242-47f5-9cc6-ee04860b5692-istio-token" pod "c0ca92b2-a242-47f5-9cc6-ee04860b5692" (UID: "c0ca92b2-a242-47f5-9cc6-ee04860b5692"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:51:45.780492 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:45.780458    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c0ca92b2-a242-47f5-9cc6-ee04860b5692-istio-podinfo" pod "c0ca92b2-a242-47f5-9cc6-ee04860b5692" (UID: "c0ca92b2-a242-47f5-9cc6-ee04860b5692"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:51:45.783204 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:45.783162    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0ca92b2-a242-47f5-9cc6-ee04860b5692-kube-api-access-6lw9n" pod "c0ca92b2-a242-47f5-9cc6-ee04860b5692" (UID: "c0ca92b2-a242-47f5-9cc6-ee04860b5692"). InnerVolumeSpecName "kube-api-access-6lw9n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:51:45.861755 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:45.861725    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6b99e2de-e8a0-4566-b14c-dd627c74edf3-kube-api-access-btml4\" (UniqueName: \"kubernetes.io/projected/6b99e2de-e8a0-4566-b14c-dd627c74edf3-kube-api-access-btml4\") pod \"6b99e2de-e8a0-4566-b14c-dd627c74edf3\" (UID: \"6b99e2de-e8a0-4566-b14c-dd627c74edf3\") "
Oct 30 21:51:45.862029 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:45.862004    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6b99e2de-e8a0-4566-b14c-dd627c74edf3-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b99e2de-e8a0-4566-b14c-dd627c74edf3-credential-socket\") pod \"6b99e2de-e8a0-4566-b14c-dd627c74edf3\" (UID: \"6b99e2de-e8a0-4566-b14c-dd627c74edf3\") "
Oct 30 21:51:45.862203 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:45.862188    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6b99e2de-e8a0-4566-b14c-dd627c74edf3-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b99e2de-e8a0-4566-b14c-dd627c74edf3-workload-socket\") pod \"6b99e2de-e8a0-4566-b14c-dd627c74edf3\" (UID: \"6b99e2de-e8a0-4566-b14c-dd627c74edf3\") "
Oct 30 21:51:45.862327 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:45.862310    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6b99e2de-e8a0-4566-b14c-dd627c74edf3-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b99e2de-e8a0-4566-b14c-dd627c74edf3-istio-podinfo\") pod \"6b99e2de-e8a0-4566-b14c-dd627c74edf3\" (UID: \"6b99e2de-e8a0-4566-b14c-dd627c74edf3\") "
Oct 30 21:51:45.862494 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:45.862477    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6b99e2de-e8a0-4566-b14c-dd627c74edf3-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b99e2de-e8a0-4566-b14c-dd627c74edf3-istio-envoy\") pod \"6b99e2de-e8a0-4566-b14c-dd627c74edf3\" (UID: \"6b99e2de-e8a0-4566-b14c-dd627c74edf3\") "
Oct 30 21:51:45.862632 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:45.862617    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6b99e2de-e8a0-4566-b14c-dd627c74edf3-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b99e2de-e8a0-4566-b14c-dd627c74edf3-istio-data\") pod \"6b99e2de-e8a0-4566-b14c-dd627c74edf3\" (UID: \"6b99e2de-e8a0-4566-b14c-dd627c74edf3\") "
Oct 30 21:51:45.862765 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:45.862747    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6b99e2de-e8a0-4566-b14c-dd627c74edf3-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b99e2de-e8a0-4566-b14c-dd627c74edf3-workload-certs\") pod \"6b99e2de-e8a0-4566-b14c-dd627c74edf3\" (UID: \"6b99e2de-e8a0-4566-b14c-dd627c74edf3\") "
Oct 30 21:51:45.862841 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:45.862789    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6b99e2de-e8a0-4566-b14c-dd627c74edf3-istio-token\" (UniqueName: \"kubernetes.io/projected/6b99e2de-e8a0-4566-b14c-dd627c74edf3-istio-token\") pod \"6b99e2de-e8a0-4566-b14c-dd627c74edf3\" (UID: \"6b99e2de-e8a0-4566-b14c-dd627c74edf3\") "
Oct 30 21:51:45.862960 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:45.862931    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b99e2de-e8a0-4566-b14c-dd627c74edf3-workload-socket" pod "6b99e2de-e8a0-4566-b14c-dd627c74edf3" (UID: "6b99e2de-e8a0-4566-b14c-dd627c74edf3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:45.863014 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:45.862998    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6b99e2de-e8a0-4566-b14c-dd627c74edf3-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b99e2de-e8a0-4566-b14c-dd627c74edf3-istiod-ca-cert\") pod \"6b99e2de-e8a0-4566-b14c-dd627c74edf3\" (UID: \"6b99e2de-e8a0-4566-b14c-dd627c74edf3\") "
Oct 30 21:51:45.863145 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:45.863126    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b99e2de-e8a0-4566-b14c-dd627c74edf3-credential-socket" pod "6b99e2de-e8a0-4566-b14c-dd627c74edf3" (UID: "6b99e2de-e8a0-4566-b14c-dd627c74edf3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:45.863330 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:45.863310    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b99e2de-e8a0-4566-b14c-dd627c74edf3-istio-data" pod "6b99e2de-e8a0-4566-b14c-dd627c74edf3" (UID: "6b99e2de-e8a0-4566-b14c-dd627c74edf3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:45.863578 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:45.863553    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b99e2de-e8a0-4566-b14c-dd627c74edf3-workload-certs" pod "6b99e2de-e8a0-4566-b14c-dd627c74edf3" (UID: "6b99e2de-e8a0-4566-b14c-dd627c74edf3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:45.863951 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:45.863623    5032 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0ca92b2-a242-47f5-9cc6-ee04860b5692-istio-podinfo\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:45.863951 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:45.863690    5032 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0ca92b2-a242-47f5-9cc6-ee04860b5692-istio-data\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:45.863951 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:45.863706    5032 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0ca92b2-a242-47f5-9cc6-ee04860b5692-credential-socket\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:45.863951 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:45.863721    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6lw9n\" (UniqueName: \"kubernetes.io/projected/c0ca92b2-a242-47f5-9cc6-ee04860b5692-kube-api-access-6lw9n\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:45.863951 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:45.863738    5032 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0ca92b2-a242-47f5-9cc6-ee04860b5692-workload-socket\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:45.863951 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:45.863752    5032 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b99e2de-e8a0-4566-b14c-dd627c74edf3-credential-socket\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:45.863951 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:45.863765    5032 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0ca92b2-a242-47f5-9cc6-ee04860b5692-istio-token\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:45.863951 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:45.863779    5032 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b99e2de-e8a0-4566-b14c-dd627c74edf3-workload-socket\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:45.863951 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:45.863793    5032 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b99e2de-e8a0-4566-b14c-dd627c74edf3-istio-data\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:45.863951 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:45.863810    5032 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0ca92b2-a242-47f5-9cc6-ee04860b5692-istio-envoy\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:45.864904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:45.864867    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b99e2de-e8a0-4566-b14c-dd627c74edf3-istiod-ca-cert" pod "6b99e2de-e8a0-4566-b14c-dd627c74edf3" (UID: "6b99e2de-e8a0-4566-b14c-dd627c74edf3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:51:45.866122 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:45.866048    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6b99e2de-e8a0-4566-b14c-dd627c74edf3-istio-podinfo" pod "6b99e2de-e8a0-4566-b14c-dd627c74edf3" (UID: "6b99e2de-e8a0-4566-b14c-dd627c74edf3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:51:45.866248 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:45.866221    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b99e2de-e8a0-4566-b14c-dd627c74edf3-istio-token" pod "6b99e2de-e8a0-4566-b14c-dd627c74edf3" (UID: "6b99e2de-e8a0-4566-b14c-dd627c74edf3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:51:45.866571 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:45.866547    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b99e2de-e8a0-4566-b14c-dd627c74edf3-kube-api-access-btml4" pod "6b99e2de-e8a0-4566-b14c-dd627c74edf3" (UID: "6b99e2de-e8a0-4566-b14c-dd627c74edf3"). InnerVolumeSpecName "kube-api-access-btml4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:51:45.866662 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:45.866606    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b99e2de-e8a0-4566-b14c-dd627c74edf3-istio-envoy" pod "6b99e2de-e8a0-4566-b14c-dd627c74edf3" (UID: "6b99e2de-e8a0-4566-b14c-dd627c74edf3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:45.965299 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:45.965262    5032 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b99e2de-e8a0-4566-b14c-dd627c74edf3-workload-certs\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:45.965299 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:45.965300    5032 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b99e2de-e8a0-4566-b14c-dd627c74edf3-istio-token\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:45.965741 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:45.965312    5032 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b99e2de-e8a0-4566-b14c-dd627c74edf3-istiod-ca-cert\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:45.965741 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:45.965324    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-btml4\" (UniqueName: \"kubernetes.io/projected/6b99e2de-e8a0-4566-b14c-dd627c74edf3-kube-api-access-btml4\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:45.965741 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:45.965337    5032 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b99e2de-e8a0-4566-b14c-dd627c74edf3-istio-podinfo\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:45.965741 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:45.965350    5032 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b99e2de-e8a0-4566-b14c-dd627c74edf3-istio-envoy\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:46.366961 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:46.366874    5032 generic.go:356] "Generic (PLEG): container finished" podID="6b99e2de-e8a0-4566-b14c-dd627c74edf3" containerID="215dd4edb5877e9e26232807f565e61d0266b5ee2227ae963f89fae8a84effb6" exitCode=0
Oct 30 21:51:46.366961 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:46.366923    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm"
Oct 30 21:51:46.367178 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:46.366960    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm" event={"ID":"6b99e2de-e8a0-4566-b14c-dd627c74edf3","Type":"ContainerDied","Data":"215dd4edb5877e9e26232807f565e61d0266b5ee2227ae963f89fae8a84effb6"}
Oct 30 21:51:46.367178 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:46.367029    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm" event={"ID":"6b99e2de-e8a0-4566-b14c-dd627c74edf3","Type":"ContainerDied","Data":"5b29de64163757ed971d4b730c3d7a1314a1448f3d6fa8e96171954836cbe22c"}
Oct 30 21:51:46.367178 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:46.367063    5032 scope.go:122] "RemoveContainer" containerID="215dd4edb5877e9e26232807f565e61d0266b5ee2227ae963f89fae8a84effb6"
Oct 30 21:51:46.368481 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:46.368453    5032 generic.go:356] "Generic (PLEG): container finished" podID="c0ca92b2-a242-47f5-9cc6-ee04860b5692" containerID="b2b97753a9fba276652fbba6d1514744ba74cfd3f791c70e1e8b7d7ec5c60cee" exitCode=0
Oct 30 21:51:46.368609 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:46.368499    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf"
Oct 30 21:51:46.368609 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:46.368543    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf" event={"ID":"c0ca92b2-a242-47f5-9cc6-ee04860b5692","Type":"ContainerDied","Data":"b2b97753a9fba276652fbba6d1514744ba74cfd3f791c70e1e8b7d7ec5c60cee"}
Oct 30 21:51:46.368609 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:46.368588    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf" event={"ID":"c0ca92b2-a242-47f5-9cc6-ee04860b5692","Type":"ContainerDied","Data":"acef7605eda87b82a8fe8d41e5fd6806a5a7fa187c0d345a515247215f483e8e"}
Oct 30 21:51:46.378945 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:46.378921    5032 scope.go:122] "RemoveContainer" containerID="215dd4edb5877e9e26232807f565e61d0266b5ee2227ae963f89fae8a84effb6"
Oct 30 21:51:46.379271 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:51:46.379252    5032 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"215dd4edb5877e9e26232807f565e61d0266b5ee2227ae963f89fae8a84effb6\": container with ID starting with 215dd4edb5877e9e26232807f565e61d0266b5ee2227ae963f89fae8a84effb6 not found: ID does not exist" containerID="215dd4edb5877e9e26232807f565e61d0266b5ee2227ae963f89fae8a84effb6"
Oct 30 21:51:46.379442 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:46.379280    5032 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"215dd4edb5877e9e26232807f565e61d0266b5ee2227ae963f89fae8a84effb6"} err="failed to get container status \"215dd4edb5877e9e26232807f565e61d0266b5ee2227ae963f89fae8a84effb6\": rpc error: code = NotFound desc = could not find container \"215dd4edb5877e9e26232807f565e61d0266b5ee2227ae963f89fae8a84effb6\": container with ID starting with 215dd4edb5877e9e26232807f565e61d0266b5ee2227ae963f89fae8a84effb6 not found: ID does not exist"
Oct 30 21:51:46.379442 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:46.379302    5032 scope.go:122] "RemoveContainer" containerID="b2b97753a9fba276652fbba6d1514744ba74cfd3f791c70e1e8b7d7ec5c60cee"
Oct 30 21:51:46.394841 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:46.394813    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm"]
Oct 30 21:51:46.400352 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:46.400324    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-m72vm"]
Oct 30 21:51:46.401052 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:46.401026    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-0fa864-76b48d9455-wxt6j" podUID="18d702a7-00ad-4a6b-8de5-806e5900cd7d" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:51:46.401157 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:46.401025    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w4-0fa864-5d85948857-tw7k6" podUID="76de8529-43f7-4285-864b-f8242fca2cae" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:51:46.401157 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:46.401141    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-1d8034-5b96976fc5-hsr78" podUID="27aef427-6b1e-4845-b8d9-a83cba81b0e0" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:51:46.404328 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:46.404308    5032 scope.go:122] "RemoveContainer" containerID="b2b97753a9fba276652fbba6d1514744ba74cfd3f791c70e1e8b7d7ec5c60cee"
Oct 30 21:51:46.404647 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:51:46.404626    5032 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b2b97753a9fba276652fbba6d1514744ba74cfd3f791c70e1e8b7d7ec5c60cee\": container with ID starting with b2b97753a9fba276652fbba6d1514744ba74cfd3f791c70e1e8b7d7ec5c60cee not found: ID does not exist" containerID="b2b97753a9fba276652fbba6d1514744ba74cfd3f791c70e1e8b7d7ec5c60cee"
Oct 30 21:51:46.404716 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:46.404657    5032 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2b97753a9fba276652fbba6d1514744ba74cfd3f791c70e1e8b7d7ec5c60cee"} err="failed to get container status \"b2b97753a9fba276652fbba6d1514744ba74cfd3f791c70e1e8b7d7ec5c60cee\": rpc error: code = NotFound desc = could not find container \"b2b97753a9fba276652fbba6d1514744ba74cfd3f791c70e1e8b7d7ec5c60cee\": container with ID starting with b2b97753a9fba276652fbba6d1514744ba74cfd3f791c70e1e8b7d7ec5c60cee not found: ID does not exist"
Oct 30 21:51:46.419952 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:46.419919    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf"]
Oct 30 21:51:46.422709 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:46.422689    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-db0b54-openshift-default-79c79b9575-xcggf"]
Oct 30 21:51:46.956037 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:46.956004    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 21:51:47.355930 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:47.355850    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6b99e2de-e8a0-4566-b14c-dd627c74edf3" path="/var/lib/kubelet/pods/6b99e2de-e8a0-4566-b14c-dd627c74edf3/volumes"
Oct 30 21:51:47.356617 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:47.356594    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c0ca92b2-a242-47f5-9cc6-ee04860b5692" path="/var/lib/kubelet/pods/c0ca92b2-a242-47f5-9cc6-ee04860b5692/volumes"
Oct 30 21:51:47.412477 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:47.412438    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w3-1d8034-6bbd97dcf7-l2gd8" podUID="e25d6f68-6e47-449f-b02f-03fff2b2bb30" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:51:48.217100 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:48.217058    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b"]
Oct 30 21:51:48.217339 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:48.217313    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b" podUID="3aee81c5-228b-4f21-b7d9-a0bd0822b4df" containerName="istio-proxy" containerID="cri-o://31706ca5ffc512e69734a1d499097c7cd0a124432df0d948d2d5ef41d7a406ad" gracePeriod=30
Oct 30 21:51:48.231928 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:48.231895    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8"]
Oct 30 21:51:48.232215 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:48.232108    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8" podUID="e7679391-8575-43d7-b7f1-5b1ab0bcee5f" containerName="istio-proxy" containerID="cri-o://b026d760849187704d451a5830315d5a4a7556c362335e17873b515117d83d93" gracePeriod=30
Oct 30 21:51:48.367120 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:51:48.367075    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 21:51:49.931919 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:49.931887    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-1d8034-d5d875bdd-ncxns"]
Oct 30 21:51:49.932407 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:49.932078    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-1d8034-d5d875bdd-ncxns" podUID="5d5e4361-e970-43dd-b1e7-66ca197b10ce" containerName="maas-api" containerID="cri-o://ad554269e6bfe36fd0e874a69813e144c8123ac4534759e48a2bcd6f404c077c" gracePeriod=30
Oct 30 21:51:50.306453 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:50.306425    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-1d8034-d5d875bdd-ncxns"
Oct 30 21:51:50.314824 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:50.314788    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-caa3dc-85fc67b867-gsjl2"]
Oct 30 21:51:50.317964 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:50.317933    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-caa3dc-76db8c686c-pxdzh"]
Oct 30 21:51:50.405136 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:50.405094    5032 generic.go:356] "Generic (PLEG): container finished" podID="5d5e4361-e970-43dd-b1e7-66ca197b10ce" containerID="ad554269e6bfe36fd0e874a69813e144c8123ac4534759e48a2bcd6f404c077c" exitCode=0
Oct 30 21:51:50.405281 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:50.405145    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-1d8034-d5d875bdd-ncxns"
Oct 30 21:51:50.405281 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:50.405141    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-1d8034-d5d875bdd-ncxns" event={"ID":"5d5e4361-e970-43dd-b1e7-66ca197b10ce","Type":"ContainerDied","Data":"ad554269e6bfe36fd0e874a69813e144c8123ac4534759e48a2bcd6f404c077c"}
Oct 30 21:51:50.405281 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:50.405266    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-1d8034-d5d875bdd-ncxns" event={"ID":"5d5e4361-e970-43dd-b1e7-66ca197b10ce","Type":"ContainerDied","Data":"6b32903f6d61ef1104600062adf3f8a337f17ca52d705df8aafe4c2e2f339dfa"}
Oct 30 21:51:50.405437 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:50.405294    5032 scope.go:122] "RemoveContainer" containerID="ad554269e6bfe36fd0e874a69813e144c8123ac4534759e48a2bcd6f404c077c"
Oct 30 21:51:50.416441 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:50.416419    5032 scope.go:122] "RemoveContainer" containerID="ad554269e6bfe36fd0e874a69813e144c8123ac4534759e48a2bcd6f404c077c"
Oct 30 21:51:50.416812 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:51:50.416785    5032 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad554269e6bfe36fd0e874a69813e144c8123ac4534759e48a2bcd6f404c077c\": container with ID starting with ad554269e6bfe36fd0e874a69813e144c8123ac4534759e48a2bcd6f404c077c not found: ID does not exist" containerID="ad554269e6bfe36fd0e874a69813e144c8123ac4534759e48a2bcd6f404c077c"
Oct 30 21:51:50.416885 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:50.416823    5032 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad554269e6bfe36fd0e874a69813e144c8123ac4534759e48a2bcd6f404c077c"} err="failed to get container status \"ad554269e6bfe36fd0e874a69813e144c8123ac4534759e48a2bcd6f404c077c\": rpc error: code = NotFound desc = could not find container \"ad554269e6bfe36fd0e874a69813e144c8123ac4534759e48a2bcd6f404c077c\": container with ID starting with ad554269e6bfe36fd0e874a69813e144c8123ac4534759e48a2bcd6f404c077c not found: ID does not exist"
Oct 30 21:51:50.463621 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:50.463589    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5d5e4361-e970-43dd-b1e7-66ca197b10ce-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d5e4361-e970-43dd-b1e7-66ca197b10ce-maas-api-tls\") pod \"5d5e4361-e970-43dd-b1e7-66ca197b10ce\" (UID: \"5d5e4361-e970-43dd-b1e7-66ca197b10ce\") "
Oct 30 21:51:50.463773 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:50.463635    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5d5e4361-e970-43dd-b1e7-66ca197b10ce-kube-api-access-m664x\" (UniqueName: \"kubernetes.io/projected/5d5e4361-e970-43dd-b1e7-66ca197b10ce-kube-api-access-m664x\") pod \"5d5e4361-e970-43dd-b1e7-66ca197b10ce\" (UID: \"5d5e4361-e970-43dd-b1e7-66ca197b10ce\") "
Oct 30 21:51:50.463833 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:50.463801    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5d5e4361-e970-43dd-b1e7-66ca197b10ce-metrics-certs\" (UniqueName: \"kubernetes.io/secret/5d5e4361-e970-43dd-b1e7-66ca197b10ce-metrics-certs\") pod \"5d5e4361-e970-43dd-b1e7-66ca197b10ce\" (UID: \"5d5e4361-e970-43dd-b1e7-66ca197b10ce\") "
Oct 30 21:51:50.463888 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:50.463870    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5d5e4361-e970-43dd-b1e7-66ca197b10ce-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d5e4361-e970-43dd-b1e7-66ca197b10ce-openshift-service-ca\") pod \"5d5e4361-e970-43dd-b1e7-66ca197b10ce\" (UID: \"5d5e4361-e970-43dd-b1e7-66ca197b10ce\") "
Oct 30 21:51:50.464318 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:50.464285    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d5e4361-e970-43dd-b1e7-66ca197b10ce-openshift-service-ca" pod "5d5e4361-e970-43dd-b1e7-66ca197b10ce" (UID: "5d5e4361-e970-43dd-b1e7-66ca197b10ce"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:51:50.464545 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:50.464477    5032 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d5e4361-e970-43dd-b1e7-66ca197b10ce-openshift-service-ca\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:50.466132 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:50.466108    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d5e4361-e970-43dd-b1e7-66ca197b10ce-metrics-certs" pod "5d5e4361-e970-43dd-b1e7-66ca197b10ce" (UID: "5d5e4361-e970-43dd-b1e7-66ca197b10ce"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:51:50.466238 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:50.466194    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d5e4361-e970-43dd-b1e7-66ca197b10ce-maas-api-tls" pod "5d5e4361-e970-43dd-b1e7-66ca197b10ce" (UID: "5d5e4361-e970-43dd-b1e7-66ca197b10ce"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:51:50.466238 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:50.466209    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d5e4361-e970-43dd-b1e7-66ca197b10ce-kube-api-access-m664x" pod "5d5e4361-e970-43dd-b1e7-66ca197b10ce" (UID: "5d5e4361-e970-43dd-b1e7-66ca197b10ce"). InnerVolumeSpecName "kube-api-access-m664x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:51:50.566135 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:50.566058    5032 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5d5e4361-e970-43dd-b1e7-66ca197b10ce-metrics-certs\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:50.566135 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:50.566089    5032 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d5e4361-e970-43dd-b1e7-66ca197b10ce-maas-api-tls\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:50.566135 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:50.566099    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m664x\" (UniqueName: \"kubernetes.io/projected/5d5e4361-e970-43dd-b1e7-66ca197b10ce-kube-api-access-m664x\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:50.786089 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:50.786058    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-1d8034-d5d875bdd-ncxns"]
Oct 30 21:51:50.817175 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:50.817102    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-1d8034-d5d875bdd-ncxns"]
Oct 30 21:51:50.985709 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:50.985671    5032 patch_prober.go:29] interesting pod/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:51:50.986095 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:50.985732    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8" podUID="e7679391-8575-43d7-b7f1-5b1ab0bcee5f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:51:51.348521 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:51.348485    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5d5e4361-e970-43dd-b1e7-66ca197b10ce" path="/var/lib/kubelet/pods/5d5e4361-e970-43dd-b1e7-66ca197b10ce/volumes"
Oct 30 21:51:51.867434 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:51.867406    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7"]
Oct 30 21:51:51.867660 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:51.867632    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7" podUID="0d5b3ad5-309e-4c34-89a3-034e4966e652" containerName="istio-proxy" containerID="cri-o://105e63cdb204bccc898f9b89b178f9a801281db6bf9fbbad88078ba3a60be6cb" gracePeriod=30
Oct 30 21:51:51.874586 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:51.874549    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76"]
Oct 30 21:51:51.874821 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:51.874789    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76" podUID="cd4228de-c71f-4f88-a54e-9ebda8b91122" containerName="istio-proxy" containerID="cri-o://4637b417d0d41dd176aed51d61219db6bdd0debc511a69a2e445a093cd626e4d" gracePeriod=30
Oct 30 21:51:51.876997 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:51.876913    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-0fa864-5d85948857-tw7k6"
Oct 30 21:51:51.920425 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:51.920353    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-0fa864-76b48d9455-wxt6j"
Oct 30 21:51:51.992135 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:51.992098    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/76de8529-43f7-4285-864b-f8242fca2cae-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76de8529-43f7-4285-864b-f8242fca2cae-plugins-config-volume\") pod \"76de8529-43f7-4285-864b-f8242fca2cae\" (UID: \"76de8529-43f7-4285-864b-f8242fca2cae\") "
Oct 30 21:51:51.992723 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:51.992244    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/76de8529-43f7-4285-864b-f8242fca2cae-kube-api-access-8fc4b\" (UniqueName: \"kubernetes.io/projected/76de8529-43f7-4285-864b-f8242fca2cae-kube-api-access-8fc4b\") pod \"76de8529-43f7-4285-864b-f8242fca2cae\" (UID: \"76de8529-43f7-4285-864b-f8242fca2cae\") "
Oct 30 21:51:51.994080 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:51.994052    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76de8529-43f7-4285-864b-f8242fca2cae-plugins-config-volume" pod "76de8529-43f7-4285-864b-f8242fca2cae" (UID: "76de8529-43f7-4285-864b-f8242fca2cae"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:51:52.013966 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:52.007418    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76de8529-43f7-4285-864b-f8242fca2cae-kube-api-access-8fc4b" pod "76de8529-43f7-4285-864b-f8242fca2cae" (UID: "76de8529-43f7-4285-864b-f8242fca2cae"). InnerVolumeSpecName "kube-api-access-8fc4b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:51:52.094953 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:52.094922    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/18d702a7-00ad-4a6b-8de5-806e5900cd7d-kube-api-access-bzrkf\" (UniqueName: \"kubernetes.io/projected/18d702a7-00ad-4a6b-8de5-806e5900cd7d-kube-api-access-bzrkf\") pod \"18d702a7-00ad-4a6b-8de5-806e5900cd7d\" (UID: \"18d702a7-00ad-4a6b-8de5-806e5900cd7d\") "
Oct 30 21:51:52.095113 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:52.095065    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/18d702a7-00ad-4a6b-8de5-806e5900cd7d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18d702a7-00ad-4a6b-8de5-806e5900cd7d-plugins-config-volume\") pod \"18d702a7-00ad-4a6b-8de5-806e5900cd7d\" (UID: \"18d702a7-00ad-4a6b-8de5-806e5900cd7d\") "
Oct 30 21:51:52.095500 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:52.095472    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18d702a7-00ad-4a6b-8de5-806e5900cd7d-plugins-config-volume" pod "18d702a7-00ad-4a6b-8de5-806e5900cd7d" (UID: "18d702a7-00ad-4a6b-8de5-806e5900cd7d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:51:52.095500 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:52.095491    5032 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76de8529-43f7-4285-864b-f8242fca2cae-plugins-config-volume\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:52.095663 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:52.095507    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8fc4b\" (UniqueName: \"kubernetes.io/projected/76de8529-43f7-4285-864b-f8242fca2cae-kube-api-access-8fc4b\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:52.097172 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:52.097149    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18d702a7-00ad-4a6b-8de5-806e5900cd7d-kube-api-access-bzrkf" pod "18d702a7-00ad-4a6b-8de5-806e5900cd7d" (UID: "18d702a7-00ad-4a6b-8de5-806e5900cd7d"). InnerVolumeSpecName "kube-api-access-bzrkf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:51:52.196502 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:52.196418    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bzrkf\" (UniqueName: \"kubernetes.io/projected/18d702a7-00ad-4a6b-8de5-806e5900cd7d-kube-api-access-bzrkf\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:52.196502 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:52.196449    5032 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18d702a7-00ad-4a6b-8de5-806e5900cd7d-plugins-config-volume\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:52.422980 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:52.422948    5032 generic.go:356] "Generic (PLEG): container finished" podID="18d702a7-00ad-4a6b-8de5-806e5900cd7d" containerID="0d899832b76b126f7ee364e0454d922673062cbfb2def07182a739f32bf85d4f" exitCode=0
Oct 30 21:51:52.423169 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:52.422991    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-0fa864-76b48d9455-wxt6j"
Oct 30 21:51:52.423169 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:52.423040    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-0fa864-76b48d9455-wxt6j" event={"ID":"18d702a7-00ad-4a6b-8de5-806e5900cd7d","Type":"ContainerDied","Data":"0d899832b76b126f7ee364e0454d922673062cbfb2def07182a739f32bf85d4f"}
Oct 30 21:51:52.423169 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:52.423099    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-0fa864-76b48d9455-wxt6j" event={"ID":"18d702a7-00ad-4a6b-8de5-806e5900cd7d","Type":"ContainerDied","Data":"49245afbb744dc5657c452638f29a817201827be5971d66b781d804f79214375"}
Oct 30 21:51:52.423169 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:52.423127    5032 scope.go:122] "RemoveContainer" containerID="0d899832b76b126f7ee364e0454d922673062cbfb2def07182a739f32bf85d4f"
Oct 30 21:51:52.424936 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:52.424910    5032 generic.go:356] "Generic (PLEG): container finished" podID="76de8529-43f7-4285-864b-f8242fca2cae" containerID="8a53fc244ecaf6746c1a1d24f9cec8a7e839613c0bb15552cb0610bb93f1b446" exitCode=0
Oct 30 21:51:52.425077 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:52.424944    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-0fa864-5d85948857-tw7k6" event={"ID":"76de8529-43f7-4285-864b-f8242fca2cae","Type":"ContainerDied","Data":"8a53fc244ecaf6746c1a1d24f9cec8a7e839613c0bb15552cb0610bb93f1b446"}
Oct 30 21:51:52.425077 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:52.425000    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-0fa864-5d85948857-tw7k6" event={"ID":"76de8529-43f7-4285-864b-f8242fca2cae","Type":"ContainerDied","Data":"18ca07d04bd0711cb0eb3e9980c52a59a34f0bcadec7249c07646d4155de1916"}
Oct 30 21:51:52.425077 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:52.424955    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-0fa864-5d85948857-tw7k6"
Oct 30 21:51:52.436414 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:52.436395    5032 scope.go:122] "RemoveContainer" containerID="0d899832b76b126f7ee364e0454d922673062cbfb2def07182a739f32bf85d4f"
Oct 30 21:51:52.437985 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:51:52.437952    5032 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d899832b76b126f7ee364e0454d922673062cbfb2def07182a739f32bf85d4f\": container with ID starting with 0d899832b76b126f7ee364e0454d922673062cbfb2def07182a739f32bf85d4f not found: ID does not exist" containerID="0d899832b76b126f7ee364e0454d922673062cbfb2def07182a739f32bf85d4f"
Oct 30 21:51:52.438089 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:52.437999    5032 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d899832b76b126f7ee364e0454d922673062cbfb2def07182a739f32bf85d4f"} err="failed to get container status \"0d899832b76b126f7ee364e0454d922673062cbfb2def07182a739f32bf85d4f\": rpc error: code = NotFound desc = could not find container \"0d899832b76b126f7ee364e0454d922673062cbfb2def07182a739f32bf85d4f\": container with ID starting with 0d899832b76b126f7ee364e0454d922673062cbfb2def07182a739f32bf85d4f not found: ID does not exist"
Oct 30 21:51:52.438089 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:52.438023    5032 scope.go:122] "RemoveContainer" containerID="8a53fc244ecaf6746c1a1d24f9cec8a7e839613c0bb15552cb0610bb93f1b446"
Oct 30 21:51:52.449004 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:52.448978    5032 scope.go:122] "RemoveContainer" containerID="8a53fc244ecaf6746c1a1d24f9cec8a7e839613c0bb15552cb0610bb93f1b446"
Oct 30 21:51:52.449349 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:51:52.449331    5032 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a53fc244ecaf6746c1a1d24f9cec8a7e839613c0bb15552cb0610bb93f1b446\": container with ID starting with 8a53fc244ecaf6746c1a1d24f9cec8a7e839613c0bb15552cb0610bb93f1b446 not found: ID does not exist" containerID="8a53fc244ecaf6746c1a1d24f9cec8a7e839613c0bb15552cb0610bb93f1b446"
Oct 30 21:51:52.450197 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:52.450166    5032 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a53fc244ecaf6746c1a1d24f9cec8a7e839613c0bb15552cb0610bb93f1b446"} err="failed to get container status \"8a53fc244ecaf6746c1a1d24f9cec8a7e839613c0bb15552cb0610bb93f1b446\": rpc error: code = NotFound desc = could not find container \"8a53fc244ecaf6746c1a1d24f9cec8a7e839613c0bb15552cb0610bb93f1b446\": container with ID starting with 8a53fc244ecaf6746c1a1d24f9cec8a7e839613c0bb15552cb0610bb93f1b446 not found: ID does not exist"
Oct 30 21:51:52.451573 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:52.451551    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-0fa864-76b48d9455-wxt6j"]
Oct 30 21:51:52.457203 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:52.457179    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-0fa864-76b48d9455-wxt6j"]
Oct 30 21:51:52.467661 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:52.467640    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-0fa864-5d85948857-tw7k6"]
Oct 30 21:51:52.475468 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:52.475449    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-0fa864-5d85948857-tw7k6"]
Oct 30 21:51:53.348466 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:53.348429    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="18d702a7-00ad-4a6b-8de5-806e5900cd7d" path="/var/lib/kubelet/pods/18d702a7-00ad-4a6b-8de5-806e5900cd7d/volumes"
Oct 30 21:51:53.348886 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:53.348872    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="76de8529-43f7-4285-864b-f8242fca2cae" path="/var/lib/kubelet/pods/76de8529-43f7-4285-864b-f8242fca2cae/volumes"
Oct 30 21:51:53.435287 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:53.435256    5032 generic.go:356] "Generic (PLEG): container finished" podID="3aee81c5-228b-4f21-b7d9-a0bd0822b4df" containerID="31706ca5ffc512e69734a1d499097c7cd0a124432df0d948d2d5ef41d7a406ad" exitCode=0
Oct 30 21:51:53.435486 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:53.435333    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b" event={"ID":"3aee81c5-228b-4f21-b7d9-a0bd0822b4df","Type":"ContainerDied","Data":"31706ca5ffc512e69734a1d499097c7cd0a124432df0d948d2d5ef41d7a406ad"}
Oct 30 21:51:53.437259 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:53.437228    5032 generic.go:356] "Generic (PLEG): container finished" podID="e7679391-8575-43d7-b7f1-5b1ab0bcee5f" containerID="b026d760849187704d451a5830315d5a4a7556c362335e17873b515117d83d93" exitCode=0
Oct 30 21:51:53.437432 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:53.437312    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8" event={"ID":"e7679391-8575-43d7-b7f1-5b1ab0bcee5f","Type":"ContainerDied","Data":"b026d760849187704d451a5830315d5a4a7556c362335e17873b515117d83d93"}
Oct 30 21:51:53.511195 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:53.511123    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b"
Oct 30 21:51:53.570450 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:53.570419    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8"
Oct 30 21:51:53.610920 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:53.610890    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3aee81c5-228b-4f21-b7d9-a0bd0822b4df-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3aee81c5-228b-4f21-b7d9-a0bd0822b4df-workload-certs\") pod \"3aee81c5-228b-4f21-b7d9-a0bd0822b4df\" (UID: \"3aee81c5-228b-4f21-b7d9-a0bd0822b4df\") "
Oct 30 21:51:53.611104 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:53.610942    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3aee81c5-228b-4f21-b7d9-a0bd0822b4df-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3aee81c5-228b-4f21-b7d9-a0bd0822b4df-credential-socket\") pod \"3aee81c5-228b-4f21-b7d9-a0bd0822b4df\" (UID: \"3aee81c5-228b-4f21-b7d9-a0bd0822b4df\") "
Oct 30 21:51:53.611104 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:53.610982    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3aee81c5-228b-4f21-b7d9-a0bd0822b4df-istio-token\" (UniqueName: \"kubernetes.io/projected/3aee81c5-228b-4f21-b7d9-a0bd0822b4df-istio-token\") pod \"3aee81c5-228b-4f21-b7d9-a0bd0822b4df\" (UID: \"3aee81c5-228b-4f21-b7d9-a0bd0822b4df\") "
Oct 30 21:51:53.611104 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:53.611011    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3aee81c5-228b-4f21-b7d9-a0bd0822b4df-kube-api-access-5z877\" (UniqueName: \"kubernetes.io/projected/3aee81c5-228b-4f21-b7d9-a0bd0822b4df-kube-api-access-5z877\") pod \"3aee81c5-228b-4f21-b7d9-a0bd0822b4df\" (UID: \"3aee81c5-228b-4f21-b7d9-a0bd0822b4df\") "
Oct 30 21:51:53.611104 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:53.611035    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e7679391-8575-43d7-b7f1-5b1ab0bcee5f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e7679391-8575-43d7-b7f1-5b1ab0bcee5f-istiod-ca-cert\") pod \"e7679391-8575-43d7-b7f1-5b1ab0bcee5f\" (UID: \"e7679391-8575-43d7-b7f1-5b1ab0bcee5f\") "
Oct 30 21:51:53.611104 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:53.611068    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3aee81c5-228b-4f21-b7d9-a0bd0822b4df-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3aee81c5-228b-4f21-b7d9-a0bd0822b4df-istio-podinfo\") pod \"3aee81c5-228b-4f21-b7d9-a0bd0822b4df\" (UID: \"3aee81c5-228b-4f21-b7d9-a0bd0822b4df\") "
Oct 30 21:51:53.611385 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:53.611112    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e7679391-8575-43d7-b7f1-5b1ab0bcee5f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e7679391-8575-43d7-b7f1-5b1ab0bcee5f-istio-podinfo\") pod \"e7679391-8575-43d7-b7f1-5b1ab0bcee5f\" (UID: \"e7679391-8575-43d7-b7f1-5b1ab0bcee5f\") "
Oct 30 21:51:53.611385 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:53.611173    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e7679391-8575-43d7-b7f1-5b1ab0bcee5f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7679391-8575-43d7-b7f1-5b1ab0bcee5f-workload-certs\") pod \"e7679391-8575-43d7-b7f1-5b1ab0bcee5f\" (UID: \"e7679391-8575-43d7-b7f1-5b1ab0bcee5f\") "
Oct 30 21:51:53.611385 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:53.611227    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3aee81c5-228b-4f21-b7d9-a0bd0822b4df-workload-certs" pod "3aee81c5-228b-4f21-b7d9-a0bd0822b4df" (UID: "3aee81c5-228b-4f21-b7d9-a0bd0822b4df"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:53.611385 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:53.611239    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3aee81c5-228b-4f21-b7d9-a0bd0822b4df-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3aee81c5-228b-4f21-b7d9-a0bd0822b4df-istio-data\") pod \"3aee81c5-228b-4f21-b7d9-a0bd0822b4df\" (UID: \"3aee81c5-228b-4f21-b7d9-a0bd0822b4df\") "
Oct 30 21:51:53.611385 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:53.611254    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3aee81c5-228b-4f21-b7d9-a0bd0822b4df-credential-socket" pod "3aee81c5-228b-4f21-b7d9-a0bd0822b4df" (UID: "3aee81c5-228b-4f21-b7d9-a0bd0822b4df"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:53.611385 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:53.611263    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e7679391-8575-43d7-b7f1-5b1ab0bcee5f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7679391-8575-43d7-b7f1-5b1ab0bcee5f-workload-socket\") pod \"e7679391-8575-43d7-b7f1-5b1ab0bcee5f\" (UID: \"e7679391-8575-43d7-b7f1-5b1ab0bcee5f\") "
Oct 30 21:51:53.611385 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:53.611371    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e7679391-8575-43d7-b7f1-5b1ab0bcee5f-kube-api-access-2hbd2\" (UniqueName: \"kubernetes.io/projected/e7679391-8575-43d7-b7f1-5b1ab0bcee5f-kube-api-access-2hbd2\") pod \"e7679391-8575-43d7-b7f1-5b1ab0bcee5f\" (UID: \"e7679391-8575-43d7-b7f1-5b1ab0bcee5f\") "
Oct 30 21:51:53.611718 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:53.611406    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e7679391-8575-43d7-b7f1-5b1ab0bcee5f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e7679391-8575-43d7-b7f1-5b1ab0bcee5f-istio-data\") pod \"e7679391-8575-43d7-b7f1-5b1ab0bcee5f\" (UID: \"e7679391-8575-43d7-b7f1-5b1ab0bcee5f\") "
Oct 30 21:51:53.611718 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:53.611453    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3aee81c5-228b-4f21-b7d9-a0bd0822b4df-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3aee81c5-228b-4f21-b7d9-a0bd0822b4df-istiod-ca-cert\") pod \"3aee81c5-228b-4f21-b7d9-a0bd0822b4df\" (UID: \"3aee81c5-228b-4f21-b7d9-a0bd0822b4df\") "
Oct 30 21:51:53.611718 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:53.611478    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7679391-8575-43d7-b7f1-5b1ab0bcee5f-workload-socket" pod "e7679391-8575-43d7-b7f1-5b1ab0bcee5f" (UID: "e7679391-8575-43d7-b7f1-5b1ab0bcee5f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:53.611718 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:53.611506    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3aee81c5-228b-4f21-b7d9-a0bd0822b4df-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3aee81c5-228b-4f21-b7d9-a0bd0822b4df-istio-envoy\") pod \"3aee81c5-228b-4f21-b7d9-a0bd0822b4df\" (UID: \"3aee81c5-228b-4f21-b7d9-a0bd0822b4df\") "
Oct 30 21:51:53.611718 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:53.611535    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e7679391-8575-43d7-b7f1-5b1ab0bcee5f-istio-token\" (UniqueName: \"kubernetes.io/projected/e7679391-8575-43d7-b7f1-5b1ab0bcee5f-istio-token\") pod \"e7679391-8575-43d7-b7f1-5b1ab0bcee5f\" (UID: \"e7679391-8575-43d7-b7f1-5b1ab0bcee5f\") "
Oct 30 21:51:53.611718 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:53.611569    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e7679391-8575-43d7-b7f1-5b1ab0bcee5f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7679391-8575-43d7-b7f1-5b1ab0bcee5f-credential-socket\") pod \"e7679391-8575-43d7-b7f1-5b1ab0bcee5f\" (UID: \"e7679391-8575-43d7-b7f1-5b1ab0bcee5f\") "
Oct 30 21:51:53.611718 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:53.611613    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e7679391-8575-43d7-b7f1-5b1ab0bcee5f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e7679391-8575-43d7-b7f1-5b1ab0bcee5f-istio-envoy\") pod \"e7679391-8575-43d7-b7f1-5b1ab0bcee5f\" (UID: \"e7679391-8575-43d7-b7f1-5b1ab0bcee5f\") "
Oct 30 21:51:53.612082 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:53.611929    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7679391-8575-43d7-b7f1-5b1ab0bcee5f-workload-certs" pod "e7679391-8575-43d7-b7f1-5b1ab0bcee5f" (UID: "e7679391-8575-43d7-b7f1-5b1ab0bcee5f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:53.612082 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:53.611968    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3aee81c5-228b-4f21-b7d9-a0bd0822b4df-istio-data" pod "3aee81c5-228b-4f21-b7d9-a0bd0822b4df" (UID: "3aee81c5-228b-4f21-b7d9-a0bd0822b4df"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:53.612252 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:53.612224    5032 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3aee81c5-228b-4f21-b7d9-a0bd0822b4df-workload-certs\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:53.612252 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:53.612246    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7679391-8575-43d7-b7f1-5b1ab0bcee5f-istiod-ca-cert" pod "e7679391-8575-43d7-b7f1-5b1ab0bcee5f" (UID: "e7679391-8575-43d7-b7f1-5b1ab0bcee5f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:51:53.612470 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:53.612255    5032 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3aee81c5-228b-4f21-b7d9-a0bd0822b4df-credential-socket\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:53.612470 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:53.612271    5032 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7679391-8575-43d7-b7f1-5b1ab0bcee5f-workload-certs\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:53.612470 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:53.612287    5032 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3aee81c5-228b-4f21-b7d9-a0bd0822b4df-istio-data\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:53.612470 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:53.612302    5032 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7679391-8575-43d7-b7f1-5b1ab0bcee5f-workload-socket\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:53.612681 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:53.612655    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3aee81c5-228b-4f21-b7d9-a0bd0822b4df-istiod-ca-cert" pod "3aee81c5-228b-4f21-b7d9-a0bd0822b4df" (UID: "3aee81c5-228b-4f21-b7d9-a0bd0822b4df"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:51:53.613679 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:53.613653    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7679391-8575-43d7-b7f1-5b1ab0bcee5f-istio-data" pod "e7679391-8575-43d7-b7f1-5b1ab0bcee5f" (UID: "e7679391-8575-43d7-b7f1-5b1ab0bcee5f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:53.613960 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:53.613891    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7679391-8575-43d7-b7f1-5b1ab0bcee5f-credential-socket" pod "e7679391-8575-43d7-b7f1-5b1ab0bcee5f" (UID: "e7679391-8575-43d7-b7f1-5b1ab0bcee5f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:53.614757 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:53.614729    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3aee81c5-228b-4f21-b7d9-a0bd0822b4df-istio-podinfo" pod "3aee81c5-228b-4f21-b7d9-a0bd0822b4df" (UID: "3aee81c5-228b-4f21-b7d9-a0bd0822b4df"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:51:53.614938 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:53.614917    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3aee81c5-228b-4f21-b7d9-a0bd0822b4df-istio-envoy" pod "3aee81c5-228b-4f21-b7d9-a0bd0822b4df" (UID: "3aee81c5-228b-4f21-b7d9-a0bd0822b4df"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:53.615523 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:53.615501    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7679391-8575-43d7-b7f1-5b1ab0bcee5f-istio-envoy" pod "e7679391-8575-43d7-b7f1-5b1ab0bcee5f" (UID: "e7679391-8575-43d7-b7f1-5b1ab0bcee5f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:53.615523 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:53.615514    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3aee81c5-228b-4f21-b7d9-a0bd0822b4df-kube-api-access-5z877" pod "3aee81c5-228b-4f21-b7d9-a0bd0822b4df" (UID: "3aee81c5-228b-4f21-b7d9-a0bd0822b4df"). InnerVolumeSpecName "kube-api-access-5z877". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:51:53.615667 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:53.615507    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e7679391-8575-43d7-b7f1-5b1ab0bcee5f-istio-podinfo" pod "e7679391-8575-43d7-b7f1-5b1ab0bcee5f" (UID: "e7679391-8575-43d7-b7f1-5b1ab0bcee5f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:51:53.615977 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:53.615958    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7679391-8575-43d7-b7f1-5b1ab0bcee5f-istio-token" pod "e7679391-8575-43d7-b7f1-5b1ab0bcee5f" (UID: "e7679391-8575-43d7-b7f1-5b1ab0bcee5f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:51:53.616854 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:53.616833    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3aee81c5-228b-4f21-b7d9-a0bd0822b4df-istio-token" pod "3aee81c5-228b-4f21-b7d9-a0bd0822b4df" (UID: "3aee81c5-228b-4f21-b7d9-a0bd0822b4df"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:51:53.617733 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:53.617714    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7679391-8575-43d7-b7f1-5b1ab0bcee5f-kube-api-access-2hbd2" pod "e7679391-8575-43d7-b7f1-5b1ab0bcee5f" (UID: "e7679391-8575-43d7-b7f1-5b1ab0bcee5f"). InnerVolumeSpecName "kube-api-access-2hbd2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:51:53.712997 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:53.712952    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3aee81c5-228b-4f21-b7d9-a0bd0822b4df-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3aee81c5-228b-4f21-b7d9-a0bd0822b4df-workload-socket\") pod \"3aee81c5-228b-4f21-b7d9-a0bd0822b4df\" (UID: \"3aee81c5-228b-4f21-b7d9-a0bd0822b4df\") "
Oct 30 21:51:53.713223 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:53.713200    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3aee81c5-228b-4f21-b7d9-a0bd0822b4df-workload-socket" pod "3aee81c5-228b-4f21-b7d9-a0bd0822b4df" (UID: "3aee81c5-228b-4f21-b7d9-a0bd0822b4df"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:53.713430 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:53.713414    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2hbd2\" (UniqueName: \"kubernetes.io/projected/e7679391-8575-43d7-b7f1-5b1ab0bcee5f-kube-api-access-2hbd2\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:53.713466 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:53.713436    5032 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e7679391-8575-43d7-b7f1-5b1ab0bcee5f-istio-data\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:53.713466 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:53.713452    5032 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3aee81c5-228b-4f21-b7d9-a0bd0822b4df-istiod-ca-cert\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:53.713532 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:53.713465    5032 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3aee81c5-228b-4f21-b7d9-a0bd0822b4df-istio-envoy\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:53.713532 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:53.713480    5032 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7679391-8575-43d7-b7f1-5b1ab0bcee5f-istio-token\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:53.713532 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:53.713493    5032 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7679391-8575-43d7-b7f1-5b1ab0bcee5f-credential-socket\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:53.713532 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:53.713507    5032 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e7679391-8575-43d7-b7f1-5b1ab0bcee5f-istio-envoy\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:53.713532 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:53.713522    5032 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3aee81c5-228b-4f21-b7d9-a0bd0822b4df-workload-socket\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:53.713748 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:53.713537    5032 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3aee81c5-228b-4f21-b7d9-a0bd0822b4df-istio-token\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:53.713748 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:53.713551    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5z877\" (UniqueName: \"kubernetes.io/projected/3aee81c5-228b-4f21-b7d9-a0bd0822b4df-kube-api-access-5z877\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:53.713748 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:53.713565    5032 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e7679391-8575-43d7-b7f1-5b1ab0bcee5f-istiod-ca-cert\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:53.713748 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:53.713580    5032 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3aee81c5-228b-4f21-b7d9-a0bd0822b4df-istio-podinfo\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:53.713748 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:53.713595    5032 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e7679391-8575-43d7-b7f1-5b1ab0bcee5f-istio-podinfo\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:54.449726 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:54.449691    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b"
Oct 30 21:51:54.450197 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:54.449693    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b" event={"ID":"3aee81c5-228b-4f21-b7d9-a0bd0822b4df","Type":"ContainerDied","Data":"adbf88a579c49b10072b440671d3d3caf1de0688f39ccc1d349261f6b595dff9"}
Oct 30 21:51:54.450197 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:54.449837    5032 scope.go:122] "RemoveContainer" containerID="31706ca5ffc512e69734a1d499097c7cd0a124432df0d948d2d5ef41d7a406ad"
Oct 30 21:51:54.451126 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:54.451062    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8" event={"ID":"e7679391-8575-43d7-b7f1-5b1ab0bcee5f","Type":"ContainerDied","Data":"7d71a3dc31c1ffee7eaa24ca73eb24e042d8c060cb046b61f75b46bbb650df2f"}
Oct 30 21:51:54.451126 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:54.451084    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8"
Oct 30 21:51:54.460401 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:54.460384    5032 scope.go:122] "RemoveContainer" containerID="b026d760849187704d451a5830315d5a4a7556c362335e17873b515117d83d93"
Oct 30 21:51:54.484711 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:54.484683    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b"]
Oct 30 21:51:54.493225 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:54.493197    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-cgp6b"]
Oct 30 21:51:54.517499 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:54.517406    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8"]
Oct 30 21:51:54.517499 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:54.517494    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-0fa864-openshift-default-97b498755-4slr8"]
Oct 30 21:51:55.315807 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:55.315739    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-caa3dc-85fc67b867-gsjl2" podUID="28ed1f60-5004-4fb1-bb80-33cd040611f6" containerName="payload-pre-processing" containerID="cri-o://525daf5fc5fd23a19e3d8a883a2bc17bc3d4a939d45c2f36b5490e6fb136c888" gracePeriod=25
Oct 30 21:51:55.319396 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:55.319324    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w5-caa3dc-76db8c686c-pxdzh" podUID="467142c6-5675-4dd8-87a0-14e1f9fddc57" containerName="payload-processing" containerID="cri-o://b8c232edce4bac248fb110534e824aea7161314ba5b558c05d1e1a78d986e4de" gracePeriod=25
Oct 30 21:51:55.347138 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:55.347105    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3aee81c5-228b-4f21-b7d9-a0bd0822b4df" path="/var/lib/kubelet/pods/3aee81c5-228b-4f21-b7d9-a0bd0822b4df/volumes"
Oct 30 21:51:55.347625 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:55.347611    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e7679391-8575-43d7-b7f1-5b1ab0bcee5f" path="/var/lib/kubelet/pods/e7679391-8575-43d7-b7f1-5b1ab0bcee5f/volumes"
Oct 30 21:51:56.401559 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:56.401522    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-1d8034-5b96976fc5-hsr78" podUID="27aef427-6b1e-4845-b8d9-a83cba81b0e0" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:51:57.019595 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.018508    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-1d8034-6bbd97dcf7-l2gd8"
Oct 30 21:51:57.024629 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.024170    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-1d8034-5b96976fc5-hsr78"
Oct 30 21:51:57.061748 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.060660    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/27aef427-6b1e-4845-b8d9-a83cba81b0e0-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/27aef427-6b1e-4845-b8d9-a83cba81b0e0-plugins-config-volume\") pod \"27aef427-6b1e-4845-b8d9-a83cba81b0e0\" (UID: \"27aef427-6b1e-4845-b8d9-a83cba81b0e0\") "
Oct 30 21:51:57.061748 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.060762    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e25d6f68-6e47-449f-b02f-03fff2b2bb30-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e25d6f68-6e47-449f-b02f-03fff2b2bb30-plugins-config-volume\") pod \"e25d6f68-6e47-449f-b02f-03fff2b2bb30\" (UID: \"e25d6f68-6e47-449f-b02f-03fff2b2bb30\") "
Oct 30 21:51:57.061748 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.060849    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e25d6f68-6e47-449f-b02f-03fff2b2bb30-kube-api-access-j6d8p\" (UniqueName: \"kubernetes.io/projected/e25d6f68-6e47-449f-b02f-03fff2b2bb30-kube-api-access-j6d8p\") pod \"e25d6f68-6e47-449f-b02f-03fff2b2bb30\" (UID: \"e25d6f68-6e47-449f-b02f-03fff2b2bb30\") "
Oct 30 21:51:57.061748 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.060868    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/27aef427-6b1e-4845-b8d9-a83cba81b0e0-kube-api-access-nggn9\" (UniqueName: \"kubernetes.io/projected/27aef427-6b1e-4845-b8d9-a83cba81b0e0-kube-api-access-nggn9\") pod \"27aef427-6b1e-4845-b8d9-a83cba81b0e0\" (UID: \"27aef427-6b1e-4845-b8d9-a83cba81b0e0\") "
Oct 30 21:51:57.064533 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.063460    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e25d6f68-6e47-449f-b02f-03fff2b2bb30-plugins-config-volume" pod "e25d6f68-6e47-449f-b02f-03fff2b2bb30" (UID: "e25d6f68-6e47-449f-b02f-03fff2b2bb30"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:51:57.064533 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.063891    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27aef427-6b1e-4845-b8d9-a83cba81b0e0-plugins-config-volume" pod "27aef427-6b1e-4845-b8d9-a83cba81b0e0" (UID: "27aef427-6b1e-4845-b8d9-a83cba81b0e0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:51:57.068973 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.068944    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e25d6f68-6e47-449f-b02f-03fff2b2bb30-kube-api-access-j6d8p" pod "e25d6f68-6e47-449f-b02f-03fff2b2bb30" (UID: "e25d6f68-6e47-449f-b02f-03fff2b2bb30"). InnerVolumeSpecName "kube-api-access-j6d8p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:51:57.069706 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.069678    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27aef427-6b1e-4845-b8d9-a83cba81b0e0-kube-api-access-nggn9" pod "27aef427-6b1e-4845-b8d9-a83cba81b0e0" (UID: "27aef427-6b1e-4845-b8d9-a83cba81b0e0"). InnerVolumeSpecName "kube-api-access-nggn9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:51:57.162777 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.162656    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j6d8p\" (UniqueName: \"kubernetes.io/projected/e25d6f68-6e47-449f-b02f-03fff2b2bb30-kube-api-access-j6d8p\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:57.162777 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.162709    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nggn9\" (UniqueName: \"kubernetes.io/projected/27aef427-6b1e-4845-b8d9-a83cba81b0e0-kube-api-access-nggn9\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:57.162777 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.162726    5032 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/27aef427-6b1e-4845-b8d9-a83cba81b0e0-plugins-config-volume\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:57.162777 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.162740    5032 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e25d6f68-6e47-449f-b02f-03fff2b2bb30-plugins-config-volume\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:57.248466 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.248435    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7"
Oct 30 21:51:57.266567 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.266538    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76"
Oct 30 21:51:57.364442 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.364344    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0d5b3ad5-309e-4c34-89a3-034e4966e652-istio-token\" (UniqueName: \"kubernetes.io/projected/0d5b3ad5-309e-4c34-89a3-034e4966e652-istio-token\") pod \"0d5b3ad5-309e-4c34-89a3-034e4966e652\" (UID: \"0d5b3ad5-309e-4c34-89a3-034e4966e652\") "
Oct 30 21:51:57.364442 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.364413    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cd4228de-c71f-4f88-a54e-9ebda8b91122-kube-api-access-8v9qm\" (UniqueName: \"kubernetes.io/projected/cd4228de-c71f-4f88-a54e-9ebda8b91122-kube-api-access-8v9qm\") pod \"cd4228de-c71f-4f88-a54e-9ebda8b91122\" (UID: \"cd4228de-c71f-4f88-a54e-9ebda8b91122\") "
Oct 30 21:51:57.364670 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.364516    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cd4228de-c71f-4f88-a54e-9ebda8b91122-istio-token\" (UniqueName: \"kubernetes.io/projected/cd4228de-c71f-4f88-a54e-9ebda8b91122-istio-token\") pod \"cd4228de-c71f-4f88-a54e-9ebda8b91122\" (UID: \"cd4228de-c71f-4f88-a54e-9ebda8b91122\") "
Oct 30 21:51:57.364670 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.364552    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0d5b3ad5-309e-4c34-89a3-034e4966e652-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d5b3ad5-309e-4c34-89a3-034e4966e652-istio-podinfo\") pod \"0d5b3ad5-309e-4c34-89a3-034e4966e652\" (UID: \"0d5b3ad5-309e-4c34-89a3-034e4966e652\") "
Oct 30 21:51:57.364670 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.364588    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cd4228de-c71f-4f88-a54e-9ebda8b91122-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd4228de-c71f-4f88-a54e-9ebda8b91122-istio-envoy\") pod \"cd4228de-c71f-4f88-a54e-9ebda8b91122\" (UID: \"cd4228de-c71f-4f88-a54e-9ebda8b91122\") "
Oct 30 21:51:57.364670 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.364641    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0d5b3ad5-309e-4c34-89a3-034e4966e652-kube-api-access-q7vwp\" (UniqueName: \"kubernetes.io/projected/0d5b3ad5-309e-4c34-89a3-034e4966e652-kube-api-access-q7vwp\") pod \"0d5b3ad5-309e-4c34-89a3-034e4966e652\" (UID: \"0d5b3ad5-309e-4c34-89a3-034e4966e652\") "
Oct 30 21:51:57.364904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.364685    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/cd4228de-c71f-4f88-a54e-9ebda8b91122-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd4228de-c71f-4f88-a54e-9ebda8b91122-istio-podinfo\") pod \"cd4228de-c71f-4f88-a54e-9ebda8b91122\" (UID: \"cd4228de-c71f-4f88-a54e-9ebda8b91122\") "
Oct 30 21:51:57.364904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.364723    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0d5b3ad5-309e-4c34-89a3-034e4966e652-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d5b3ad5-309e-4c34-89a3-034e4966e652-istiod-ca-cert\") pod \"0d5b3ad5-309e-4c34-89a3-034e4966e652\" (UID: \"0d5b3ad5-309e-4c34-89a3-034e4966e652\") "
Oct 30 21:51:57.364904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.364770    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d5b3ad5-309e-4c34-89a3-034e4966e652-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d5b3ad5-309e-4c34-89a3-034e4966e652-istio-data\") pod \"0d5b3ad5-309e-4c34-89a3-034e4966e652\" (UID: \"0d5b3ad5-309e-4c34-89a3-034e4966e652\") "
Oct 30 21:51:57.364904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.364800    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cd4228de-c71f-4f88-a54e-9ebda8b91122-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd4228de-c71f-4f88-a54e-9ebda8b91122-credential-socket\") pod \"cd4228de-c71f-4f88-a54e-9ebda8b91122\" (UID: \"cd4228de-c71f-4f88-a54e-9ebda8b91122\") "
Oct 30 21:51:57.364904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.364852    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d5b3ad5-309e-4c34-89a3-034e4966e652-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d5b3ad5-309e-4c34-89a3-034e4966e652-workload-socket\") pod \"0d5b3ad5-309e-4c34-89a3-034e4966e652\" (UID: \"0d5b3ad5-309e-4c34-89a3-034e4966e652\") "
Oct 30 21:51:57.364904 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.364881    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cd4228de-c71f-4f88-a54e-9ebda8b91122-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd4228de-c71f-4f88-a54e-9ebda8b91122-workload-socket\") pod \"cd4228de-c71f-4f88-a54e-9ebda8b91122\" (UID: \"cd4228de-c71f-4f88-a54e-9ebda8b91122\") "
Oct 30 21:51:57.365193 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.364969    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cd4228de-c71f-4f88-a54e-9ebda8b91122-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd4228de-c71f-4f88-a54e-9ebda8b91122-istio-data\") pod \"cd4228de-c71f-4f88-a54e-9ebda8b91122\" (UID: \"cd4228de-c71f-4f88-a54e-9ebda8b91122\") "
Oct 30 21:51:57.365193 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.364998    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d5b3ad5-309e-4c34-89a3-034e4966e652-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d5b3ad5-309e-4c34-89a3-034e4966e652-credential-socket\") pod \"0d5b3ad5-309e-4c34-89a3-034e4966e652\" (UID: \"0d5b3ad5-309e-4c34-89a3-034e4966e652\") "
Oct 30 21:51:57.365193 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.365054    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d5b3ad5-309e-4c34-89a3-034e4966e652-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d5b3ad5-309e-4c34-89a3-034e4966e652-istio-envoy\") pod \"0d5b3ad5-309e-4c34-89a3-034e4966e652\" (UID: \"0d5b3ad5-309e-4c34-89a3-034e4966e652\") "
Oct 30 21:51:57.365193 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.365098    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cd4228de-c71f-4f88-a54e-9ebda8b91122-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd4228de-c71f-4f88-a54e-9ebda8b91122-istiod-ca-cert\") pod \"cd4228de-c71f-4f88-a54e-9ebda8b91122\" (UID: \"cd4228de-c71f-4f88-a54e-9ebda8b91122\") "
Oct 30 21:51:57.365193 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.365154    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d5b3ad5-309e-4c34-89a3-034e4966e652-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d5b3ad5-309e-4c34-89a3-034e4966e652-workload-certs\") pod \"0d5b3ad5-309e-4c34-89a3-034e4966e652\" (UID: \"0d5b3ad5-309e-4c34-89a3-034e4966e652\") "
Oct 30 21:51:57.365193 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.365190    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cd4228de-c71f-4f88-a54e-9ebda8b91122-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd4228de-c71f-4f88-a54e-9ebda8b91122-workload-certs\") pod \"cd4228de-c71f-4f88-a54e-9ebda8b91122\" (UID: \"cd4228de-c71f-4f88-a54e-9ebda8b91122\") "
Oct 30 21:51:57.366004 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.365931    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd4228de-c71f-4f88-a54e-9ebda8b91122-workload-certs" pod "cd4228de-c71f-4f88-a54e-9ebda8b91122" (UID: "cd4228de-c71f-4f88-a54e-9ebda8b91122"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:57.366601 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.366571    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d5b3ad5-309e-4c34-89a3-034e4966e652-istio-data" pod "0d5b3ad5-309e-4c34-89a3-034e4966e652" (UID: "0d5b3ad5-309e-4c34-89a3-034e4966e652"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:57.366782 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.366709    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd4228de-c71f-4f88-a54e-9ebda8b91122-istio-data" pod "cd4228de-c71f-4f88-a54e-9ebda8b91122" (UID: "cd4228de-c71f-4f88-a54e-9ebda8b91122"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:57.367121 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.366938    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd4228de-c71f-4f88-a54e-9ebda8b91122-workload-socket" pod "cd4228de-c71f-4f88-a54e-9ebda8b91122" (UID: "cd4228de-c71f-4f88-a54e-9ebda8b91122"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:57.367121 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.367066    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d5b3ad5-309e-4c34-89a3-034e4966e652-workload-socket" pod "0d5b3ad5-309e-4c34-89a3-034e4966e652" (UID: "0d5b3ad5-309e-4c34-89a3-034e4966e652"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:57.368024 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.367352    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd4228de-c71f-4f88-a54e-9ebda8b91122-credential-socket" pod "cd4228de-c71f-4f88-a54e-9ebda8b91122" (UID: "cd4228de-c71f-4f88-a54e-9ebda8b91122"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:57.368024 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.367623    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d5b3ad5-309e-4c34-89a3-034e4966e652-credential-socket" pod "0d5b3ad5-309e-4c34-89a3-034e4966e652" (UID: "0d5b3ad5-309e-4c34-89a3-034e4966e652"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:57.368024 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.367665    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd4228de-c71f-4f88-a54e-9ebda8b91122-istiod-ca-cert" pod "cd4228de-c71f-4f88-a54e-9ebda8b91122" (UID: "cd4228de-c71f-4f88-a54e-9ebda8b91122"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:51:57.369886 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.369867    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d5b3ad5-309e-4c34-89a3-034e4966e652-workload-certs" pod "0d5b3ad5-309e-4c34-89a3-034e4966e652" (UID: "0d5b3ad5-309e-4c34-89a3-034e4966e652"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:57.370439 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.370413    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd4228de-c71f-4f88-a54e-9ebda8b91122-istio-token" pod "cd4228de-c71f-4f88-a54e-9ebda8b91122" (UID: "cd4228de-c71f-4f88-a54e-9ebda8b91122"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:51:57.370799 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.370776    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d5b3ad5-309e-4c34-89a3-034e4966e652-istiod-ca-cert" pod "0d5b3ad5-309e-4c34-89a3-034e4966e652" (UID: "0d5b3ad5-309e-4c34-89a3-034e4966e652"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:51:57.370984 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.370962    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d5b3ad5-309e-4c34-89a3-034e4966e652-istio-envoy" pod "0d5b3ad5-309e-4c34-89a3-034e4966e652" (UID: "0d5b3ad5-309e-4c34-89a3-034e4966e652"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:57.371052 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.370995    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d5b3ad5-309e-4c34-89a3-034e4966e652-istio-token" pod "0d5b3ad5-309e-4c34-89a3-034e4966e652" (UID: "0d5b3ad5-309e-4c34-89a3-034e4966e652"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:51:57.371323 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.371298    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd4228de-c71f-4f88-a54e-9ebda8b91122-kube-api-access-8v9qm" pod "cd4228de-c71f-4f88-a54e-9ebda8b91122" (UID: "cd4228de-c71f-4f88-a54e-9ebda8b91122"). InnerVolumeSpecName "kube-api-access-8v9qm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:51:57.372146 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.372065    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0d5b3ad5-309e-4c34-89a3-034e4966e652-istio-podinfo" pod "0d5b3ad5-309e-4c34-89a3-034e4966e652" (UID: "0d5b3ad5-309e-4c34-89a3-034e4966e652"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:51:57.372343 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.372322    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d5b3ad5-309e-4c34-89a3-034e4966e652-kube-api-access-q7vwp" pod "0d5b3ad5-309e-4c34-89a3-034e4966e652" (UID: "0d5b3ad5-309e-4c34-89a3-034e4966e652"). InnerVolumeSpecName "kube-api-access-q7vwp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:51:57.373172 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.373153    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd4228de-c71f-4f88-a54e-9ebda8b91122-istio-envoy" pod "cd4228de-c71f-4f88-a54e-9ebda8b91122" (UID: "cd4228de-c71f-4f88-a54e-9ebda8b91122"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:57.376181 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.373841    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cd4228de-c71f-4f88-a54e-9ebda8b91122-istio-podinfo" pod "cd4228de-c71f-4f88-a54e-9ebda8b91122" (UID: "cd4228de-c71f-4f88-a54e-9ebda8b91122"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:51:57.413103 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.413062    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w5-caa3dc-76db8c686c-pxdzh" podUID="467142c6-5675-4dd8-87a0-14e1f9fddc57" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:51:57.466834 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.466582    5032 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d5b3ad5-309e-4c34-89a3-034e4966e652-workload-socket\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:57.466834 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.466618    5032 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd4228de-c71f-4f88-a54e-9ebda8b91122-workload-socket\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:57.466834 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.466632    5032 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd4228de-c71f-4f88-a54e-9ebda8b91122-istio-data\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:57.466834 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.466648    5032 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d5b3ad5-309e-4c34-89a3-034e4966e652-credential-socket\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:57.466834 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.466662    5032 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d5b3ad5-309e-4c34-89a3-034e4966e652-istio-envoy\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:57.466834 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.466675    5032 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd4228de-c71f-4f88-a54e-9ebda8b91122-istiod-ca-cert\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:57.466834 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.466688    5032 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d5b3ad5-309e-4c34-89a3-034e4966e652-workload-certs\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:57.466834 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.466703    5032 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd4228de-c71f-4f88-a54e-9ebda8b91122-workload-certs\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:57.466834 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.466716    5032 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d5b3ad5-309e-4c34-89a3-034e4966e652-istio-token\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:57.466834 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.466730    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8v9qm\" (UniqueName: \"kubernetes.io/projected/cd4228de-c71f-4f88-a54e-9ebda8b91122-kube-api-access-8v9qm\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:57.466834 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.466745    5032 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd4228de-c71f-4f88-a54e-9ebda8b91122-istio-token\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:57.466834 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.466759    5032 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d5b3ad5-309e-4c34-89a3-034e4966e652-istio-podinfo\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:57.466834 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.466773    5032 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd4228de-c71f-4f88-a54e-9ebda8b91122-istio-envoy\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:57.466834 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.466786    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q7vwp\" (UniqueName: \"kubernetes.io/projected/0d5b3ad5-309e-4c34-89a3-034e4966e652-kube-api-access-q7vwp\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:57.466834 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.466801    5032 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd4228de-c71f-4f88-a54e-9ebda8b91122-istio-podinfo\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:57.466834 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.466814    5032 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d5b3ad5-309e-4c34-89a3-034e4966e652-istiod-ca-cert\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:57.466834 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.466827    5032 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d5b3ad5-309e-4c34-89a3-034e4966e652-istio-data\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:57.466834 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.466841    5032 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd4228de-c71f-4f88-a54e-9ebda8b91122-credential-socket\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:57.473245 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.473212    5032 generic.go:356] "Generic (PLEG): container finished" podID="0d5b3ad5-309e-4c34-89a3-034e4966e652" containerID="105e63cdb204bccc898f9b89b178f9a801281db6bf9fbbad88078ba3a60be6cb" exitCode=0
Oct 30 21:51:57.473402 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.473253    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7"
Oct 30 21:51:57.473402 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.473299    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7" event={"ID":"0d5b3ad5-309e-4c34-89a3-034e4966e652","Type":"ContainerDied","Data":"105e63cdb204bccc898f9b89b178f9a801281db6bf9fbbad88078ba3a60be6cb"}
Oct 30 21:51:57.473402 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.473351    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7" event={"ID":"0d5b3ad5-309e-4c34-89a3-034e4966e652","Type":"ContainerDied","Data":"d4bfa675eb96d503b527fe91ff678783808a554a1094a2c9cb1306cbfd6dcecc"}
Oct 30 21:51:57.473402 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.473393    5032 scope.go:122] "RemoveContainer" containerID="105e63cdb204bccc898f9b89b178f9a801281db6bf9fbbad88078ba3a60be6cb"
Oct 30 21:51:57.474889 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.474862    5032 generic.go:356] "Generic (PLEG): container finished" podID="e25d6f68-6e47-449f-b02f-03fff2b2bb30" containerID="5e56a198bf2559217d743ea619f914b5b821cf25512ea712631b76860cd9dd7e" exitCode=0
Oct 30 21:51:57.475015 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.474890    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-1d8034-6bbd97dcf7-l2gd8" event={"ID":"e25d6f68-6e47-449f-b02f-03fff2b2bb30","Type":"ContainerDied","Data":"5e56a198bf2559217d743ea619f914b5b821cf25512ea712631b76860cd9dd7e"}
Oct 30 21:51:57.475015 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.474957    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-1d8034-6bbd97dcf7-l2gd8" event={"ID":"e25d6f68-6e47-449f-b02f-03fff2b2bb30","Type":"ContainerDied","Data":"c18a102100a9a20e66a0e5cb794b547fb308896b667e989e0c6c31da400f58fb"}
Oct 30 21:51:57.475015 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.474917    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-1d8034-6bbd97dcf7-l2gd8"
Oct 30 21:51:57.476601 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.476579    5032 generic.go:356] "Generic (PLEG): container finished" podID="cd4228de-c71f-4f88-a54e-9ebda8b91122" containerID="4637b417d0d41dd176aed51d61219db6bdd0debc511a69a2e445a093cd626e4d" exitCode=0
Oct 30 21:51:57.476728 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.476648    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76" event={"ID":"cd4228de-c71f-4f88-a54e-9ebda8b91122","Type":"ContainerDied","Data":"4637b417d0d41dd176aed51d61219db6bdd0debc511a69a2e445a093cd626e4d"}
Oct 30 21:51:57.476728 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.476690    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76" event={"ID":"cd4228de-c71f-4f88-a54e-9ebda8b91122","Type":"ContainerDied","Data":"519894c4789453a0a85fe8a84f06fb0900004326c673254d6c184489ca3b4f10"}
Oct 30 21:51:57.476728 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.476651    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76"
Oct 30 21:51:57.479334 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.479307    5032 generic.go:356] "Generic (PLEG): container finished" podID="27aef427-6b1e-4845-b8d9-a83cba81b0e0" containerID="4a6b5c624625d96186bc32eb917cd7d6554b6357f49b5e34177e00d4855580ff" exitCode=0
Oct 30 21:51:57.479445 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.479392    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-1d8034-5b96976fc5-hsr78" event={"ID":"27aef427-6b1e-4845-b8d9-a83cba81b0e0","Type":"ContainerDied","Data":"4a6b5c624625d96186bc32eb917cd7d6554b6357f49b5e34177e00d4855580ff"}
Oct 30 21:51:57.479445 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.479421    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-1d8034-5b96976fc5-hsr78" event={"ID":"27aef427-6b1e-4845-b8d9-a83cba81b0e0","Type":"ContainerDied","Data":"5d69c766bffc128b32ea44997ca5922eff6540283897651764612cec07cb9a85"}
Oct 30 21:51:57.479530 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.479464    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-1d8034-5b96976fc5-hsr78"
Oct 30 21:51:57.485424 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.485399    5032 scope.go:122] "RemoveContainer" containerID="105e63cdb204bccc898f9b89b178f9a801281db6bf9fbbad88078ba3a60be6cb"
Oct 30 21:51:57.485686 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:51:57.485666    5032 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"105e63cdb204bccc898f9b89b178f9a801281db6bf9fbbad88078ba3a60be6cb\": container with ID starting with 105e63cdb204bccc898f9b89b178f9a801281db6bf9fbbad88078ba3a60be6cb not found: ID does not exist" containerID="105e63cdb204bccc898f9b89b178f9a801281db6bf9fbbad88078ba3a60be6cb"
Oct 30 21:51:57.485763 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.485696    5032 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"105e63cdb204bccc898f9b89b178f9a801281db6bf9fbbad88078ba3a60be6cb"} err="failed to get container status \"105e63cdb204bccc898f9b89b178f9a801281db6bf9fbbad88078ba3a60be6cb\": rpc error: code = NotFound desc = could not find container \"105e63cdb204bccc898f9b89b178f9a801281db6bf9fbbad88078ba3a60be6cb\": container with ID starting with 105e63cdb204bccc898f9b89b178f9a801281db6bf9fbbad88078ba3a60be6cb not found: ID does not exist"
Oct 30 21:51:57.485763 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.485720    5032 scope.go:122] "RemoveContainer" containerID="5e56a198bf2559217d743ea619f914b5b821cf25512ea712631b76860cd9dd7e"
Oct 30 21:51:57.495140 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.495122    5032 scope.go:122] "RemoveContainer" containerID="5e56a198bf2559217d743ea619f914b5b821cf25512ea712631b76860cd9dd7e"
Oct 30 21:51:57.495517 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:51:57.495495    5032 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e56a198bf2559217d743ea619f914b5b821cf25512ea712631b76860cd9dd7e\": container with ID starting with 5e56a198bf2559217d743ea619f914b5b821cf25512ea712631b76860cd9dd7e not found: ID does not exist" containerID="5e56a198bf2559217d743ea619f914b5b821cf25512ea712631b76860cd9dd7e"
Oct 30 21:51:57.495611 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.495523    5032 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e56a198bf2559217d743ea619f914b5b821cf25512ea712631b76860cd9dd7e"} err="failed to get container status \"5e56a198bf2559217d743ea619f914b5b821cf25512ea712631b76860cd9dd7e\": rpc error: code = NotFound desc = could not find container \"5e56a198bf2559217d743ea619f914b5b821cf25512ea712631b76860cd9dd7e\": container with ID starting with 5e56a198bf2559217d743ea619f914b5b821cf25512ea712631b76860cd9dd7e not found: ID does not exist"
Oct 30 21:51:57.495611 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.495542    5032 scope.go:122] "RemoveContainer" containerID="4637b417d0d41dd176aed51d61219db6bdd0debc511a69a2e445a093cd626e4d"
Oct 30 21:51:57.504420 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.504397    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7"]
Oct 30 21:51:57.505654 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.505634    5032 scope.go:122] "RemoveContainer" containerID="4637b417d0d41dd176aed51d61219db6bdd0debc511a69a2e445a093cd626e4d"
Oct 30 21:51:57.505944 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:51:57.505927    5032 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4637b417d0d41dd176aed51d61219db6bdd0debc511a69a2e445a093cd626e4d\": container with ID starting with 4637b417d0d41dd176aed51d61219db6bdd0debc511a69a2e445a093cd626e4d not found: ID does not exist" containerID="4637b417d0d41dd176aed51d61219db6bdd0debc511a69a2e445a093cd626e4d"
Oct 30 21:51:57.506012 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.505951    5032 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4637b417d0d41dd176aed51d61219db6bdd0debc511a69a2e445a093cd626e4d"} err="failed to get container status \"4637b417d0d41dd176aed51d61219db6bdd0debc511a69a2e445a093cd626e4d\": rpc error: code = NotFound desc = could not find container \"4637b417d0d41dd176aed51d61219db6bdd0debc511a69a2e445a093cd626e4d\": container with ID starting with 4637b417d0d41dd176aed51d61219db6bdd0debc511a69a2e445a093cd626e4d not found: ID does not exist"
Oct 30 21:51:57.506012 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.505968    5032 scope.go:122] "RemoveContainer" containerID="4a6b5c624625d96186bc32eb917cd7d6554b6357f49b5e34177e00d4855580ff"
Oct 30 21:51:57.507327 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.507308    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-czxf7"]
Oct 30 21:51:57.515226 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.515211    5032 scope.go:122] "RemoveContainer" containerID="4a6b5c624625d96186bc32eb917cd7d6554b6357f49b5e34177e00d4855580ff"
Oct 30 21:51:57.515625 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:51:57.515561    5032 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a6b5c624625d96186bc32eb917cd7d6554b6357f49b5e34177e00d4855580ff\": container with ID starting with 4a6b5c624625d96186bc32eb917cd7d6554b6357f49b5e34177e00d4855580ff not found: ID does not exist" containerID="4a6b5c624625d96186bc32eb917cd7d6554b6357f49b5e34177e00d4855580ff"
Oct 30 21:51:57.516565 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.515622    5032 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a6b5c624625d96186bc32eb917cd7d6554b6357f49b5e34177e00d4855580ff"} err="failed to get container status \"4a6b5c624625d96186bc32eb917cd7d6554b6357f49b5e34177e00d4855580ff\": rpc error: code = NotFound desc = could not find container \"4a6b5c624625d96186bc32eb917cd7d6554b6357f49b5e34177e00d4855580ff\": container with ID starting with 4a6b5c624625d96186bc32eb917cd7d6554b6357f49b5e34177e00d4855580ff not found: ID does not exist"
Oct 30 21:51:57.519013 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.518991    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-1d8034-6bbd97dcf7-l2gd8"]
Oct 30 21:51:57.525673 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.525653    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-1d8034-6bbd97dcf7-l2gd8"]
Oct 30 21:51:57.554541 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.554511    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-1d8034-5b96976fc5-hsr78"]
Oct 30 21:51:57.562828 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.562801    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-1d8034-5b96976fc5-hsr78"]
Oct 30 21:51:57.591219 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.591179    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76"]
Oct 30 21:51:57.604996 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:57.604960    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-1a81ae-openshift-default-cc77b7797-dhd76"]
Oct 30 21:51:58.435290 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:58.435213    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-caa3dc-85fc67b867-gsjl2" podUID="28ed1f60-5004-4fb1-bb80-33cd040611f6" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:51:59.352249 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:59.352216    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0d5b3ad5-309e-4c34-89a3-034e4966e652" path="/var/lib/kubelet/pods/0d5b3ad5-309e-4c34-89a3-034e4966e652/volumes"
Oct 30 21:51:59.352663 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:59.352649    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="27aef427-6b1e-4845-b8d9-a83cba81b0e0" path="/var/lib/kubelet/pods/27aef427-6b1e-4845-b8d9-a83cba81b0e0/volumes"
Oct 30 21:51:59.353068 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:59.353056    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cd4228de-c71f-4f88-a54e-9ebda8b91122" path="/var/lib/kubelet/pods/cd4228de-c71f-4f88-a54e-9ebda8b91122/volumes"
Oct 30 21:51:59.353747 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:59.353731    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e25d6f68-6e47-449f-b02f-03fff2b2bb30" path="/var/lib/kubelet/pods/e25d6f68-6e47-449f-b02f-03fff2b2bb30/volumes"
Oct 30 21:51:59.590797 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:59.590760    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf"]
Oct 30 21:51:59.591401 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:59.591189    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf" podUID="2b720f88-2fd8-4c4d-a300-941b1a9db6e2" containerName="istio-proxy" containerID="cri-o://dcd9676b4bf074fc246addfff9469b2223555476176e11d6aa22a488effc8ac7" gracePeriod=30
Oct 30 21:51:59.594857 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:59.594823    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv"]
Oct 30 21:51:59.595070 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:59.595048    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv" podUID="aed33297-3e27-4d4a-a767-4191e8bc5127" containerName="istio-proxy" containerID="cri-o://996f7341f0007a4f636c0f27df8ba37ce893d6675da2f33bffc987c040b54633" gracePeriod=30
Oct 30 21:51:59.626609 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:59.626526    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz"]
Oct 30 21:51:59.626763 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:59.626729    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz" podUID="ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74" containerName="istio-proxy" containerID="cri-o://eb74413af42ee9771451cc538cb1e339d53b7020e6d1fa2ef1baf2e878f9817b" gracePeriod=30
Oct 30 21:51:59.906520 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:51:59.906432    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj" podUID="d0ff5023-fd3f-4734-94c6-76b666010adf" containerName="main" containerID="cri-o://ea94fdb68f46ee7796c3cfa2a0f1a71db77b61558ac603043e078636e6316c03" gracePeriod=45
Oct 30 21:52:00.179478 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:00.179451    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj"
Oct 30 21:52:00.280900 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:00.280863    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d0ff5023-fd3f-4734-94c6-76b666010adf-tls-certs\" (UniqueName: \"kubernetes.io/secret/d0ff5023-fd3f-4734-94c6-76b666010adf-tls-certs\") pod \"d0ff5023-fd3f-4734-94c6-76b666010adf\" (UID: \"d0ff5023-fd3f-4734-94c6-76b666010adf\") "
Oct 30 21:52:00.281198 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:00.281170    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d0ff5023-fd3f-4734-94c6-76b666010adf-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0ff5023-fd3f-4734-94c6-76b666010adf-model-cache\") pod \"d0ff5023-fd3f-4734-94c6-76b666010adf\" (UID: \"d0ff5023-fd3f-4734-94c6-76b666010adf\") "
Oct 30 21:52:00.281779 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:00.281485    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0ff5023-fd3f-4734-94c6-76b666010adf-model-cache" pod "d0ff5023-fd3f-4734-94c6-76b666010adf" (UID: "d0ff5023-fd3f-4734-94c6-76b666010adf"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:00.281779 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:00.281563    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d0ff5023-fd3f-4734-94c6-76b666010adf-home\" (UniqueName: \"kubernetes.io/empty-dir/d0ff5023-fd3f-4734-94c6-76b666010adf-home\") pod \"d0ff5023-fd3f-4734-94c6-76b666010adf\" (UID: \"d0ff5023-fd3f-4734-94c6-76b666010adf\") "
Oct 30 21:52:00.283039 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:00.283000    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0ff5023-fd3f-4734-94c6-76b666010adf-home" pod "d0ff5023-fd3f-4734-94c6-76b666010adf" (UID: "d0ff5023-fd3f-4734-94c6-76b666010adf"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:00.283164 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:00.283046    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d0ff5023-fd3f-4734-94c6-76b666010adf-dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0ff5023-fd3f-4734-94c6-76b666010adf-dshm\") pod \"d0ff5023-fd3f-4734-94c6-76b666010adf\" (UID: \"d0ff5023-fd3f-4734-94c6-76b666010adf\") "
Oct 30 21:52:00.283230 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:00.283169    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d0ff5023-fd3f-4734-94c6-76b666010adf-kube-api-access-7mfgq\" (UniqueName: \"kubernetes.io/projected/d0ff5023-fd3f-4734-94c6-76b666010adf-kube-api-access-7mfgq\") pod \"d0ff5023-fd3f-4734-94c6-76b666010adf\" (UID: \"d0ff5023-fd3f-4734-94c6-76b666010adf\") "
Oct 30 21:52:00.283288 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:00.283265    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d0ff5023-fd3f-4734-94c6-76b666010adf-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0ff5023-fd3f-4734-94c6-76b666010adf-tmp-dir\") pod \"d0ff5023-fd3f-4734-94c6-76b666010adf\" (UID: \"d0ff5023-fd3f-4734-94c6-76b666010adf\") "
Oct 30 21:52:00.284131 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:00.283906    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0ff5023-fd3f-4734-94c6-76b666010adf-tmp-dir" pod "d0ff5023-fd3f-4734-94c6-76b666010adf" (UID: "d0ff5023-fd3f-4734-94c6-76b666010adf"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:00.284131 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:00.284077    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0ff5023-fd3f-4734-94c6-76b666010adf-tls-certs" pod "d0ff5023-fd3f-4734-94c6-76b666010adf" (UID: "d0ff5023-fd3f-4734-94c6-76b666010adf"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:52:00.284469 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:00.284451    5032 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0ff5023-fd3f-4734-94c6-76b666010adf-model-cache\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:00.284558 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:00.284472    5032 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0ff5023-fd3f-4734-94c6-76b666010adf-home\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:00.284558 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:00.284481    5032 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0ff5023-fd3f-4734-94c6-76b666010adf-tmp-dir\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:00.284558 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:00.284489    5032 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0ff5023-fd3f-4734-94c6-76b666010adf-tls-certs\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:00.286141 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:00.286114    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0ff5023-fd3f-4734-94c6-76b666010adf-dshm" pod "d0ff5023-fd3f-4734-94c6-76b666010adf" (UID: "d0ff5023-fd3f-4734-94c6-76b666010adf"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:00.286566 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:00.286527    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0ff5023-fd3f-4734-94c6-76b666010adf-kube-api-access-7mfgq" pod "d0ff5023-fd3f-4734-94c6-76b666010adf" (UID: "d0ff5023-fd3f-4734-94c6-76b666010adf"). InnerVolumeSpecName "kube-api-access-7mfgq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:52:00.386321 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:00.386283    5032 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0ff5023-fd3f-4734-94c6-76b666010adf-dshm\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:00.386321 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:00.386322    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7mfgq\" (UniqueName: \"kubernetes.io/projected/d0ff5023-fd3f-4734-94c6-76b666010adf-kube-api-access-7mfgq\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:00.701331 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:00.701296    5032 generic.go:356] "Generic (PLEG): container finished" podID="d0ff5023-fd3f-4734-94c6-76b666010adf" containerID="ea94fdb68f46ee7796c3cfa2a0f1a71db77b61558ac603043e078636e6316c03" exitCode=0
Oct 30 21:52:00.701829 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:00.701338    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj"
Oct 30 21:52:00.701829 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:00.701409    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj" event={"ID":"d0ff5023-fd3f-4734-94c6-76b666010adf","Type":"ContainerDied","Data":"ea94fdb68f46ee7796c3cfa2a0f1a71db77b61558ac603043e078636e6316c03"}
Oct 30 21:52:00.701829 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:00.701458    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj" event={"ID":"d0ff5023-fd3f-4734-94c6-76b666010adf","Type":"ContainerDied","Data":"118a0f2de3563d5cb90ed43134d621c9e958b41f296c5b96f0b0e85241a9b6bb"}
Oct 30 21:52:00.701829 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:00.701483    5032 scope.go:122] "RemoveContainer" containerID="ea94fdb68f46ee7796c3cfa2a0f1a71db77b61558ac603043e078636e6316c03"
Oct 30 21:52:00.716033 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:00.716002    5032 scope.go:122] "RemoveContainer" containerID="ea94fdb68f46ee7796c3cfa2a0f1a71db77b61558ac603043e078636e6316c03"
Oct 30 21:52:00.717591 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:52:00.717561    5032 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea94fdb68f46ee7796c3cfa2a0f1a71db77b61558ac603043e078636e6316c03\": container with ID starting with ea94fdb68f46ee7796c3cfa2a0f1a71db77b61558ac603043e078636e6316c03 not found: ID does not exist" containerID="ea94fdb68f46ee7796c3cfa2a0f1a71db77b61558ac603043e078636e6316c03"
Oct 30 21:52:00.717713 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:00.717599    5032 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea94fdb68f46ee7796c3cfa2a0f1a71db77b61558ac603043e078636e6316c03"} err="failed to get container status \"ea94fdb68f46ee7796c3cfa2a0f1a71db77b61558ac603043e078636e6316c03\": rpc error: code = NotFound desc = could not find container \"ea94fdb68f46ee7796c3cfa2a0f1a71db77b61558ac603043e078636e6316c03\": container with ID starting with ea94fdb68f46ee7796c3cfa2a0f1a71db77b61558ac603043e078636e6316c03 not found: ID does not exist"
Oct 30 21:52:00.731016 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:00.730987    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj"]
Oct 30 21:52:00.737162 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:00.737134    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-caa3dc/distinct-w5-caa3dc-kserve-74fd699b9c-h7bqj"]
Oct 30 21:52:01.347070 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:01.347039    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d0ff5023-fd3f-4734-94c6-76b666010adf" path="/var/lib/kubelet/pods/d0ff5023-fd3f-4734-94c6-76b666010adf/volumes"
Oct 30 21:52:02.892061 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:02.892022    5032 patch_prober.go:29] interesting pod/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:52:02.892476 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:02.892082    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf" podUID="2b720f88-2fd8-4c4d-a300-941b1a9db6e2" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:52:04.733893 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:04.733861    5032 generic.go:356] "Generic (PLEG): container finished" podID="aed33297-3e27-4d4a-a767-4191e8bc5127" containerID="996f7341f0007a4f636c0f27df8ba37ce893d6675da2f33bffc987c040b54633" exitCode=0
Oct 30 21:52:04.734264 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:04.733969    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv" event={"ID":"aed33297-3e27-4d4a-a767-4191e8bc5127","Type":"ContainerDied","Data":"996f7341f0007a4f636c0f27df8ba37ce893d6675da2f33bffc987c040b54633"}
Oct 30 21:52:04.737496 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:04.737467    5032 generic.go:356] "Generic (PLEG): container finished" podID="2b720f88-2fd8-4c4d-a300-941b1a9db6e2" containerID="dcd9676b4bf074fc246addfff9469b2223555476176e11d6aa22a488effc8ac7" exitCode=0
Oct 30 21:52:04.737600 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:04.737512    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf" event={"ID":"2b720f88-2fd8-4c4d-a300-941b1a9db6e2","Type":"ContainerDied","Data":"dcd9676b4bf074fc246addfff9469b2223555476176e11d6aa22a488effc8ac7"}
Oct 30 21:52:04.916111 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:04.916080    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv"
Oct 30 21:52:04.946455 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:04.946427    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz"
Oct 30 21:52:04.973954 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:04.973932    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf"
Oct 30 21:52:05.035725 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.035698    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aed33297-3e27-4d4a-a767-4191e8bc5127-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aed33297-3e27-4d4a-a767-4191e8bc5127-workload-certs\") pod \"aed33297-3e27-4d4a-a767-4191e8bc5127\" (UID: \"aed33297-3e27-4d4a-a767-4191e8bc5127\") "
Oct 30 21:52:05.035894 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.035739    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74-istio-envoy\") pod \"ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74\" (UID: \"ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74\") "
Oct 30 21:52:05.035894 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.035769    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74-istio-podinfo\") pod \"ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74\" (UID: \"ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74\") "
Oct 30 21:52:05.035894 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.035803    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2b720f88-2fd8-4c4d-a300-941b1a9db6e2-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b720f88-2fd8-4c4d-a300-941b1a9db6e2-istio-envoy\") pod \"2b720f88-2fd8-4c4d-a300-941b1a9db6e2\" (UID: \"2b720f88-2fd8-4c4d-a300-941b1a9db6e2\") "
Oct 30 21:52:05.035894 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.035834    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aed33297-3e27-4d4a-a767-4191e8bc5127-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aed33297-3e27-4d4a-a767-4191e8bc5127-istio-envoy\") pod \"aed33297-3e27-4d4a-a767-4191e8bc5127\" (UID: \"aed33297-3e27-4d4a-a767-4191e8bc5127\") "
Oct 30 21:52:05.035894 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.035873    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aed33297-3e27-4d4a-a767-4191e8bc5127-kube-api-access-njq55\" (UniqueName: \"kubernetes.io/projected/aed33297-3e27-4d4a-a767-4191e8bc5127-kube-api-access-njq55\") pod \"aed33297-3e27-4d4a-a767-4191e8bc5127\" (UID: \"aed33297-3e27-4d4a-a767-4191e8bc5127\") "
Oct 30 21:52:05.036184 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.035899    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74-istio-token\" (UniqueName: \"kubernetes.io/projected/ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74-istio-token\") pod \"ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74\" (UID: \"ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74\") "
Oct 30 21:52:05.036184 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.035959    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/aed33297-3e27-4d4a-a767-4191e8bc5127-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aed33297-3e27-4d4a-a767-4191e8bc5127-istiod-ca-cert\") pod \"aed33297-3e27-4d4a-a767-4191e8bc5127\" (UID: \"aed33297-3e27-4d4a-a767-4191e8bc5127\") "
Oct 30 21:52:05.036184 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.035991    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aed33297-3e27-4d4a-a767-4191e8bc5127-istio-token\" (UniqueName: \"kubernetes.io/projected/aed33297-3e27-4d4a-a767-4191e8bc5127-istio-token\") pod \"aed33297-3e27-4d4a-a767-4191e8bc5127\" (UID: \"aed33297-3e27-4d4a-a767-4191e8bc5127\") "
Oct 30 21:52:05.036184 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.036025    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aed33297-3e27-4d4a-a767-4191e8bc5127-workload-certs" pod "aed33297-3e27-4d4a-a767-4191e8bc5127" (UID: "aed33297-3e27-4d4a-a767-4191e8bc5127"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:05.036184 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.036030    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74-workload-socket\") pod \"ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74\" (UID: \"ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74\") "
Oct 30 21:52:05.036184 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.036083    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aed33297-3e27-4d4a-a767-4191e8bc5127-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aed33297-3e27-4d4a-a767-4191e8bc5127-workload-socket\") pod \"aed33297-3e27-4d4a-a767-4191e8bc5127\" (UID: \"aed33297-3e27-4d4a-a767-4191e8bc5127\") "
Oct 30 21:52:05.036184 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.036127    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74-kube-api-access-2pk7x\" (UniqueName: \"kubernetes.io/projected/ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74-kube-api-access-2pk7x\") pod \"ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74\" (UID: \"ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74\") "
Oct 30 21:52:05.036585 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.036244    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74-workload-socket" pod "ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74" (UID: "ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:05.036585 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.036330    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2b720f88-2fd8-4c4d-a300-941b1a9db6e2-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b720f88-2fd8-4c4d-a300-941b1a9db6e2-workload-certs\") pod \"2b720f88-2fd8-4c4d-a300-941b1a9db6e2\" (UID: \"2b720f88-2fd8-4c4d-a300-941b1a9db6e2\") "
Oct 30 21:52:05.036585 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.036411    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74-credential-socket\") pod \"ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74\" (UID: \"ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74\") "
Oct 30 21:52:05.036585 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.036455    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2b720f88-2fd8-4c4d-a300-941b1a9db6e2-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b720f88-2fd8-4c4d-a300-941b1a9db6e2-workload-socket\") pod \"2b720f88-2fd8-4c4d-a300-941b1a9db6e2\" (UID: \"2b720f88-2fd8-4c4d-a300-941b1a9db6e2\") "
Oct 30 21:52:05.036585 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.036452    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aed33297-3e27-4d4a-a767-4191e8bc5127-workload-socket" pod "aed33297-3e27-4d4a-a767-4191e8bc5127" (UID: "aed33297-3e27-4d4a-a767-4191e8bc5127"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:05.036585 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.036495    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/aed33297-3e27-4d4a-a767-4191e8bc5127-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aed33297-3e27-4d4a-a767-4191e8bc5127-istio-podinfo\") pod \"aed33297-3e27-4d4a-a767-4191e8bc5127\" (UID: \"aed33297-3e27-4d4a-a767-4191e8bc5127\") "
Oct 30 21:52:05.036585 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.036552    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2b720f88-2fd8-4c4d-a300-941b1a9db6e2-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b720f88-2fd8-4c4d-a300-941b1a9db6e2-credential-socket\") pod \"2b720f88-2fd8-4c4d-a300-941b1a9db6e2\" (UID: \"2b720f88-2fd8-4c4d-a300-941b1a9db6e2\") "
Oct 30 21:52:05.036950 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.036602    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2b720f88-2fd8-4c4d-a300-941b1a9db6e2-kube-api-access-vnl2w\" (UniqueName: \"kubernetes.io/projected/2b720f88-2fd8-4c4d-a300-941b1a9db6e2-kube-api-access-vnl2w\") pod \"2b720f88-2fd8-4c4d-a300-941b1a9db6e2\" (UID: \"2b720f88-2fd8-4c4d-a300-941b1a9db6e2\") "
Oct 30 21:52:05.036950 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.036632    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74-workload-certs\") pod \"ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74\" (UID: \"ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74\") "
Oct 30 21:52:05.036950 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.036662    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2b720f88-2fd8-4c4d-a300-941b1a9db6e2-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b720f88-2fd8-4c4d-a300-941b1a9db6e2-istio-data\") pod \"2b720f88-2fd8-4c4d-a300-941b1a9db6e2\" (UID: \"2b720f88-2fd8-4c4d-a300-941b1a9db6e2\") "
Oct 30 21:52:05.036950 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.036711    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74-istiod-ca-cert\") pod \"ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74\" (UID: \"ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74\") "
Oct 30 21:52:05.036950 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.036787    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aed33297-3e27-4d4a-a767-4191e8bc5127-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aed33297-3e27-4d4a-a767-4191e8bc5127-credential-socket\") pod \"aed33297-3e27-4d4a-a767-4191e8bc5127\" (UID: \"aed33297-3e27-4d4a-a767-4191e8bc5127\") "
Oct 30 21:52:05.036950 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.036872    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74-istio-data\") pod \"ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74\" (UID: \"ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74\") "
Oct 30 21:52:05.037259 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.036955    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2b720f88-2fd8-4c4d-a300-941b1a9db6e2-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b720f88-2fd8-4c4d-a300-941b1a9db6e2-istiod-ca-cert\") pod \"2b720f88-2fd8-4c4d-a300-941b1a9db6e2\" (UID: \"2b720f88-2fd8-4c4d-a300-941b1a9db6e2\") "
Oct 30 21:52:05.037259 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.036990    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2b720f88-2fd8-4c4d-a300-941b1a9db6e2-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b720f88-2fd8-4c4d-a300-941b1a9db6e2-istio-podinfo\") pod \"2b720f88-2fd8-4c4d-a300-941b1a9db6e2\" (UID: \"2b720f88-2fd8-4c4d-a300-941b1a9db6e2\") "
Oct 30 21:52:05.037259 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.037023    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aed33297-3e27-4d4a-a767-4191e8bc5127-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aed33297-3e27-4d4a-a767-4191e8bc5127-istio-data\") pod \"aed33297-3e27-4d4a-a767-4191e8bc5127\" (UID: \"aed33297-3e27-4d4a-a767-4191e8bc5127\") "
Oct 30 21:52:05.037259 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.037056    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2b720f88-2fd8-4c4d-a300-941b1a9db6e2-istio-token\" (UniqueName: \"kubernetes.io/projected/2b720f88-2fd8-4c4d-a300-941b1a9db6e2-istio-token\") pod \"2b720f88-2fd8-4c4d-a300-941b1a9db6e2\" (UID: \"2b720f88-2fd8-4c4d-a300-941b1a9db6e2\") "
Oct 30 21:52:05.038279 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.037499    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aed33297-3e27-4d4a-a767-4191e8bc5127-istiod-ca-cert" pod "aed33297-3e27-4d4a-a767-4191e8bc5127" (UID: "aed33297-3e27-4d4a-a767-4191e8bc5127"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:52:05.038279 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.037729    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74-workload-certs" pod "ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74" (UID: "ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:05.038279 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.037830    5032 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aed33297-3e27-4d4a-a767-4191e8bc5127-workload-socket\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:05.038279 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.037855    5032 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aed33297-3e27-4d4a-a767-4191e8bc5127-workload-certs\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:05.038279 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.037872    5032 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aed33297-3e27-4d4a-a767-4191e8bc5127-istiod-ca-cert\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:05.038279 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.037887    5032 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74-workload-socket\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:05.038279 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.037934    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b720f88-2fd8-4c4d-a300-941b1a9db6e2-workload-certs" pod "2b720f88-2fd8-4c4d-a300-941b1a9db6e2" (UID: "2b720f88-2fd8-4c4d-a300-941b1a9db6e2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:05.038279 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.038158    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b720f88-2fd8-4c4d-a300-941b1a9db6e2-credential-socket" pod "2b720f88-2fd8-4c4d-a300-941b1a9db6e2" (UID: "2b720f88-2fd8-4c4d-a300-941b1a9db6e2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:05.040660 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.038744    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b720f88-2fd8-4c4d-a300-941b1a9db6e2-istio-data" pod "2b720f88-2fd8-4c4d-a300-941b1a9db6e2" (UID: "2b720f88-2fd8-4c4d-a300-941b1a9db6e2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:05.040660 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.039049    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74-istiod-ca-cert" pod "ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74" (UID: "ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:52:05.040660 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.039282    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aed33297-3e27-4d4a-a767-4191e8bc5127-credential-socket" pod "aed33297-3e27-4d4a-a767-4191e8bc5127" (UID: "aed33297-3e27-4d4a-a767-4191e8bc5127"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:05.040660 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.039481    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74-istio-data" pod "ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74" (UID: "ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:05.040660 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.039825    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b720f88-2fd8-4c4d-a300-941b1a9db6e2-istiod-ca-cert" pod "2b720f88-2fd8-4c4d-a300-941b1a9db6e2" (UID: "2b720f88-2fd8-4c4d-a300-941b1a9db6e2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:52:05.040660 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.040323    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aed33297-3e27-4d4a-a767-4191e8bc5127-istio-data" pod "aed33297-3e27-4d4a-a767-4191e8bc5127" (UID: "aed33297-3e27-4d4a-a767-4191e8bc5127"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:05.040660 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.040440    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b720f88-2fd8-4c4d-a300-941b1a9db6e2-istio-token" pod "2b720f88-2fd8-4c4d-a300-941b1a9db6e2" (UID: "2b720f88-2fd8-4c4d-a300-941b1a9db6e2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:52:05.041555 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.040675    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74-credential-socket" pod "ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74" (UID: "ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:05.041555 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.040862    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b720f88-2fd8-4c4d-a300-941b1a9db6e2-workload-socket" pod "2b720f88-2fd8-4c4d-a300-941b1a9db6e2" (UID: "2b720f88-2fd8-4c4d-a300-941b1a9db6e2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:05.041555 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.040946    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74-istio-podinfo" pod "ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74" (UID: "ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:52:05.041555 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.041163    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aed33297-3e27-4d4a-a767-4191e8bc5127-kube-api-access-njq55" pod "aed33297-3e27-4d4a-a767-4191e8bc5127" (UID: "aed33297-3e27-4d4a-a767-4191e8bc5127"). InnerVolumeSpecName "kube-api-access-njq55". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:52:05.041555 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.041206    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aed33297-3e27-4d4a-a767-4191e8bc5127-istio-token" pod "aed33297-3e27-4d4a-a767-4191e8bc5127" (UID: "aed33297-3e27-4d4a-a767-4191e8bc5127"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:52:05.041555 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.041263    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b720f88-2fd8-4c4d-a300-941b1a9db6e2-kube-api-access-vnl2w" pod "2b720f88-2fd8-4c4d-a300-941b1a9db6e2" (UID: "2b720f88-2fd8-4c4d-a300-941b1a9db6e2"). InnerVolumeSpecName "kube-api-access-vnl2w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:52:05.041555 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.041320    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74-istio-envoy" pod "ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74" (UID: "ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:05.041555 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.041325    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74-istio-token" pod "ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74" (UID: "ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:52:05.041555 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.041449    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74-kube-api-access-2pk7x" pod "ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74" (UID: "ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74"). InnerVolumeSpecName "kube-api-access-2pk7x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:52:05.042556 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.041753    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/aed33297-3e27-4d4a-a767-4191e8bc5127-istio-podinfo" pod "aed33297-3e27-4d4a-a767-4191e8bc5127" (UID: "aed33297-3e27-4d4a-a767-4191e8bc5127"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:52:05.042556 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.042409    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2b720f88-2fd8-4c4d-a300-941b1a9db6e2-istio-podinfo" pod "2b720f88-2fd8-4c4d-a300-941b1a9db6e2" (UID: "2b720f88-2fd8-4c4d-a300-941b1a9db6e2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:52:05.042667 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.042624    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b720f88-2fd8-4c4d-a300-941b1a9db6e2-istio-envoy" pod "2b720f88-2fd8-4c4d-a300-941b1a9db6e2" (UID: "2b720f88-2fd8-4c4d-a300-941b1a9db6e2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:05.043113 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.043086    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aed33297-3e27-4d4a-a767-4191e8bc5127-istio-envoy" pod "aed33297-3e27-4d4a-a767-4191e8bc5127" (UID: "aed33297-3e27-4d4a-a767-4191e8bc5127"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:05.138863 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.138786    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vnl2w\" (UniqueName: \"kubernetes.io/projected/2b720f88-2fd8-4c4d-a300-941b1a9db6e2-kube-api-access-vnl2w\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:05.138863 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.138818    5032 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74-workload-certs\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:05.138863 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.138828    5032 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b720f88-2fd8-4c4d-a300-941b1a9db6e2-istio-data\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:05.138863 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.138836    5032 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74-istiod-ca-cert\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:05.138863 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.138845    5032 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aed33297-3e27-4d4a-a767-4191e8bc5127-credential-socket\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:05.138863 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.138853    5032 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74-istio-data\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:05.138863 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.138861    5032 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b720f88-2fd8-4c4d-a300-941b1a9db6e2-istiod-ca-cert\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:05.139174 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.138871    5032 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b720f88-2fd8-4c4d-a300-941b1a9db6e2-istio-podinfo\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:05.139174 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.138879    5032 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aed33297-3e27-4d4a-a767-4191e8bc5127-istio-data\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:05.139174 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.138887    5032 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b720f88-2fd8-4c4d-a300-941b1a9db6e2-istio-token\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:05.139174 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.138896    5032 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74-istio-envoy\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:05.139174 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.138903    5032 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74-istio-podinfo\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:05.139174 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.138914    5032 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b720f88-2fd8-4c4d-a300-941b1a9db6e2-istio-envoy\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:05.139174 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.138924    5032 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aed33297-3e27-4d4a-a767-4191e8bc5127-istio-envoy\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:05.139174 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.138935    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-njq55\" (UniqueName: \"kubernetes.io/projected/aed33297-3e27-4d4a-a767-4191e8bc5127-kube-api-access-njq55\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:05.139174 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.138943    5032 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74-istio-token\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:05.139174 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.138952    5032 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aed33297-3e27-4d4a-a767-4191e8bc5127-istio-token\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:05.139174 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.138960    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2pk7x\" (UniqueName: \"kubernetes.io/projected/ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74-kube-api-access-2pk7x\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:05.139174 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.138968    5032 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b720f88-2fd8-4c4d-a300-941b1a9db6e2-workload-certs\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:05.139174 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.138976    5032 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74-credential-socket\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:05.139174 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.138984    5032 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b720f88-2fd8-4c4d-a300-941b1a9db6e2-workload-socket\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:05.139174 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.138993    5032 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aed33297-3e27-4d4a-a767-4191e8bc5127-istio-podinfo\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:05.139174 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.139003    5032 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b720f88-2fd8-4c4d-a300-941b1a9db6e2-credential-socket\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:05.746535 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.746505    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv"
Oct 30 21:52:05.747022 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.746504    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv" event={"ID":"aed33297-3e27-4d4a-a767-4191e8bc5127","Type":"ContainerDied","Data":"09224aafbc69a125d0efe7f890ef8f01c13ecffe3b072db75c11066ffc7402cf"}
Oct 30 21:52:05.747022 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.746648    5032 scope.go:122] "RemoveContainer" containerID="996f7341f0007a4f636c0f27df8ba37ce893d6675da2f33bffc987c040b54633"
Oct 30 21:52:05.748124 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.748097    5032 generic.go:356] "Generic (PLEG): container finished" podID="ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74" containerID="eb74413af42ee9771451cc538cb1e339d53b7020e6d1fa2ef1baf2e878f9817b" exitCode=0
Oct 30 21:52:05.748305 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.748139    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz"
Oct 30 21:52:05.748305 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.748189    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz" event={"ID":"ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74","Type":"ContainerDied","Data":"eb74413af42ee9771451cc538cb1e339d53b7020e6d1fa2ef1baf2e878f9817b"}
Oct 30 21:52:05.748305 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.748242    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz" event={"ID":"ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74","Type":"ContainerDied","Data":"20bc2e525eb6ca86d1306ddee03586f0f76681d114d0f6401decfd69f90c7148"}
Oct 30 21:52:05.750176 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.750153    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf" event={"ID":"2b720f88-2fd8-4c4d-a300-941b1a9db6e2","Type":"ContainerDied","Data":"ff65bcc6977427c5ca3e2a446c7694e9dc79a1d484fdb0f82a88c54cb8be1c8c"}
Oct 30 21:52:05.750319 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.750162    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf"
Oct 30 21:52:05.759247 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.759224    5032 scope.go:122] "RemoveContainer" containerID="eb74413af42ee9771451cc538cb1e339d53b7020e6d1fa2ef1baf2e878f9817b"
Oct 30 21:52:05.769388 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.769354    5032 scope.go:122] "RemoveContainer" containerID="eb74413af42ee9771451cc538cb1e339d53b7020e6d1fa2ef1baf2e878f9817b"
Oct 30 21:52:05.769687 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:52:05.769660    5032 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb74413af42ee9771451cc538cb1e339d53b7020e6d1fa2ef1baf2e878f9817b\": container with ID starting with eb74413af42ee9771451cc538cb1e339d53b7020e6d1fa2ef1baf2e878f9817b not found: ID does not exist" containerID="eb74413af42ee9771451cc538cb1e339d53b7020e6d1fa2ef1baf2e878f9817b"
Oct 30 21:52:05.769767 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.769691    5032 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb74413af42ee9771451cc538cb1e339d53b7020e6d1fa2ef1baf2e878f9817b"} err="failed to get container status \"eb74413af42ee9771451cc538cb1e339d53b7020e6d1fa2ef1baf2e878f9817b\": rpc error: code = NotFound desc = could not find container \"eb74413af42ee9771451cc538cb1e339d53b7020e6d1fa2ef1baf2e878f9817b\": container with ID starting with eb74413af42ee9771451cc538cb1e339d53b7020e6d1fa2ef1baf2e878f9817b not found: ID does not exist"
Oct 30 21:52:05.769767 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.769716    5032 scope.go:122] "RemoveContainer" containerID="dcd9676b4bf074fc246addfff9469b2223555476176e11d6aa22a488effc8ac7"
Oct 30 21:52:05.779981 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.779955    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf"]
Oct 30 21:52:05.783990 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.783966    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-bwmkf"]
Oct 30 21:52:05.795925 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.795898    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz"]
Oct 30 21:52:05.808612 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.808583    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-hfvcz"]
Oct 30 21:52:05.828653 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.828618    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv"]
Oct 30 21:52:05.832722 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:05.832698    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-1d8034-openshift-default-74b55b95f9-nm8dv"]
Oct 30 21:52:06.076472 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:06.076440    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 21:52:07.366704 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:07.366669    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2b720f88-2fd8-4c4d-a300-941b1a9db6e2" path="/var/lib/kubelet/pods/2b720f88-2fd8-4c4d-a300-941b1a9db6e2/volumes"
Oct 30 21:52:07.367407 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:07.367385    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="aed33297-3e27-4d4a-a767-4191e8bc5127" path="/var/lib/kubelet/pods/aed33297-3e27-4d4a-a767-4191e8bc5127/volumes"
Oct 30 21:52:07.368040 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:07.368020    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74" path="/var/lib/kubelet/pods/ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74/volumes"
Oct 30 21:52:07.413289 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:07.413245    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w5-caa3dc-76db8c686c-pxdzh" podUID="467142c6-5675-4dd8-87a0-14e1f9fddc57" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:52:08.422218 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:08.422180    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-caa3dc-85fc67b867-gsjl2" podUID="28ed1f60-5004-4fb1-bb80-33cd040611f6" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:52:15.605851 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:15.605822    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-caa3dc-85fc67b867-gsjl2"
Oct 30 21:52:15.635459 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:15.635430    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w5-caa3dc-76db8c686c-pxdzh"
Oct 30 21:52:15.667150 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:15.667090    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/467142c6-5675-4dd8-87a0-14e1f9fddc57-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/467142c6-5675-4dd8-87a0-14e1f9fddc57-plugins-config-volume\") pod \"467142c6-5675-4dd8-87a0-14e1f9fddc57\" (UID: \"467142c6-5675-4dd8-87a0-14e1f9fddc57\") "
Oct 30 21:52:15.667344 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:15.667269    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/467142c6-5675-4dd8-87a0-14e1f9fddc57-kube-api-access-6cgfr\" (UniqueName: \"kubernetes.io/projected/467142c6-5675-4dd8-87a0-14e1f9fddc57-kube-api-access-6cgfr\") pod \"467142c6-5675-4dd8-87a0-14e1f9fddc57\" (UID: \"467142c6-5675-4dd8-87a0-14e1f9fddc57\") "
Oct 30 21:52:15.667983 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:15.667354    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/28ed1f60-5004-4fb1-bb80-33cd040611f6-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28ed1f60-5004-4fb1-bb80-33cd040611f6-plugins-config-volume\") pod \"28ed1f60-5004-4fb1-bb80-33cd040611f6\" (UID: \"28ed1f60-5004-4fb1-bb80-33cd040611f6\") "
Oct 30 21:52:15.667983 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:15.667415    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/28ed1f60-5004-4fb1-bb80-33cd040611f6-kube-api-access-x99kb\" (UniqueName: \"kubernetes.io/projected/28ed1f60-5004-4fb1-bb80-33cd040611f6-kube-api-access-x99kb\") pod \"28ed1f60-5004-4fb1-bb80-33cd040611f6\" (UID: \"28ed1f60-5004-4fb1-bb80-33cd040611f6\") "
Oct 30 21:52:15.667983 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:15.667571    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/467142c6-5675-4dd8-87a0-14e1f9fddc57-plugins-config-volume" pod "467142c6-5675-4dd8-87a0-14e1f9fddc57" (UID: "467142c6-5675-4dd8-87a0-14e1f9fddc57"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:52:15.668246 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:15.668111    5032 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/467142c6-5675-4dd8-87a0-14e1f9fddc57-plugins-config-volume\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:15.668309 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:15.668268    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28ed1f60-5004-4fb1-bb80-33cd040611f6-plugins-config-volume" pod "28ed1f60-5004-4fb1-bb80-33cd040611f6" (UID: "28ed1f60-5004-4fb1-bb80-33cd040611f6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:52:15.672827 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:15.672803    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/467142c6-5675-4dd8-87a0-14e1f9fddc57-kube-api-access-6cgfr" pod "467142c6-5675-4dd8-87a0-14e1f9fddc57" (UID: "467142c6-5675-4dd8-87a0-14e1f9fddc57"). InnerVolumeSpecName "kube-api-access-6cgfr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:52:15.672827 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:15.672820    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28ed1f60-5004-4fb1-bb80-33cd040611f6-kube-api-access-x99kb" pod "28ed1f60-5004-4fb1-bb80-33cd040611f6" (UID: "28ed1f60-5004-4fb1-bb80-33cd040611f6"). InnerVolumeSpecName "kube-api-access-x99kb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:52:15.769666 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:15.769628    5032 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28ed1f60-5004-4fb1-bb80-33cd040611f6-plugins-config-volume\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:15.769666 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:15.769666    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x99kb\" (UniqueName: \"kubernetes.io/projected/28ed1f60-5004-4fb1-bb80-33cd040611f6-kube-api-access-x99kb\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:15.769880 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:15.769684    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6cgfr\" (UniqueName: \"kubernetes.io/projected/467142c6-5675-4dd8-87a0-14e1f9fddc57-kube-api-access-6cgfr\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:15.816876 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:15.816842    5032 generic.go:356] "Generic (PLEG): container finished" podID="467142c6-5675-4dd8-87a0-14e1f9fddc57" containerID="b8c232edce4bac248fb110534e824aea7161314ba5b558c05d1e1a78d986e4de" exitCode=0
Oct 30 21:52:15.817123 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:15.816887    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w5-caa3dc-76db8c686c-pxdzh"
Oct 30 21:52:15.817123 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:15.816919    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w5-caa3dc-76db8c686c-pxdzh" event={"ID":"467142c6-5675-4dd8-87a0-14e1f9fddc57","Type":"ContainerDied","Data":"b8c232edce4bac248fb110534e824aea7161314ba5b558c05d1e1a78d986e4de"}
Oct 30 21:52:15.817123 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:15.816985    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w5-caa3dc-76db8c686c-pxdzh" event={"ID":"467142c6-5675-4dd8-87a0-14e1f9fddc57","Type":"ContainerDied","Data":"1e90032cb941b3d3473723c4c817b8d4018ea09786354f5a11861794cc9f14aa"}
Oct 30 21:52:15.817123 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:15.817019    5032 scope.go:122] "RemoveContainer" containerID="b8c232edce4bac248fb110534e824aea7161314ba5b558c05d1e1a78d986e4de"
Oct 30 21:52:15.818440 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:15.818412    5032 generic.go:356] "Generic (PLEG): container finished" podID="28ed1f60-5004-4fb1-bb80-33cd040611f6" containerID="525daf5fc5fd23a19e3d8a883a2bc17bc3d4a939d45c2f36b5490e6fb136c888" exitCode=0
Oct 30 21:52:15.818586 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:15.818443    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-caa3dc-85fc67b867-gsjl2"
Oct 30 21:52:15.818649 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:15.818615    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-caa3dc-85fc67b867-gsjl2" event={"ID":"28ed1f60-5004-4fb1-bb80-33cd040611f6","Type":"ContainerDied","Data":"525daf5fc5fd23a19e3d8a883a2bc17bc3d4a939d45c2f36b5490e6fb136c888"}
Oct 30 21:52:15.818703 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:15.818657    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-caa3dc-85fc67b867-gsjl2" event={"ID":"28ed1f60-5004-4fb1-bb80-33cd040611f6","Type":"ContainerDied","Data":"e9cd86444d42cda2a7f2113aaf4b869272b0f295d04de2799d245b1e993afdff"}
Oct 30 21:52:15.828059 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:15.828040    5032 scope.go:122] "RemoveContainer" containerID="b8c232edce4bac248fb110534e824aea7161314ba5b558c05d1e1a78d986e4de"
Oct 30 21:52:15.828456 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:52:15.828436    5032 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8c232edce4bac248fb110534e824aea7161314ba5b558c05d1e1a78d986e4de\": container with ID starting with b8c232edce4bac248fb110534e824aea7161314ba5b558c05d1e1a78d986e4de not found: ID does not exist" containerID="b8c232edce4bac248fb110534e824aea7161314ba5b558c05d1e1a78d986e4de"
Oct 30 21:52:15.828541 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:15.828465    5032 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8c232edce4bac248fb110534e824aea7161314ba5b558c05d1e1a78d986e4de"} err="failed to get container status \"b8c232edce4bac248fb110534e824aea7161314ba5b558c05d1e1a78d986e4de\": rpc error: code = NotFound desc = could not find container \"b8c232edce4bac248fb110534e824aea7161314ba5b558c05d1e1a78d986e4de\": container with ID starting with b8c232edce4bac248fb110534e824aea7161314ba5b558c05d1e1a78d986e4de not found: ID does not exist"
Oct 30 21:52:15.828541 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:15.828485    5032 scope.go:122] "RemoveContainer" containerID="525daf5fc5fd23a19e3d8a883a2bc17bc3d4a939d45c2f36b5490e6fb136c888"
Oct 30 21:52:15.840083 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:15.840059    5032 scope.go:122] "RemoveContainer" containerID="525daf5fc5fd23a19e3d8a883a2bc17bc3d4a939d45c2f36b5490e6fb136c888"
Oct 30 21:52:15.840626 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:52:15.840542    5032 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"525daf5fc5fd23a19e3d8a883a2bc17bc3d4a939d45c2f36b5490e6fb136c888\": container with ID starting with 525daf5fc5fd23a19e3d8a883a2bc17bc3d4a939d45c2f36b5490e6fb136c888 not found: ID does not exist" containerID="525daf5fc5fd23a19e3d8a883a2bc17bc3d4a939d45c2f36b5490e6fb136c888"
Oct 30 21:52:15.840626 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:15.840581    5032 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"525daf5fc5fd23a19e3d8a883a2bc17bc3d4a939d45c2f36b5490e6fb136c888"} err="failed to get container status \"525daf5fc5fd23a19e3d8a883a2bc17bc3d4a939d45c2f36b5490e6fb136c888\": rpc error: code = NotFound desc = could not find container \"525daf5fc5fd23a19e3d8a883a2bc17bc3d4a939d45c2f36b5490e6fb136c888\": container with ID starting with 525daf5fc5fd23a19e3d8a883a2bc17bc3d4a939d45c2f36b5490e6fb136c888 not found: ID does not exist"
Oct 30 21:52:15.858666 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:15.858634    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-caa3dc-76db8c686c-pxdzh"]
Oct 30 21:52:15.863467 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:15.863444    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-caa3dc-76db8c686c-pxdzh"]
Oct 30 21:52:15.879514 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:15.879492    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-caa3dc-85fc67b867-gsjl2"]
Oct 30 21:52:15.881549 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:15.881528    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-caa3dc-85fc67b867-gsjl2"]
Oct 30 21:52:16.037165 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:16.037131    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq"]
Oct 30 21:52:16.037356 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:16.037335    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq" podUID="968e86d5-7b2e-4f83-ad1d-68b62aadfa18" containerName="istio-proxy" containerID="cri-o://ed9efed975857e81ed94a5d09e635a296b0334d2a218ab09ffc137bb102c3abc" gracePeriod=30
Oct 30 21:52:16.544890 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:16.544853    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 21:52:17.345806 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:17.345776    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="28ed1f60-5004-4fb1-bb80-33cd040611f6" path="/var/lib/kubelet/pods/28ed1f60-5004-4fb1-bb80-33cd040611f6/volumes"
Oct 30 21:52:17.346291 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:17.346169    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="467142c6-5675-4dd8-87a0-14e1f9fddc57" path="/var/lib/kubelet/pods/467142c6-5675-4dd8-87a0-14e1f9fddc57/volumes"
Oct 30 21:52:18.344699 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:52:18.344664    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 21:52:18.741715 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:18.741676    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7bff7d9b4d-tzcgw"]
Oct 30 21:52:18.742175 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:18.741905    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-7bff7d9b4d-tzcgw" podUID="ca2836d0-0021-4c48-9b62-03e24eb03ac8" containerName="manager" containerID="cri-o://e6759070c57fb32641b40c1505a100acd50d78ada8ff82fadbdd1fc52da36871" gracePeriod=30
Oct 30 21:52:18.998667 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:18.998607    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7bff7d9b4d-tzcgw"
Oct 30 21:52:19.102639 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:19.102602    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ca2836d0-0021-4c48-9b62-03e24eb03ac8-metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca2836d0-0021-4c48-9b62-03e24eb03ac8-metrics-certs\") pod \"ca2836d0-0021-4c48-9b62-03e24eb03ac8\" (UID: \"ca2836d0-0021-4c48-9b62-03e24eb03ac8\") "
Oct 30 21:52:19.102844 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:19.102787    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ca2836d0-0021-4c48-9b62-03e24eb03ac8-webhook-cert\" (UniqueName: \"kubernetes.io/secret/ca2836d0-0021-4c48-9b62-03e24eb03ac8-webhook-cert\") pod \"ca2836d0-0021-4c48-9b62-03e24eb03ac8\" (UID: \"ca2836d0-0021-4c48-9b62-03e24eb03ac8\") "
Oct 30 21:52:19.102844 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:19.102834    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ca2836d0-0021-4c48-9b62-03e24eb03ac8-kube-api-access-9cr42\" (UniqueName: \"kubernetes.io/projected/ca2836d0-0021-4c48-9b62-03e24eb03ac8-kube-api-access-9cr42\") pod \"ca2836d0-0021-4c48-9b62-03e24eb03ac8\" (UID: \"ca2836d0-0021-4c48-9b62-03e24eb03ac8\") "
Oct 30 21:52:19.104890 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:19.104840    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca2836d0-0021-4c48-9b62-03e24eb03ac8-metrics-certs" pod "ca2836d0-0021-4c48-9b62-03e24eb03ac8" (UID: "ca2836d0-0021-4c48-9b62-03e24eb03ac8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:52:19.105532 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:19.105343    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca2836d0-0021-4c48-9b62-03e24eb03ac8-webhook-cert" pod "ca2836d0-0021-4c48-9b62-03e24eb03ac8" (UID: "ca2836d0-0021-4c48-9b62-03e24eb03ac8"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:52:19.105532 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:19.105499    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca2836d0-0021-4c48-9b62-03e24eb03ac8-kube-api-access-9cr42" pod "ca2836d0-0021-4c48-9b62-03e24eb03ac8" (UID: "ca2836d0-0021-4c48-9b62-03e24eb03ac8"). InnerVolumeSpecName "kube-api-access-9cr42". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:52:19.204793 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:19.204758    5032 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ca2836d0-0021-4c48-9b62-03e24eb03ac8-webhook-cert\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:19.204793 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:19.204791    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9cr42\" (UniqueName: \"kubernetes.io/projected/ca2836d0-0021-4c48-9b62-03e24eb03ac8-kube-api-access-9cr42\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:19.205003 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:19.204807    5032 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca2836d0-0021-4c48-9b62-03e24eb03ac8-metrics-certs\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:19.845740 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:19.845707    5032 generic.go:356] "Generic (PLEG): container finished" podID="ca2836d0-0021-4c48-9b62-03e24eb03ac8" containerID="e6759070c57fb32641b40c1505a100acd50d78ada8ff82fadbdd1fc52da36871" exitCode=0
Oct 30 21:52:19.846182 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:19.845756    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7bff7d9b4d-tzcgw"
Oct 30 21:52:19.846182 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:19.845791    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7bff7d9b4d-tzcgw" event={"ID":"ca2836d0-0021-4c48-9b62-03e24eb03ac8","Type":"ContainerDied","Data":"e6759070c57fb32641b40c1505a100acd50d78ada8ff82fadbdd1fc52da36871"}
Oct 30 21:52:19.846182 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:19.845844    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7bff7d9b4d-tzcgw" event={"ID":"ca2836d0-0021-4c48-9b62-03e24eb03ac8","Type":"ContainerDied","Data":"d3a4c5af9d24de6098fc6b43f71099c6062f11922e9cf51561ef638d6abc8c2b"}
Oct 30 21:52:19.846182 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:19.845867    5032 scope.go:122] "RemoveContainer" containerID="e6759070c57fb32641b40c1505a100acd50d78ada8ff82fadbdd1fc52da36871"
Oct 30 21:52:19.855704 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:19.855683    5032 scope.go:122] "RemoveContainer" containerID="e6759070c57fb32641b40c1505a100acd50d78ada8ff82fadbdd1fc52da36871"
Oct 30 21:52:19.855968 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:52:19.855947    5032 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6759070c57fb32641b40c1505a100acd50d78ada8ff82fadbdd1fc52da36871\": container with ID starting with e6759070c57fb32641b40c1505a100acd50d78ada8ff82fadbdd1fc52da36871 not found: ID does not exist" containerID="e6759070c57fb32641b40c1505a100acd50d78ada8ff82fadbdd1fc52da36871"
Oct 30 21:52:19.856022 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:19.855981    5032 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6759070c57fb32641b40c1505a100acd50d78ada8ff82fadbdd1fc52da36871"} err="failed to get container status \"e6759070c57fb32641b40c1505a100acd50d78ada8ff82fadbdd1fc52da36871\": rpc error: code = NotFound desc = could not find container \"e6759070c57fb32641b40c1505a100acd50d78ada8ff82fadbdd1fc52da36871\": container with ID starting with e6759070c57fb32641b40c1505a100acd50d78ada8ff82fadbdd1fc52da36871 not found: ID does not exist"
Oct 30 21:52:19.866554 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:19.866529    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7bff7d9b4d-tzcgw"]
Oct 30 21:52:19.870263 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:19.870239    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7bff7d9b4d-tzcgw"]
Oct 30 21:52:21.401751 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:21.401710    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ca2836d0-0021-4c48-9b62-03e24eb03ac8" path="/var/lib/kubelet/pods/ca2836d0-0021-4c48-9b62-03e24eb03ac8/volumes"
Oct 30 21:52:21.453941 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:21.453908    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq"
Oct 30 21:52:21.558498 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:21.558459    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/968e86d5-7b2e-4f83-ad1d-68b62aadfa18-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/968e86d5-7b2e-4f83-ad1d-68b62aadfa18-workload-certs\") pod \"968e86d5-7b2e-4f83-ad1d-68b62aadfa18\" (UID: \"968e86d5-7b2e-4f83-ad1d-68b62aadfa18\") "
Oct 30 21:52:21.689765 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:21.596481    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/968e86d5-7b2e-4f83-ad1d-68b62aadfa18-workload-certs" pod "968e86d5-7b2e-4f83-ad1d-68b62aadfa18" (UID: "968e86d5-7b2e-4f83-ad1d-68b62aadfa18"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:21.690428 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:21.690401    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/968e86d5-7b2e-4f83-ad1d-68b62aadfa18-kube-api-access-zbbz2\" (UniqueName: \"kubernetes.io/projected/968e86d5-7b2e-4f83-ad1d-68b62aadfa18-kube-api-access-zbbz2\") pod \"968e86d5-7b2e-4f83-ad1d-68b62aadfa18\" (UID: \"968e86d5-7b2e-4f83-ad1d-68b62aadfa18\") "
Oct 30 21:52:21.690664 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:21.690646    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/968e86d5-7b2e-4f83-ad1d-68b62aadfa18-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/968e86d5-7b2e-4f83-ad1d-68b62aadfa18-workload-socket\") pod \"968e86d5-7b2e-4f83-ad1d-68b62aadfa18\" (UID: \"968e86d5-7b2e-4f83-ad1d-68b62aadfa18\") "
Oct 30 21:52:21.690927 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:21.690812    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/968e86d5-7b2e-4f83-ad1d-68b62aadfa18-istio-token\" (UniqueName: \"kubernetes.io/projected/968e86d5-7b2e-4f83-ad1d-68b62aadfa18-istio-token\") pod \"968e86d5-7b2e-4f83-ad1d-68b62aadfa18\" (UID: \"968e86d5-7b2e-4f83-ad1d-68b62aadfa18\") "
Oct 30 21:52:21.691106 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:21.691089    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/968e86d5-7b2e-4f83-ad1d-68b62aadfa18-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/968e86d5-7b2e-4f83-ad1d-68b62aadfa18-istio-data\") pod \"968e86d5-7b2e-4f83-ad1d-68b62aadfa18\" (UID: \"968e86d5-7b2e-4f83-ad1d-68b62aadfa18\") "
Oct 30 21:52:21.691258 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:21.691243    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/968e86d5-7b2e-4f83-ad1d-68b62aadfa18-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/968e86d5-7b2e-4f83-ad1d-68b62aadfa18-istio-podinfo\") pod \"968e86d5-7b2e-4f83-ad1d-68b62aadfa18\" (UID: \"968e86d5-7b2e-4f83-ad1d-68b62aadfa18\") "
Oct 30 21:52:21.691419 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:21.691349    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/968e86d5-7b2e-4f83-ad1d-68b62aadfa18-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/968e86d5-7b2e-4f83-ad1d-68b62aadfa18-credential-socket\") pod \"968e86d5-7b2e-4f83-ad1d-68b62aadfa18\" (UID: \"968e86d5-7b2e-4f83-ad1d-68b62aadfa18\") "
Oct 30 21:52:21.691578 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:21.691561    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/968e86d5-7b2e-4f83-ad1d-68b62aadfa18-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/968e86d5-7b2e-4f83-ad1d-68b62aadfa18-istiod-ca-cert\") pod \"968e86d5-7b2e-4f83-ad1d-68b62aadfa18\" (UID: \"968e86d5-7b2e-4f83-ad1d-68b62aadfa18\") "
Oct 30 21:52:21.691694 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:21.691680    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/968e86d5-7b2e-4f83-ad1d-68b62aadfa18-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/968e86d5-7b2e-4f83-ad1d-68b62aadfa18-istio-envoy\") pod \"968e86d5-7b2e-4f83-ad1d-68b62aadfa18\" (UID: \"968e86d5-7b2e-4f83-ad1d-68b62aadfa18\") "
Oct 30 21:52:21.692248 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:21.692223    5032 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/968e86d5-7b2e-4f83-ad1d-68b62aadfa18-workload-certs\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:21.692820 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:21.692798    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/968e86d5-7b2e-4f83-ad1d-68b62aadfa18-istio-data" pod "968e86d5-7b2e-4f83-ad1d-68b62aadfa18" (UID: "968e86d5-7b2e-4f83-ad1d-68b62aadfa18"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:21.693118 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:21.693099    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/968e86d5-7b2e-4f83-ad1d-68b62aadfa18-workload-socket" pod "968e86d5-7b2e-4f83-ad1d-68b62aadfa18" (UID: "968e86d5-7b2e-4f83-ad1d-68b62aadfa18"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:21.694023 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:21.693761    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/968e86d5-7b2e-4f83-ad1d-68b62aadfa18-credential-socket" pod "968e86d5-7b2e-4f83-ad1d-68b62aadfa18" (UID: "968e86d5-7b2e-4f83-ad1d-68b62aadfa18"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:21.694526 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:21.694501    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/968e86d5-7b2e-4f83-ad1d-68b62aadfa18-istiod-ca-cert" pod "968e86d5-7b2e-4f83-ad1d-68b62aadfa18" (UID: "968e86d5-7b2e-4f83-ad1d-68b62aadfa18"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:52:21.774472 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:21.774395    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/968e86d5-7b2e-4f83-ad1d-68b62aadfa18-kube-api-access-zbbz2" pod "968e86d5-7b2e-4f83-ad1d-68b62aadfa18" (UID: "968e86d5-7b2e-4f83-ad1d-68b62aadfa18"). InnerVolumeSpecName "kube-api-access-zbbz2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:52:21.796642 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:21.796194    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zbbz2\" (UniqueName: \"kubernetes.io/projected/968e86d5-7b2e-4f83-ad1d-68b62aadfa18-kube-api-access-zbbz2\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:21.796642 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:21.796231    5032 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/968e86d5-7b2e-4f83-ad1d-68b62aadfa18-workload-socket\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:21.796642 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:21.796251    5032 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/968e86d5-7b2e-4f83-ad1d-68b62aadfa18-istio-data\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:21.796642 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:21.796266    5032 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/968e86d5-7b2e-4f83-ad1d-68b62aadfa18-credential-socket\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:21.796642 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:21.796281    5032 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/968e86d5-7b2e-4f83-ad1d-68b62aadfa18-istiod-ca-cert\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:21.804440 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:21.803884    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/968e86d5-7b2e-4f83-ad1d-68b62aadfa18-istio-token" pod "968e86d5-7b2e-4f83-ad1d-68b62aadfa18" (UID: "968e86d5-7b2e-4f83-ad1d-68b62aadfa18"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:52:21.804440 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:21.803980    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/968e86d5-7b2e-4f83-ad1d-68b62aadfa18-istio-podinfo" pod "968e86d5-7b2e-4f83-ad1d-68b62aadfa18" (UID: "968e86d5-7b2e-4f83-ad1d-68b62aadfa18"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:52:21.805546 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:21.805489    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/968e86d5-7b2e-4f83-ad1d-68b62aadfa18-istio-envoy" pod "968e86d5-7b2e-4f83-ad1d-68b62aadfa18" (UID: "968e86d5-7b2e-4f83-ad1d-68b62aadfa18"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:21.900255 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:21.900217    5032 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/968e86d5-7b2e-4f83-ad1d-68b62aadfa18-istio-token\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:21.900558 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:21.900540    5032 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/968e86d5-7b2e-4f83-ad1d-68b62aadfa18-istio-podinfo\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:21.900667 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:21.900654    5032 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/968e86d5-7b2e-4f83-ad1d-68b62aadfa18-istio-envoy\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:21.916408 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:21.909571    5032 generic.go:356] "Generic (PLEG): container finished" podID="968e86d5-7b2e-4f83-ad1d-68b62aadfa18" containerID="ed9efed975857e81ed94a5d09e635a296b0334d2a218ab09ffc137bb102c3abc" exitCode=0
Oct 30 21:52:21.916408 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:21.909707    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq" event={"ID":"968e86d5-7b2e-4f83-ad1d-68b62aadfa18","Type":"ContainerDied","Data":"ed9efed975857e81ed94a5d09e635a296b0334d2a218ab09ffc137bb102c3abc"}
Oct 30 21:52:21.916408 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:21.909760    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq" event={"ID":"968e86d5-7b2e-4f83-ad1d-68b62aadfa18","Type":"ContainerDied","Data":"5d4d902a7f1deb25319533676bbfec2625aabaece20e66d678bddfad038e8422"}
Oct 30 21:52:21.916408 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:21.909792    5032 scope.go:122] "RemoveContainer" containerID="ed9efed975857e81ed94a5d09e635a296b0334d2a218ab09ffc137bb102c3abc"
Oct 30 21:52:21.916408 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:21.909959    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq"
Oct 30 21:52:22.047002 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:22.046961    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq"]
Oct 30 21:52:22.095228 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:22.095124    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-caa3dc-openshift-default-69ff984c7c-gz9pq"]
Oct 30 21:52:22.100545 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:22.100394    5032 scope.go:122] "RemoveContainer" containerID="ed9efed975857e81ed94a5d09e635a296b0334d2a218ab09ffc137bb102c3abc"
Oct 30 21:52:22.101540 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:52:22.101429    5032 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed9efed975857e81ed94a5d09e635a296b0334d2a218ab09ffc137bb102c3abc\": container with ID starting with ed9efed975857e81ed94a5d09e635a296b0334d2a218ab09ffc137bb102c3abc not found: ID does not exist" containerID="ed9efed975857e81ed94a5d09e635a296b0334d2a218ab09ffc137bb102c3abc"
Oct 30 21:52:22.101540 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:22.101473    5032 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed9efed975857e81ed94a5d09e635a296b0334d2a218ab09ffc137bb102c3abc"} err="failed to get container status \"ed9efed975857e81ed94a5d09e635a296b0334d2a218ab09ffc137bb102c3abc\": rpc error: code = NotFound desc = could not find container \"ed9efed975857e81ed94a5d09e635a296b0334d2a218ab09ffc137bb102c3abc\": container with ID starting with ed9efed975857e81ed94a5d09e635a296b0334d2a218ab09ffc137bb102c3abc not found: ID does not exist"
Oct 30 21:52:23.351620 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:52:23.351582    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="968e86d5-7b2e-4f83-ad1d-68b62aadfa18" path="/var/lib/kubelet/pods/968e86d5-7b2e-4f83-ad1d-68b62aadfa18/volumes"
Oct 30 21:52:48.344997 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:52:48.344917    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 21:53:10.040030 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:53:10.040004    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-sklxd_67292bdc-7fe1-449d-af75-2baa90c16ea9/console-operator/1.log"
Oct 30 21:53:10.189970 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:53:10.189881    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-sklxd_67292bdc-7fe1-449d-af75-2baa90c16ea9/console-operator/1.log"
Oct 30 21:53:18.344802 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:53:18.344765    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 21:53:29.522431 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:53:29.522391    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 21:53:31.953407 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:53:31.952521    5032 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-mxsln container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:53:31.953407 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:53:31.952628    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mxsln" podUID="f77afa6c-51f5-4bc2-b10e-89101bd067c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:53:32.020523 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:53:32.020474    5032 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-c46gq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:53:32.020730 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:53:32.020566    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-c46gq" podUID="e3b353c1-3812-44aa-b77d-d7ddfc3b7057" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:53:32.088896 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:53:32.088523    5032 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-7s44p container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:53:32.088896 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:53:32.088585    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-7s44p" podUID="457cd47c-9543-4d82-875f-f1e45d6d5b70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:53:45.012306 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:53:45.012273    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 21:53:48.402158 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:53:48.402119    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 21:53:49.840625 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:53:49.840591    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 21:54:03.827401 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:54:03.827347    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 21:54:16.956603 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:54:16.956561    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 21:54:18.406912 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:54:18.406836    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 21:54:41.636080 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:54:41.636044    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 21:54:48.344936 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:54:48.344891    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 21:54:51.639000 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:54:51.638957    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 21:54:56.809560 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:54:56.809524    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 21:55:12.615509 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:55:12.615471    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 21:55:16.529384 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:55:16.529293    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 21:55:16.952475 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:55:16.951484    5032 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-mxsln container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:55:16.952475 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:55:16.951552    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mxsln" podUID="f77afa6c-51f5-4bc2-b10e-89101bd067c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:17.024561 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:55:17.024522    5032 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-c46gq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:55:17.024906 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:55:17.024875    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-c46gq" podUID="e3b353c1-3812-44aa-b77d-d7ddfc3b7057" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:17.091580 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:55:17.091527    5032 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-7s44p container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:55:17.091911 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:55:17.091876    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-7s44p" podUID="457cd47c-9543-4d82-875f-f1e45d6d5b70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:18.344801 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:55:18.344763    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 21:55:18.791748 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:55:18.791715    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-54h7q"]
Oct 30 21:55:18.791935 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:55:18.791917    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-54h7q" podUID="56d64255-3745-4df3-b9f3-9a15db845698" containerName="discovery" containerID="cri-o://d4d452f3a4fcedff664a52ee2e3b2fe1fd9311fa6b4219d56cc165516f404f94" gracePeriod=30
Oct 30 21:55:19.067195 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:55:19.067170    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-54h7q"
Oct 30 21:55:19.086537 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:55:19.086508    5032 generic.go:356] "Generic (PLEG): container finished" podID="56d64255-3745-4df3-b9f3-9a15db845698" containerID="d4d452f3a4fcedff664a52ee2e3b2fe1fd9311fa6b4219d56cc165516f404f94" exitCode=0
Oct 30 21:55:19.086672 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:55:19.086558    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-54h7q"
Oct 30 21:55:19.086672 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:55:19.086587    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-54h7q" event={"ID":"56d64255-3745-4df3-b9f3-9a15db845698","Type":"ContainerDied","Data":"d4d452f3a4fcedff664a52ee2e3b2fe1fd9311fa6b4219d56cc165516f404f94"}
Oct 30 21:55:19.086672 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:55:19.086636    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-54h7q" event={"ID":"56d64255-3745-4df3-b9f3-9a15db845698","Type":"ContainerDied","Data":"56ba11ead5c7fa1d91d7b73f756fa69b529233047e3ff4cda084b3ad3a63bbab"}
Oct 30 21:55:19.086672 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:55:19.086656    5032 scope.go:122] "RemoveContainer" containerID="d4d452f3a4fcedff664a52ee2e3b2fe1fd9311fa6b4219d56cc165516f404f94"
Oct 30 21:55:19.098475 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:55:19.098449    5032 scope.go:122] "RemoveContainer" containerID="d4d452f3a4fcedff664a52ee2e3b2fe1fd9311fa6b4219d56cc165516f404f94"
Oct 30 21:55:19.098828 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:55:19.098806    5032 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4d452f3a4fcedff664a52ee2e3b2fe1fd9311fa6b4219d56cc165516f404f94\": container with ID starting with d4d452f3a4fcedff664a52ee2e3b2fe1fd9311fa6b4219d56cc165516f404f94 not found: ID does not exist" containerID="d4d452f3a4fcedff664a52ee2e3b2fe1fd9311fa6b4219d56cc165516f404f94"
Oct 30 21:55:19.098911 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:55:19.098842    5032 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4d452f3a4fcedff664a52ee2e3b2fe1fd9311fa6b4219d56cc165516f404f94"} err="failed to get container status \"d4d452f3a4fcedff664a52ee2e3b2fe1fd9311fa6b4219d56cc165516f404f94\": rpc error: code = NotFound desc = could not find container \"d4d452f3a4fcedff664a52ee2e3b2fe1fd9311fa6b4219d56cc165516f404f94\": container with ID starting with d4d452f3a4fcedff664a52ee2e3b2fe1fd9311fa6b4219d56cc165516f404f94 not found: ID does not exist"
Oct 30 21:55:19.109583 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:55:19.109560    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/56d64255-3745-4df3-b9f3-9a15db845698-istio-token\" (UniqueName: \"kubernetes.io/projected/56d64255-3745-4df3-b9f3-9a15db845698-istio-token\") pod \"56d64255-3745-4df3-b9f3-9a15db845698\" (UID: \"56d64255-3745-4df3-b9f3-9a15db845698\") "
Oct 30 21:55:19.109773 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:55:19.109627    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/56d64255-3745-4df3-b9f3-9a15db845698-cacerts\" (UniqueName: \"kubernetes.io/secret/56d64255-3745-4df3-b9f3-9a15db845698-cacerts\") pod \"56d64255-3745-4df3-b9f3-9a15db845698\" (UID: \"56d64255-3745-4df3-b9f3-9a15db845698\") "
Oct 30 21:55:19.109773 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:55:19.109717    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/56d64255-3745-4df3-b9f3-9a15db845698-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/56d64255-3745-4df3-b9f3-9a15db845698-istio-csr-dns-cert\") pod \"56d64255-3745-4df3-b9f3-9a15db845698\" (UID: \"56d64255-3745-4df3-b9f3-9a15db845698\") "
Oct 30 21:55:19.109773 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:55:19.109754    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/56d64255-3745-4df3-b9f3-9a15db845698-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/56d64255-3745-4df3-b9f3-9a15db845698-istio-kubeconfig\") pod \"56d64255-3745-4df3-b9f3-9a15db845698\" (UID: \"56d64255-3745-4df3-b9f3-9a15db845698\") "
Oct 30 21:55:19.109988 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:55:19.109798    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/56d64255-3745-4df3-b9f3-9a15db845698-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/56d64255-3745-4df3-b9f3-9a15db845698-istio-csr-ca-configmap\") pod \"56d64255-3745-4df3-b9f3-9a15db845698\" (UID: \"56d64255-3745-4df3-b9f3-9a15db845698\") "
Oct 30 21:55:19.109988 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:55:19.109918    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/56d64255-3745-4df3-b9f3-9a15db845698-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/56d64255-3745-4df3-b9f3-9a15db845698-local-certs\") pod \"56d64255-3745-4df3-b9f3-9a15db845698\" (UID: \"56d64255-3745-4df3-b9f3-9a15db845698\") "
Oct 30 21:55:19.109988 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:55:19.109981    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/56d64255-3745-4df3-b9f3-9a15db845698-kube-api-access-ntqw5\" (UniqueName: \"kubernetes.io/projected/56d64255-3745-4df3-b9f3-9a15db845698-kube-api-access-ntqw5\") pod \"56d64255-3745-4df3-b9f3-9a15db845698\" (UID: \"56d64255-3745-4df3-b9f3-9a15db845698\") "
Oct 30 21:55:19.110407 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:55:19.110287    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56d64255-3745-4df3-b9f3-9a15db845698-istio-csr-ca-configmap" pod "56d64255-3745-4df3-b9f3-9a15db845698" (UID: "56d64255-3745-4df3-b9f3-9a15db845698"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:55:19.110489 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:55:19.110469    5032 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/56d64255-3745-4df3-b9f3-9a15db845698-istio-csr-ca-configmap\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:19.112003 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:55:19.111981    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56d64255-3745-4df3-b9f3-9a15db845698-istio-token" pod "56d64255-3745-4df3-b9f3-9a15db845698" (UID: "56d64255-3745-4df3-b9f3-9a15db845698"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:55:19.112316 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:55:19.112293    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/56d64255-3745-4df3-b9f3-9a15db845698-cacerts" pod "56d64255-3745-4df3-b9f3-9a15db845698" (UID: "56d64255-3745-4df3-b9f3-9a15db845698"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:55:19.112638 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:55:19.112602    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56d64255-3745-4df3-b9f3-9a15db845698-kube-api-access-ntqw5" pod "56d64255-3745-4df3-b9f3-9a15db845698" (UID: "56d64255-3745-4df3-b9f3-9a15db845698"). InnerVolumeSpecName "kube-api-access-ntqw5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:55:19.112638 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:55:19.112620    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/56d64255-3745-4df3-b9f3-9a15db845698-istio-csr-dns-cert" pod "56d64255-3745-4df3-b9f3-9a15db845698" (UID: "56d64255-3745-4df3-b9f3-9a15db845698"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:55:19.112817 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:55:19.112790    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/56d64255-3745-4df3-b9f3-9a15db845698-istio-kubeconfig" pod "56d64255-3745-4df3-b9f3-9a15db845698" (UID: "56d64255-3745-4df3-b9f3-9a15db845698"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:55:19.112893 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:55:19.112875    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56d64255-3745-4df3-b9f3-9a15db845698-local-certs" pod "56d64255-3745-4df3-b9f3-9a15db845698" (UID: "56d64255-3745-4df3-b9f3-9a15db845698"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:55:19.211036 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:55:19.211005    5032 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/56d64255-3745-4df3-b9f3-9a15db845698-local-certs\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:19.211036 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:55:19.211033    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ntqw5\" (UniqueName: \"kubernetes.io/projected/56d64255-3745-4df3-b9f3-9a15db845698-kube-api-access-ntqw5\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:19.211036 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:55:19.211043    5032 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56d64255-3745-4df3-b9f3-9a15db845698-istio-token\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:19.211343 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:55:19.211052    5032 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/56d64255-3745-4df3-b9f3-9a15db845698-cacerts\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:19.211343 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:55:19.211061    5032 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/56d64255-3745-4df3-b9f3-9a15db845698-istio-csr-dns-cert\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:19.211343 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:55:19.211069    5032 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/56d64255-3745-4df3-b9f3-9a15db845698-istio-kubeconfig\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:19.404930 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:55:19.404894    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-54h7q"]
Oct 30 21:55:19.409173 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:55:19.409146    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-54h7q"]
Oct 30 21:55:21.346194 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:55:21.346156    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="56d64255-3745-4df3-b9f3-9a15db845698" path="/var/lib/kubelet/pods/56d64255-3745-4df3-b9f3-9a15db845698/volumes"
Oct 30 21:55:31.096062 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:55:31.096022    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 21:55:36.227387 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:55:36.227005    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 21:55:48.347915 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:55:48.347822    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 21:55:49.956707 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:55:49.956669    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 21:55:53.841264 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:55:53.841224    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 21:56:07.855378 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:56:07.855323    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 21:56:18.344947 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:56:18.344906    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 21:56:18.553493 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:56:18.553459    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 21:56:23.017382 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:56:23.017320    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 21:56:33.893869 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:56:33.893824    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-v4k4f"]
Oct 30 21:56:33.894391 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:56:33.894097    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-v4k4f" podUID="9b50efc9-2123-439d-9a07-e027fe6a6bfa" containerName="discovery" containerID="cri-o://3dd3fd2a5d100e6265c56eb20ed982a2abe7964a65c3d4b0485f87a5c891010a" gracePeriod=30
Oct 30 21:56:34.165023 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:56:34.164991    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-v4k4f"
Oct 30 21:56:34.174856 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:56:34.174831    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9b50efc9-2123-439d-9a07-e027fe6a6bfa-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b50efc9-2123-439d-9a07-e027fe6a6bfa-local-certs\") pod \"9b50efc9-2123-439d-9a07-e027fe6a6bfa\" (UID: \"9b50efc9-2123-439d-9a07-e027fe6a6bfa\") "
Oct 30 21:56:34.174986 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:56:34.174889    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9b50efc9-2123-439d-9a07-e027fe6a6bfa-kube-api-access-84xzr\" (UniqueName: \"kubernetes.io/projected/9b50efc9-2123-439d-9a07-e027fe6a6bfa-kube-api-access-84xzr\") pod \"9b50efc9-2123-439d-9a07-e027fe6a6bfa\" (UID: \"9b50efc9-2123-439d-9a07-e027fe6a6bfa\") "
Oct 30 21:56:34.174986 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:56:34.174924    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9b50efc9-2123-439d-9a07-e027fe6a6bfa-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9b50efc9-2123-439d-9a07-e027fe6a6bfa-istio-csr-dns-cert\") pod \"9b50efc9-2123-439d-9a07-e027fe6a6bfa\" (UID: \"9b50efc9-2123-439d-9a07-e027fe6a6bfa\") "
Oct 30 21:56:34.174986 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:56:34.174953    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9b50efc9-2123-439d-9a07-e027fe6a6bfa-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9b50efc9-2123-439d-9a07-e027fe6a6bfa-istio-kubeconfig\") pod \"9b50efc9-2123-439d-9a07-e027fe6a6bfa\" (UID: \"9b50efc9-2123-439d-9a07-e027fe6a6bfa\") "
Oct 30 21:56:34.175091 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:56:34.174999    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9b50efc9-2123-439d-9a07-e027fe6a6bfa-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9b50efc9-2123-439d-9a07-e027fe6a6bfa-istio-csr-ca-configmap\") pod \"9b50efc9-2123-439d-9a07-e027fe6a6bfa\" (UID: \"9b50efc9-2123-439d-9a07-e027fe6a6bfa\") "
Oct 30 21:56:34.175129 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:56:34.175105    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9b50efc9-2123-439d-9a07-e027fe6a6bfa-cacerts\" (UniqueName: \"kubernetes.io/secret/9b50efc9-2123-439d-9a07-e027fe6a6bfa-cacerts\") pod \"9b50efc9-2123-439d-9a07-e027fe6a6bfa\" (UID: \"9b50efc9-2123-439d-9a07-e027fe6a6bfa\") "
Oct 30 21:56:34.175233 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:56:34.175172    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9b50efc9-2123-439d-9a07-e027fe6a6bfa-istio-token\" (UniqueName: \"kubernetes.io/projected/9b50efc9-2123-439d-9a07-e027fe6a6bfa-istio-token\") pod \"9b50efc9-2123-439d-9a07-e027fe6a6bfa\" (UID: \"9b50efc9-2123-439d-9a07-e027fe6a6bfa\") "
Oct 30 21:56:34.175453 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:56:34.175413    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b50efc9-2123-439d-9a07-e027fe6a6bfa-istio-csr-ca-configmap" pod "9b50efc9-2123-439d-9a07-e027fe6a6bfa" (UID: "9b50efc9-2123-439d-9a07-e027fe6a6bfa"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:56:34.175644 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:56:34.175614    5032 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9b50efc9-2123-439d-9a07-e027fe6a6bfa-istio-csr-ca-configmap\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:56:34.177885 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:56:34.177857    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b50efc9-2123-439d-9a07-e027fe6a6bfa-istio-kubeconfig" pod "9b50efc9-2123-439d-9a07-e027fe6a6bfa" (UID: "9b50efc9-2123-439d-9a07-e027fe6a6bfa"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:56:34.177996 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:56:34.177900    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b50efc9-2123-439d-9a07-e027fe6a6bfa-istio-csr-dns-cert" pod "9b50efc9-2123-439d-9a07-e027fe6a6bfa" (UID: "9b50efc9-2123-439d-9a07-e027fe6a6bfa"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:56:34.178069 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:56:34.178050    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b50efc9-2123-439d-9a07-e027fe6a6bfa-kube-api-access-84xzr" pod "9b50efc9-2123-439d-9a07-e027fe6a6bfa" (UID: "9b50efc9-2123-439d-9a07-e027fe6a6bfa"). InnerVolumeSpecName "kube-api-access-84xzr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:56:34.178109 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:56:34.178081    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b50efc9-2123-439d-9a07-e027fe6a6bfa-istio-token" pod "9b50efc9-2123-439d-9a07-e027fe6a6bfa" (UID: "9b50efc9-2123-439d-9a07-e027fe6a6bfa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:56:34.181464 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:56:34.181434    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b50efc9-2123-439d-9a07-e027fe6a6bfa-cacerts" pod "9b50efc9-2123-439d-9a07-e027fe6a6bfa" (UID: "9b50efc9-2123-439d-9a07-e027fe6a6bfa"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:56:34.182498 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:56:34.182477    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b50efc9-2123-439d-9a07-e027fe6a6bfa-local-certs" pod "9b50efc9-2123-439d-9a07-e027fe6a6bfa" (UID: "9b50efc9-2123-439d-9a07-e027fe6a6bfa"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:56:34.276355 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:56:34.276319    5032 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9b50efc9-2123-439d-9a07-e027fe6a6bfa-cacerts\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:56:34.276355 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:56:34.276349    5032 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b50efc9-2123-439d-9a07-e027fe6a6bfa-istio-token\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:56:34.276355 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:56:34.276385    5032 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b50efc9-2123-439d-9a07-e027fe6a6bfa-local-certs\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:56:34.276611 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:56:34.276396    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-84xzr\" (UniqueName: \"kubernetes.io/projected/9b50efc9-2123-439d-9a07-e027fe6a6bfa-kube-api-access-84xzr\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:56:34.276611 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:56:34.276406    5032 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9b50efc9-2123-439d-9a07-e027fe6a6bfa-istio-csr-dns-cert\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:56:34.276611 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:56:34.276415    5032 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9b50efc9-2123-439d-9a07-e027fe6a6bfa-istio-kubeconfig\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:56:34.557714 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:56:34.557680    5032 generic.go:356] "Generic (PLEG): container finished" podID="9b50efc9-2123-439d-9a07-e027fe6a6bfa" containerID="3dd3fd2a5d100e6265c56eb20ed982a2abe7964a65c3d4b0485f87a5c891010a" exitCode=0
Oct 30 21:56:34.557894 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:56:34.557733    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-v4k4f"
Oct 30 21:56:34.557894 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:56:34.557767    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-v4k4f" event={"ID":"9b50efc9-2123-439d-9a07-e027fe6a6bfa","Type":"ContainerDied","Data":"3dd3fd2a5d100e6265c56eb20ed982a2abe7964a65c3d4b0485f87a5c891010a"}
Oct 30 21:56:34.557894 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:56:34.557822    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-v4k4f" event={"ID":"9b50efc9-2123-439d-9a07-e027fe6a6bfa","Type":"ContainerDied","Data":"c349f7c92ba9f1ba8b52ccd87eb229b978549af068812237b4227198a211468a"}
Oct 30 21:56:34.557894 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:56:34.557847    5032 scope.go:122] "RemoveContainer" containerID="3dd3fd2a5d100e6265c56eb20ed982a2abe7964a65c3d4b0485f87a5c891010a"
Oct 30 21:56:34.569503 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:56:34.569485    5032 scope.go:122] "RemoveContainer" containerID="3dd3fd2a5d100e6265c56eb20ed982a2abe7964a65c3d4b0485f87a5c891010a"
Oct 30 21:56:34.569753 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:56:34.569734    5032 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3dd3fd2a5d100e6265c56eb20ed982a2abe7964a65c3d4b0485f87a5c891010a\": container with ID starting with 3dd3fd2a5d100e6265c56eb20ed982a2abe7964a65c3d4b0485f87a5c891010a not found: ID does not exist" containerID="3dd3fd2a5d100e6265c56eb20ed982a2abe7964a65c3d4b0485f87a5c891010a"
Oct 30 21:56:34.569797 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:56:34.569762    5032 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3dd3fd2a5d100e6265c56eb20ed982a2abe7964a65c3d4b0485f87a5c891010a"} err="failed to get container status \"3dd3fd2a5d100e6265c56eb20ed982a2abe7964a65c3d4b0485f87a5c891010a\": rpc error: code = NotFound desc = could not find container \"3dd3fd2a5d100e6265c56eb20ed982a2abe7964a65c3d4b0485f87a5c891010a\": container with ID starting with 3dd3fd2a5d100e6265c56eb20ed982a2abe7964a65c3d4b0485f87a5c891010a not found: ID does not exist"
Oct 30 21:56:34.581329 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:56:34.581298    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-v4k4f"]
Oct 30 21:56:34.584130 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:56:34.584108    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-v4k4f"]
Oct 30 21:56:35.345759 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:56:35.345726    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9b50efc9-2123-439d-9a07-e027fe6a6bfa" path="/var/lib/kubelet/pods/9b50efc9-2123-439d-9a07-e027fe6a6bfa/volumes"
Oct 30 21:56:37.336397 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:56:37.336347    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 21:56:41.354250 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:56:41.354216    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 21:56:47.227092 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:56:47.227054    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 21:56:48.347700 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:56:48.347663    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 21:57:05.641168 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:57:05.641130    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 21:57:15.865137 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:57:15.865096    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 21:57:18.343529 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:57:18.343453    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 21:57:48.344243 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:57:48.344204    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 21:58:10.074709 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:58:10.074681    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-sklxd_67292bdc-7fe1-449d-af75-2baa90c16ea9/console-operator/1.log"
Oct 30 21:58:10.222967 ip-10-0-129-195 kubenswrapper[5032]: I1030 21:58:10.222941    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-sklxd_67292bdc-7fe1-449d-af75-2baa90c16ea9/console-operator/1.log"
Oct 30 21:58:18.344440 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:58:18.344397    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 21:58:48.343940 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:58:48.343855    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 21:59:18.345012 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:59:18.344977    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 21:59:48.344243 ip-10-0-129-195 kubenswrapper[5032]: E1030 21:59:48.344206    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 22:00:18.344687 ip-10-0-129-195 kubenswrapper[5032]: E1030 22:00:18.344611    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 22:00:48.344296 ip-10-0-129-195 kubenswrapper[5032]: E1030 22:00:48.344260    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 22:01:18.345102 ip-10-0-129-195 kubenswrapper[5032]: E1030 22:01:18.345061    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 22:01:48.344290 ip-10-0-129-195 kubenswrapper[5032]: E1030 22:01:48.344213    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 22:02:18.344720 ip-10-0-129-195 kubenswrapper[5032]: E1030 22:02:18.344672    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 22:02:32.122418 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:32.122383    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 22:02:36.049442 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:36.049406    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 22:02:44.749405 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.749351    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-fa15d4-75dbfb6d49-jrq42"]
Oct 30 22:02:44.752385 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.752344    5032 cpu_manager.go:414] "containerMap: removing container" podUID="6b99e2de-e8a0-4566-b14c-dd627c74edf3" containerName="istio-proxy"
Oct 30 22:02:44.752565 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.752551    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6b99e2de-e8a0-4566-b14c-dd627c74edf3" containerName="istio-proxy"
Oct 30 22:02:44.752657 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.752648    5032 cpu_manager.go:414] "containerMap: removing container" podUID="86937e06-85cc-48d5-9935-9ae922c3518c" containerName="istio-proxy"
Oct 30 22:02:44.752737 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.752718    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="86937e06-85cc-48d5-9935-9ae922c3518c" containerName="istio-proxy"
Oct 30 22:02:44.752737 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.752741    5032 cpu_manager.go:414] "containerMap: removing container" podUID="fd9a1e32-2b8b-4f2c-bcfc-5b7390d9a4a2" containerName="maas-api"
Oct 30 22:02:44.752737 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.752751    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fd9a1e32-2b8b-4f2c-bcfc-5b7390d9a4a2" containerName="maas-api"
Oct 30 22:02:44.752967 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.752767    5032 cpu_manager.go:414] "containerMap: removing container" podUID="b19b4f72-13ba-4a3f-b3a8-73bc3c4c1d4b" containerName="maas-api"
Oct 30 22:02:44.752967 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.752775    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b19b4f72-13ba-4a3f-b3a8-73bc3c4c1d4b" containerName="maas-api"
Oct 30 22:02:44.752967 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.752791    5032 cpu_manager.go:414] "containerMap: removing container" podUID="2d543f80-60d7-4c58-ae3d-667f2e577282" containerName="maas-api"
Oct 30 22:02:44.752967 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.752798    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2d543f80-60d7-4c58-ae3d-667f2e577282" containerName="maas-api"
Oct 30 22:02:44.752967 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.752815    5032 cpu_manager.go:414] "containerMap: removing container" podUID="28ed1f60-5004-4fb1-bb80-33cd040611f6" containerName="payload-pre-processing"
Oct 30 22:02:44.752967 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.752822    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="28ed1f60-5004-4fb1-bb80-33cd040611f6" containerName="payload-pre-processing"
Oct 30 22:02:44.752967 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.752865    5032 cpu_manager.go:414] "containerMap: removing container" podUID="4c6d3630-8f1a-4298-ab10-d5619869e909" containerName="maas-api"
Oct 30 22:02:44.752967 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.752873    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4c6d3630-8f1a-4298-ab10-d5619869e909" containerName="maas-api"
Oct 30 22:02:44.752967 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.752888    5032 cpu_manager.go:414] "containerMap: removing container" podUID="5d5e4361-e970-43dd-b1e7-66ca197b10ce" containerName="maas-api"
Oct 30 22:02:44.752967 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.752895    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5d5e4361-e970-43dd-b1e7-66ca197b10ce" containerName="maas-api"
Oct 30 22:02:44.752967 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.752911    5032 cpu_manager.go:414] "containerMap: removing container" podUID="76de8529-43f7-4285-864b-f8242fca2cae" containerName="payload-processing"
Oct 30 22:02:44.752967 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.752918    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="76de8529-43f7-4285-864b-f8242fca2cae" containerName="payload-processing"
Oct 30 22:02:44.752967 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.752930    5032 cpu_manager.go:414] "containerMap: removing container" podUID="cd4228de-c71f-4f88-a54e-9ebda8b91122" containerName="istio-proxy"
Oct 30 22:02:44.752967 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.752937    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cd4228de-c71f-4f88-a54e-9ebda8b91122" containerName="istio-proxy"
Oct 30 22:02:44.752967 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.752952    5032 cpu_manager.go:414] "containerMap: removing container" podUID="34fdf57d-86f6-474f-a4f6-811e5063c405" containerName="main"
Oct 30 22:02:44.752967 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.752959    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="34fdf57d-86f6-474f-a4f6-811e5063c405" containerName="main"
Oct 30 22:02:44.752967 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.752971    5032 cpu_manager.go:414] "containerMap: removing container" podUID="aed33297-3e27-4d4a-a767-4191e8bc5127" containerName="istio-proxy"
Oct 30 22:02:44.752967 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.752978    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aed33297-3e27-4d4a-a767-4191e8bc5127" containerName="istio-proxy"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.752988    5032 cpu_manager.go:414] "containerMap: removing container" podUID="3eecfc0e-8c6b-4d24-b14b-242e6e52829d" containerName="maas-api"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.752996    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3eecfc0e-8c6b-4d24-b14b-242e6e52829d" containerName="maas-api"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753006    5032 cpu_manager.go:414] "containerMap: removing container" podUID="6ce45345-9ad2-4e54-b19b-ffa9a73db1be" containerName="maas-api"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753012    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6ce45345-9ad2-4e54-b19b-ffa9a73db1be" containerName="maas-api"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753023    5032 cpu_manager.go:414] "containerMap: removing container" podUID="27aef427-6b1e-4845-b8d9-a83cba81b0e0" containerName="payload-pre-processing"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753029    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="27aef427-6b1e-4845-b8d9-a83cba81b0e0" containerName="payload-pre-processing"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753049    5032 cpu_manager.go:414] "containerMap: removing container" podUID="b68282ae-6782-4b8f-9686-73089f54ff78" containerName="maas-api"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753056    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b68282ae-6782-4b8f-9686-73089f54ff78" containerName="maas-api"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753066    5032 cpu_manager.go:414] "containerMap: removing container" podUID="56d64255-3745-4df3-b9f3-9a15db845698" containerName="discovery"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753073    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="56d64255-3745-4df3-b9f3-9a15db845698" containerName="discovery"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753082    5032 cpu_manager.go:414] "containerMap: removing container" podUID="3d633840-2f60-4a22-9492-b53908c54f33" containerName="istio-proxy"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753089    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3d633840-2f60-4a22-9492-b53908c54f33" containerName="istio-proxy"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753100    5032 cpu_manager.go:414] "containerMap: removing container" podUID="00d3249f-1824-4ba1-8c18-6f6ab1f7ff92" containerName="payload-processing"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753106    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="00d3249f-1824-4ba1-8c18-6f6ab1f7ff92" containerName="payload-processing"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753117    5032 cpu_manager.go:414] "containerMap: removing container" podUID="968e86d5-7b2e-4f83-ad1d-68b62aadfa18" containerName="istio-proxy"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753124    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="968e86d5-7b2e-4f83-ad1d-68b62aadfa18" containerName="istio-proxy"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753134    5032 cpu_manager.go:414] "containerMap: removing container" podUID="ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74" containerName="istio-proxy"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753141    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74" containerName="istio-proxy"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753153    5032 cpu_manager.go:414] "containerMap: removing container" podUID="18d702a7-00ad-4a6b-8de5-806e5900cd7d" containerName="payload-pre-processing"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753160    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="18d702a7-00ad-4a6b-8de5-806e5900cd7d" containerName="payload-pre-processing"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753172    5032 cpu_manager.go:414] "containerMap: removing container" podUID="3aee81c5-228b-4f21-b7d9-a0bd0822b4df" containerName="istio-proxy"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753179    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3aee81c5-228b-4f21-b7d9-a0bd0822b4df" containerName="istio-proxy"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753190    5032 cpu_manager.go:414] "containerMap: removing container" podUID="2d0ea058-b6ca-4181-8823-2b719883e545" containerName="main"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753197    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2d0ea058-b6ca-4181-8823-2b719883e545" containerName="main"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753210    5032 cpu_manager.go:414] "containerMap: removing container" podUID="467142c6-5675-4dd8-87a0-14e1f9fddc57" containerName="payload-processing"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753217    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="467142c6-5675-4dd8-87a0-14e1f9fddc57" containerName="payload-processing"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753228    5032 cpu_manager.go:414] "containerMap: removing container" podUID="0d5b3ad5-309e-4c34-89a3-034e4966e652" containerName="istio-proxy"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753234    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0d5b3ad5-309e-4c34-89a3-034e4966e652" containerName="istio-proxy"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753244    5032 cpu_manager.go:414] "containerMap: removing container" podUID="d0ff5023-fd3f-4734-94c6-76b666010adf" containerName="main"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753251    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d0ff5023-fd3f-4734-94c6-76b666010adf" containerName="main"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753260    5032 cpu_manager.go:414] "containerMap: removing container" podUID="914f7544-6ef8-4496-b98d-b7ebc4123cda" containerName="main"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753266    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="914f7544-6ef8-4496-b98d-b7ebc4123cda" containerName="main"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753277    5032 cpu_manager.go:414] "containerMap: removing container" podUID="9b50efc9-2123-439d-9a07-e027fe6a6bfa" containerName="discovery"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753296    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9b50efc9-2123-439d-9a07-e027fe6a6bfa" containerName="discovery"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753312    5032 cpu_manager.go:414] "containerMap: removing container" podUID="23b97dbb-8488-4c44-a810-882ea18ed919" containerName="main"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753319    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="23b97dbb-8488-4c44-a810-882ea18ed919" containerName="main"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753332    5032 cpu_manager.go:414] "containerMap: removing container" podUID="ca2836d0-0021-4c48-9b62-03e24eb03ac8" containerName="manager"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753339    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ca2836d0-0021-4c48-9b62-03e24eb03ac8" containerName="manager"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753347    5032 cpu_manager.go:414] "containerMap: removing container" podUID="2b720f88-2fd8-4c4d-a300-941b1a9db6e2" containerName="istio-proxy"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753355    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2b720f88-2fd8-4c4d-a300-941b1a9db6e2" containerName="istio-proxy"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753381    5032 cpu_manager.go:414] "containerMap: removing container" podUID="e7679391-8575-43d7-b7f1-5b1ab0bcee5f" containerName="istio-proxy"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753388    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e7679391-8575-43d7-b7f1-5b1ab0bcee5f" containerName="istio-proxy"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753397    5032 cpu_manager.go:414] "containerMap: removing container" podUID="e25d6f68-6e47-449f-b02f-03fff2b2bb30" containerName="payload-processing"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753404    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e25d6f68-6e47-449f-b02f-03fff2b2bb30" containerName="payload-processing"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753414    5032 cpu_manager.go:414] "containerMap: removing container" podUID="fb5d58b6-89f9-4735-ba24-e4165dcb4c81" containerName="istio-proxy"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753420    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fb5d58b6-89f9-4735-ba24-e4165dcb4c81" containerName="istio-proxy"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753431    5032 cpu_manager.go:414] "containerMap: removing container" podUID="629450ab-662b-43c5-b93d-e264b1ce2034" containerName="istio-proxy"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753439    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="629450ab-662b-43c5-b93d-e264b1ce2034" containerName="istio-proxy"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753448    5032 cpu_manager.go:414] "containerMap: removing container" podUID="c0ca92b2-a242-47f5-9cc6-ee04860b5692" containerName="istio-proxy"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753454    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c0ca92b2-a242-47f5-9cc6-ee04860b5692" containerName="istio-proxy"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753583    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="28ed1f60-5004-4fb1-bb80-33cd040611f6" containerName="payload-pre-processing"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753597    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="4c6d3630-8f1a-4298-ab10-d5619869e909" containerName="maas-api"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753606    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="5d5e4361-e970-43dd-b1e7-66ca197b10ce" containerName="maas-api"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753619    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="76de8529-43f7-4285-864b-f8242fca2cae" containerName="payload-processing"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753628    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="cd4228de-c71f-4f88-a54e-9ebda8b91122" containerName="istio-proxy"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753637    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="34fdf57d-86f6-474f-a4f6-811e5063c405" containerName="main"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753646    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="aed33297-3e27-4d4a-a767-4191e8bc5127" containerName="istio-proxy"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753657    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="3eecfc0e-8c6b-4d24-b14b-242e6e52829d" containerName="maas-api"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753666    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="6ce45345-9ad2-4e54-b19b-ffa9a73db1be" containerName="maas-api"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753673    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="27aef427-6b1e-4845-b8d9-a83cba81b0e0" containerName="payload-pre-processing"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753682    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="b68282ae-6782-4b8f-9686-73089f54ff78" containerName="maas-api"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753690    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="56d64255-3745-4df3-b9f3-9a15db845698" containerName="discovery"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753699    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="3d633840-2f60-4a22-9492-b53908c54f33" containerName="istio-proxy"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753709    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="00d3249f-1824-4ba1-8c18-6f6ab1f7ff92" containerName="payload-processing"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753719    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="968e86d5-7b2e-4f83-ad1d-68b62aadfa18" containerName="istio-proxy"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753726    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="ec3d16a7-d7ee-4ae9-9d0c-f5f330280c74" containerName="istio-proxy"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753733    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="18d702a7-00ad-4a6b-8de5-806e5900cd7d" containerName="payload-pre-processing"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753742    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="3aee81c5-228b-4f21-b7d9-a0bd0822b4df" containerName="istio-proxy"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753751    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="2d0ea058-b6ca-4181-8823-2b719883e545" containerName="main"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753760    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="467142c6-5675-4dd8-87a0-14e1f9fddc57" containerName="payload-processing"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753768    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="0d5b3ad5-309e-4c34-89a3-034e4966e652" containerName="istio-proxy"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753776    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="d0ff5023-fd3f-4734-94c6-76b666010adf" containerName="main"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753785    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="914f7544-6ef8-4496-b98d-b7ebc4123cda" containerName="main"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753791    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="9b50efc9-2123-439d-9a07-e027fe6a6bfa" containerName="discovery"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753800    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="23b97dbb-8488-4c44-a810-882ea18ed919" containerName="main"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753807    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="ca2836d0-0021-4c48-9b62-03e24eb03ac8" containerName="manager"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753816    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="2b720f88-2fd8-4c4d-a300-941b1a9db6e2" containerName="istio-proxy"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753825    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="e7679391-8575-43d7-b7f1-5b1ab0bcee5f" containerName="istio-proxy"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753832    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="e25d6f68-6e47-449f-b02f-03fff2b2bb30" containerName="payload-processing"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753839    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="fb5d58b6-89f9-4735-ba24-e4165dcb4c81" containerName="istio-proxy"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753851    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="629450ab-662b-43c5-b93d-e264b1ce2034" containerName="istio-proxy"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753858    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="c0ca92b2-a242-47f5-9cc6-ee04860b5692" containerName="istio-proxy"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753867    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="6b99e2de-e8a0-4566-b14c-dd627c74edf3" containerName="istio-proxy"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753875    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="86937e06-85cc-48d5-9935-9ae922c3518c" containerName="istio-proxy"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753884    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="fd9a1e32-2b8b-4f2c-bcfc-5b7390d9a4a2" containerName="maas-api"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753895    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="b19b4f72-13ba-4a3f-b3a8-73bc3c4c1d4b" containerName="maas-api"
Oct 30 22:02:44.753871 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.753905    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="2d543f80-60d7-4c58-ae3d-667f2e577282" containerName="maas-api"
Oct 30 22:02:44.759615 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.758286    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-fa15d4-75dbfb6d49-jrq42"
Oct 30 22:02:44.769807 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.767912    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-single-main-fa15d4-serving-cert\""
Oct 30 22:02:44.773735 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.772170    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-fa15d4-75dbfb6d49-jrq42"]
Oct 30 22:02:44.774815 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.774786    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cb401365-1875-49f2-ac03-07d9aa93ebb7-metrics-certs\") pod \"maas-api-e2e-worker-single-main-fa15d4-75dbfb6d49-jrq42\" (UID: \"cb401365-1875-49f2-ac03-07d9aa93ebb7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-fa15d4-75dbfb6d49-jrq42"
Oct 30 22:02:44.774987 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.774972    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njvb6\" (UniqueName: \"kubernetes.io/projected/cb401365-1875-49f2-ac03-07d9aa93ebb7-kube-api-access-njvb6\") pod \"maas-api-e2e-worker-single-main-fa15d4-75dbfb6d49-jrq42\" (UID: \"cb401365-1875-49f2-ac03-07d9aa93ebb7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-fa15d4-75dbfb6d49-jrq42"
Oct 30 22:02:44.775144 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.775130    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cb401365-1875-49f2-ac03-07d9aa93ebb7-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-fa15d4-75dbfb6d49-jrq42\" (UID: \"cb401365-1875-49f2-ac03-07d9aa93ebb7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-fa15d4-75dbfb6d49-jrq42"
Oct 30 22:02:44.775767 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.775333    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb401365-1875-49f2-ac03-07d9aa93ebb7-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-fa15d4-75dbfb6d49-jrq42\" (UID: \"cb401365-1875-49f2-ac03-07d9aa93ebb7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-fa15d4-75dbfb6d49-jrq42"
Oct 30 22:02:44.877430 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.877390    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb401365-1875-49f2-ac03-07d9aa93ebb7-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-fa15d4-75dbfb6d49-jrq42\" (UID: \"cb401365-1875-49f2-ac03-07d9aa93ebb7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-fa15d4-75dbfb6d49-jrq42"
Oct 30 22:02:44.877601 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.877578    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cb401365-1875-49f2-ac03-07d9aa93ebb7-metrics-certs\") pod \"maas-api-e2e-worker-single-main-fa15d4-75dbfb6d49-jrq42\" (UID: \"cb401365-1875-49f2-ac03-07d9aa93ebb7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-fa15d4-75dbfb6d49-jrq42"
Oct 30 22:02:44.877644 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.877626    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njvb6\" (UniqueName: \"kubernetes.io/projected/cb401365-1875-49f2-ac03-07d9aa93ebb7-kube-api-access-njvb6\") pod \"maas-api-e2e-worker-single-main-fa15d4-75dbfb6d49-jrq42\" (UID: \"cb401365-1875-49f2-ac03-07d9aa93ebb7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-fa15d4-75dbfb6d49-jrq42"
Oct 30 22:02:44.877684 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.877663    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cb401365-1875-49f2-ac03-07d9aa93ebb7-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-fa15d4-75dbfb6d49-jrq42\" (UID: \"cb401365-1875-49f2-ac03-07d9aa93ebb7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-fa15d4-75dbfb6d49-jrq42"
Oct 30 22:02:44.878148 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.878126    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb401365-1875-49f2-ac03-07d9aa93ebb7-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-fa15d4-75dbfb6d49-jrq42\" (UID: \"cb401365-1875-49f2-ac03-07d9aa93ebb7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-fa15d4-75dbfb6d49-jrq42"
Oct 30 22:02:44.880107 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.880077    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cb401365-1875-49f2-ac03-07d9aa93ebb7-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-fa15d4-75dbfb6d49-jrq42\" (UID: \"cb401365-1875-49f2-ac03-07d9aa93ebb7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-fa15d4-75dbfb6d49-jrq42"
Oct 30 22:02:44.880219 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.880184    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cb401365-1875-49f2-ac03-07d9aa93ebb7-metrics-certs\") pod \"maas-api-e2e-worker-single-main-fa15d4-75dbfb6d49-jrq42\" (UID: \"cb401365-1875-49f2-ac03-07d9aa93ebb7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-fa15d4-75dbfb6d49-jrq42"
Oct 30 22:02:44.903568 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:44.903538    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-njvb6\" (UniqueName: \"kubernetes.io/projected/cb401365-1875-49f2-ac03-07d9aa93ebb7-kube-api-access-njvb6\") pod \"maas-api-e2e-worker-single-main-fa15d4-75dbfb6d49-jrq42\" (UID: \"cb401365-1875-49f2-ac03-07d9aa93ebb7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-fa15d4-75dbfb6d49-jrq42"
Oct 30 22:02:45.084656 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:45.084573    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-fa15d4-75dbfb6d49-jrq42"
Oct 30 22:02:45.373198 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:45.373171    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-fa15d4-75dbfb6d49-jrq42"]
Oct 30 22:02:45.375612 ip-10-0-129-195 kubenswrapper[5032]: W1030 22:02:45.375583    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb401365_1875_49f2_ac03_07d9aa93ebb7.slice/crio-c0972a373960aa684dd9fabffbe30002f795bb78027e72e2a4d1eb9bb7576cd5 WatchSource:0}: Error finding container c0972a373960aa684dd9fabffbe30002f795bb78027e72e2a4d1eb9bb7576cd5: Status 404 returned error can't find the container with id c0972a373960aa684dd9fabffbe30002f795bb78027e72e2a4d1eb9bb7576cd5
Oct 30 22:02:45.376958 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:45.376936    5032 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:02:45.568317 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:45.568275    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-fa15d4-75dbfb6d49-jrq42" event={"ID":"cb401365-1875-49f2-ac03-07d9aa93ebb7","Type":"ContainerStarted","Data":"c0972a373960aa684dd9fabffbe30002f795bb78027e72e2a4d1eb9bb7576cd5"}
Oct 30 22:02:46.574719 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:46.574682    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-fa15d4-75dbfb6d49-jrq42" event={"ID":"cb401365-1875-49f2-ac03-07d9aa93ebb7","Type":"ContainerStarted","Data":"93ed759b1a6978ba4fb5c2e9f349e93ab8002379ebe76e0801711118192861b6"}
Oct 30 22:02:46.575197 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:46.574762    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-fa15d4-75dbfb6d49-jrq42"
Oct 30 22:02:46.597041 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:46.596997    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-fa15d4-75dbfb6d49-jrq42" podStartSLOduration=2.219134478 podStartE2EDuration="2.596984026s" totalImagesPullingTime="377.849548ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:02:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:02:46.59544215 +0000 UTC m=+2079.397254614" watchObservedRunningTime="2026-10-30 22:02:46.596984026 +0000 UTC m=+2079.398796510"
Oct 30 22:02:47.349279 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:47.349249    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 22:02:48.344146 ip-10-0-129-195 kubenswrapper[5032]: E1030 22:02:48.344111    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 22:02:52.585560 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:02:52.585533    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-fa15d4-75dbfb6d49-jrq42"
Oct 30 22:03:03.222063 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:03:03.222018    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 22:03:10.110535 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:03:10.110460    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-sklxd_67292bdc-7fe1-449d-af75-2baa90c16ea9/console-operator/1.log"
Oct 30 22:03:10.255639 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:03:10.255611    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-sklxd_67292bdc-7fe1-449d-af75-2baa90c16ea9/console-operator/1.log"
Oct 30 22:03:18.344885 ip-10-0-129-195 kubenswrapper[5032]: E1030 22:03:18.344812    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 22:03:19.719782 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:03:19.719744    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 22:03:44.928598 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:03:44.928559    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 22:03:48.345948 ip-10-0-129-195 kubenswrapper[5032]: E1030 22:03:48.345910    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 22:03:54.624104 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:03:54.624062    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-fa15d4-75dbfb6d49-jrq42"]
Oct 30 22:03:54.624560 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:03:54.624286    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-fa15d4-75dbfb6d49-jrq42" podUID="cb401365-1875-49f2-ac03-07d9aa93ebb7" containerName="maas-api" containerID="cri-o://93ed759b1a6978ba4fb5c2e9f349e93ab8002379ebe76e0801711118192861b6" gracePeriod=30
Oct 30 22:03:54.895659 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:03:54.895631    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-fa15d4-75dbfb6d49-jrq42"
Oct 30 22:03:54.920006 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:03:54.919971    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cb401365-1875-49f2-ac03-07d9aa93ebb7-kube-api-access-njvb6\" (UniqueName: \"kubernetes.io/projected/cb401365-1875-49f2-ac03-07d9aa93ebb7-kube-api-access-njvb6\") pod \"cb401365-1875-49f2-ac03-07d9aa93ebb7\" (UID: \"cb401365-1875-49f2-ac03-07d9aa93ebb7\") "
Oct 30 22:03:54.920212 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:03:54.920041    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cb401365-1875-49f2-ac03-07d9aa93ebb7-metrics-certs\" (UniqueName: \"kubernetes.io/secret/cb401365-1875-49f2-ac03-07d9aa93ebb7-metrics-certs\") pod \"cb401365-1875-49f2-ac03-07d9aa93ebb7\" (UID: \"cb401365-1875-49f2-ac03-07d9aa93ebb7\") "
Oct 30 22:03:54.920212 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:03:54.920162    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cb401365-1875-49f2-ac03-07d9aa93ebb7-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cb401365-1875-49f2-ac03-07d9aa93ebb7-maas-api-tls\") pod \"cb401365-1875-49f2-ac03-07d9aa93ebb7\" (UID: \"cb401365-1875-49f2-ac03-07d9aa93ebb7\") "
Oct 30 22:03:54.920333 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:03:54.920217    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cb401365-1875-49f2-ac03-07d9aa93ebb7-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb401365-1875-49f2-ac03-07d9aa93ebb7-openshift-service-ca\") pod \"cb401365-1875-49f2-ac03-07d9aa93ebb7\" (UID: \"cb401365-1875-49f2-ac03-07d9aa93ebb7\") "
Oct 30 22:03:54.920939 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:03:54.920910    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb401365-1875-49f2-ac03-07d9aa93ebb7-openshift-service-ca" pod "cb401365-1875-49f2-ac03-07d9aa93ebb7" (UID: "cb401365-1875-49f2-ac03-07d9aa93ebb7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:03:54.925552 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:03:54.925517    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb401365-1875-49f2-ac03-07d9aa93ebb7-metrics-certs" pod "cb401365-1875-49f2-ac03-07d9aa93ebb7" (UID: "cb401365-1875-49f2-ac03-07d9aa93ebb7"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:03:54.925694 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:03:54.925594    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb401365-1875-49f2-ac03-07d9aa93ebb7-maas-api-tls" pod "cb401365-1875-49f2-ac03-07d9aa93ebb7" (UID: "cb401365-1875-49f2-ac03-07d9aa93ebb7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:03:54.927171 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:03:54.927143    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb401365-1875-49f2-ac03-07d9aa93ebb7-kube-api-access-njvb6" pod "cb401365-1875-49f2-ac03-07d9aa93ebb7" (UID: "cb401365-1875-49f2-ac03-07d9aa93ebb7"). InnerVolumeSpecName "kube-api-access-njvb6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:03:54.944097 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:03:54.944066    5032 generic.go:356] "Generic (PLEG): container finished" podID="cb401365-1875-49f2-ac03-07d9aa93ebb7" containerID="93ed759b1a6978ba4fb5c2e9f349e93ab8002379ebe76e0801711118192861b6" exitCode=0
Oct 30 22:03:54.944257 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:03:54.944133    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-fa15d4-75dbfb6d49-jrq42"
Oct 30 22:03:54.944257 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:03:54.944151    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-fa15d4-75dbfb6d49-jrq42" event={"ID":"cb401365-1875-49f2-ac03-07d9aa93ebb7","Type":"ContainerDied","Data":"93ed759b1a6978ba4fb5c2e9f349e93ab8002379ebe76e0801711118192861b6"}
Oct 30 22:03:54.944257 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:03:54.944221    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-fa15d4-75dbfb6d49-jrq42" event={"ID":"cb401365-1875-49f2-ac03-07d9aa93ebb7","Type":"ContainerDied","Data":"c0972a373960aa684dd9fabffbe30002f795bb78027e72e2a4d1eb9bb7576cd5"}
Oct 30 22:03:54.944257 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:03:54.944253    5032 scope.go:122] "RemoveContainer" containerID="93ed759b1a6978ba4fb5c2e9f349e93ab8002379ebe76e0801711118192861b6"
Oct 30 22:03:54.955355 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:03:54.955330    5032 scope.go:122] "RemoveContainer" containerID="93ed759b1a6978ba4fb5c2e9f349e93ab8002379ebe76e0801711118192861b6"
Oct 30 22:03:54.955674 ip-10-0-129-195 kubenswrapper[5032]: E1030 22:03:54.955652    5032 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93ed759b1a6978ba4fb5c2e9f349e93ab8002379ebe76e0801711118192861b6\": container with ID starting with 93ed759b1a6978ba4fb5c2e9f349e93ab8002379ebe76e0801711118192861b6 not found: ID does not exist" containerID="93ed759b1a6978ba4fb5c2e9f349e93ab8002379ebe76e0801711118192861b6"
Oct 30 22:03:54.955764 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:03:54.955683    5032 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93ed759b1a6978ba4fb5c2e9f349e93ab8002379ebe76e0801711118192861b6"} err="failed to get container status \"93ed759b1a6978ba4fb5c2e9f349e93ab8002379ebe76e0801711118192861b6\": rpc error: code = NotFound desc = could not find container \"93ed759b1a6978ba4fb5c2e9f349e93ab8002379ebe76e0801711118192861b6\": container with ID starting with 93ed759b1a6978ba4fb5c2e9f349e93ab8002379ebe76e0801711118192861b6 not found: ID does not exist"
Oct 30 22:03:54.968995 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:03:54.968969    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-fa15d4-75dbfb6d49-jrq42"]
Oct 30 22:03:54.973236 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:03:54.973210    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-fa15d4-75dbfb6d49-jrq42"]
Oct 30 22:03:55.022069 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:03:55.022035    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-njvb6\" (UniqueName: \"kubernetes.io/projected/cb401365-1875-49f2-ac03-07d9aa93ebb7-kube-api-access-njvb6\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:03:55.022069 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:03:55.022066    5032 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cb401365-1875-49f2-ac03-07d9aa93ebb7-metrics-certs\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:03:55.022069 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:03:55.022078    5032 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cb401365-1875-49f2-ac03-07d9aa93ebb7-maas-api-tls\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:03:55.022305 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:03:55.022088    5032 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb401365-1875-49f2-ac03-07d9aa93ebb7-openshift-service-ca\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:03:55.345880 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:03:55.345850    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cb401365-1875-49f2-ac03-07d9aa93ebb7" path="/var/lib/kubelet/pods/cb401365-1875-49f2-ac03-07d9aa93ebb7/volumes"
Oct 30 22:03:56.355702 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:03:56.355661    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 22:04:14.927587 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:04:14.927550    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 22:04:16.950538 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:04:16.950488    5032 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-mxsln container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:04:16.951382 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:04:16.951321    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mxsln" podUID="f77afa6c-51f5-4bc2-b10e-89101bd067c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:17.018615 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:04:17.018565    5032 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-c46gq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:04:17.018967 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:04:17.018936    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-c46gq" podUID="e3b353c1-3812-44aa-b77d-d7ddfc3b7057" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:18.347828 ip-10-0-129-195 kubenswrapper[5032]: E1030 22:04:18.347790    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 22:04:19.627862 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:04:19.627819    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 22:04:48.344199 ip-10-0-129-195 kubenswrapper[5032]: E1030 22:04:48.344116    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 22:05:18.343815 ip-10-0-129-195 kubenswrapper[5032]: E1030 22:05:18.343776    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 22:05:26.434114 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:05:26.434069    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-wv78j"]
Oct 30 22:05:26.434837 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:05:26.434814    5032 cpu_manager.go:414] "containerMap: removing container" podUID="cb401365-1875-49f2-ac03-07d9aa93ebb7" containerName="maas-api"
Oct 30 22:05:26.434895 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:05:26.434840    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cb401365-1875-49f2-ac03-07d9aa93ebb7" containerName="maas-api"
Oct 30 22:05:26.435040 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:05:26.435027    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="cb401365-1875-49f2-ac03-07d9aa93ebb7" containerName="maas-api"
Oct 30 22:05:26.438910 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:05:26.438890    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-wv78j"
Oct 30 22:05:26.443107 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:05:26.443081    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-xn5tn\""
Oct 30 22:05:26.450900 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:05:26.450872    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-wv78j"]
Oct 30 22:05:26.566595 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:05:26.566547    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ww2bn\" (UniqueName: \"kubernetes.io/projected/4b7d8b7c-acb7-4d75-8377-db8e0a0b1a4b-kube-api-access-ww2bn\") pod \"kuadrant-operator-controller-manager-678d87766f-wv78j\" (UID: \"4b7d8b7c-acb7-4d75-8377-db8e0a0b1a4b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-wv78j"
Oct 30 22:05:26.566784 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:05:26.566616    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4b7d8b7c-acb7-4d75-8377-db8e0a0b1a4b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-wv78j\" (UID: \"4b7d8b7c-acb7-4d75-8377-db8e0a0b1a4b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-wv78j"
Oct 30 22:05:26.668110 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:05:26.668066    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ww2bn\" (UniqueName: \"kubernetes.io/projected/4b7d8b7c-acb7-4d75-8377-db8e0a0b1a4b-kube-api-access-ww2bn\") pod \"kuadrant-operator-controller-manager-678d87766f-wv78j\" (UID: \"4b7d8b7c-acb7-4d75-8377-db8e0a0b1a4b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-wv78j"
Oct 30 22:05:26.668303 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:05:26.668154    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4b7d8b7c-acb7-4d75-8377-db8e0a0b1a4b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-wv78j\" (UID: \"4b7d8b7c-acb7-4d75-8377-db8e0a0b1a4b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-wv78j"
Oct 30 22:05:26.668570 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:05:26.668548    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4b7d8b7c-acb7-4d75-8377-db8e0a0b1a4b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-wv78j\" (UID: \"4b7d8b7c-acb7-4d75-8377-db8e0a0b1a4b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-wv78j"
Oct 30 22:05:26.681228 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:05:26.681198    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ww2bn\" (UniqueName: \"kubernetes.io/projected/4b7d8b7c-acb7-4d75-8377-db8e0a0b1a4b-kube-api-access-ww2bn\") pod \"kuadrant-operator-controller-manager-678d87766f-wv78j\" (UID: \"4b7d8b7c-acb7-4d75-8377-db8e0a0b1a4b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-wv78j"
Oct 30 22:05:26.753020 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:05:26.752985    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-wv78j"
Oct 30 22:05:27.036784 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:05:27.036699    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-wv78j"]
Oct 30 22:05:27.040970 ip-10-0-129-195 kubenswrapper[5032]: W1030 22:05:27.040938    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b7d8b7c_acb7_4d75_8377_db8e0a0b1a4b.slice/crio-cf821433f800304b828bfe3aaa9e6797087f809191453399b51c48d4c3785596 WatchSource:0}: Error finding container cf821433f800304b828bfe3aaa9e6797087f809191453399b51c48d4c3785596: Status 404 returned error can't find the container with id cf821433f800304b828bfe3aaa9e6797087f809191453399b51c48d4c3785596
Oct 30 22:05:27.475831 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:05:27.475790    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-wv78j" event={"ID":"4b7d8b7c-acb7-4d75-8377-db8e0a0b1a4b","Type":"ContainerStarted","Data":"eccff949d9c1685197f3efc02a5e872f95f4052d6bf67eed26a2753eced6add1"}
Oct 30 22:05:27.475831 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:05:27.475848    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-wv78j" event={"ID":"4b7d8b7c-acb7-4d75-8377-db8e0a0b1a4b","Type":"ContainerStarted","Data":"cf821433f800304b828bfe3aaa9e6797087f809191453399b51c48d4c3785596"}
Oct 30 22:05:27.476401 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:05:27.475873    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-wv78j"
Oct 30 22:05:27.503958 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:05:27.503907    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-wv78j" podStartSLOduration=1.503892146 podStartE2EDuration="1.503892146s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:05:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:05:27.501033113 +0000 UTC m=+2240.302845578" watchObservedRunningTime="2026-10-30 22:05:27.503892146 +0000 UTC m=+2240.305704603"
Oct 30 22:05:38.484892 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:05:38.484862    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-wv78j"
Oct 30 22:05:44.014386 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:05:44.014329    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 22:05:46.949599 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:05:46.949536    5032 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-mxsln container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:05:46.950279 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:05:46.950244    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mxsln" podUID="f77afa6c-51f5-4bc2-b10e-89101bd067c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:05:48.344077 ip-10-0-129-195 kubenswrapper[5032]: E1030 22:05:48.344042    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 22:05:59.091608 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:05:59.091570    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 22:06:02.943049 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:06:02.943009    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 22:06:18.191338 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:06:18.191258    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 22:06:18.346214 ip-10-0-129-195 kubenswrapper[5032]: E1030 22:06:18.346166    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 22:06:26.553721 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:06:26.553684    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 22:06:42.013657 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:06:42.013620    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 22:06:48.345024 ip-10-0-129-195 kubenswrapper[5032]: E1030 22:06:48.344991    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 22:06:58.225097 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:06:58.225058    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 22:07:18.344736 ip-10-0-129-195 kubenswrapper[5032]: E1030 22:07:18.344693    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 22:07:22.375122 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:07:22.375083    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 22:07:34.011415 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:07:34.008582    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 22:07:45.819494 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:07:45.819447    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 22:07:48.423420 ip-10-0-129-195 kubenswrapper[5032]: E1030 22:07:48.423185    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 22:08:03.136849 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:08:03.136802    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 22:08:10.147148 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:08:10.146999    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-sklxd_67292bdc-7fe1-449d-af75-2baa90c16ea9/console-operator/1.log"
Oct 30 22:08:10.290040 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:08:10.290010    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-sklxd_67292bdc-7fe1-449d-af75-2baa90c16ea9/console-operator/1.log"
Oct 30 22:08:18.344840 ip-10-0-129-195 kubenswrapper[5032]: E1030 22:08:18.344801    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 22:08:19.498368 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:08:19.498332    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 22:08:30.703053 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:08:30.703016    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 22:08:36.716737 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:08:36.716680    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 22:08:45.307104 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:08:45.307068    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 22:08:46.960591 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:08:46.960511    5032 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-mxsln container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:08:46.961265 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:08:46.961206    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mxsln" podUID="f77afa6c-51f5-4bc2-b10e-89101bd067c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:08:47.018226 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:08:47.018182    5032 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-c46gq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:08:47.018548 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:08:47.018517    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-c46gq" podUID="e3b353c1-3812-44aa-b77d-d7ddfc3b7057" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:08:47.091677 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:08:47.091635    5032 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-7s44p container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:08:47.091997 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:08:47.091951    5032 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-7s44p" podUID="457cd47c-9543-4d82-875f-f1e45d6d5b70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:08:48.355956 ip-10-0-129-195 kubenswrapper[5032]: E1030 22:08:48.355911    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 22:08:49.733393 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:08:49.732784    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 22:09:05.999252 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:05.999218    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 22:09:07.473065 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:07.473028    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6fdb49dd46-zp66m"]
Oct 30 22:09:07.477687 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:07.477660    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6fdb49dd46-zp66m"
Oct 30 22:09:07.497992 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:07.497962    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6fdb49dd46-zp66m"]
Oct 30 22:09:07.671196 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:07.671161    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afa4bfff-93da-4cdc-abdf-303292d1656c-metrics-certs\") pod \"maas-api-6fdb49dd46-zp66m\" (UID: \"afa4bfff-93da-4cdc-abdf-303292d1656c\") " pod="odh-ai-gateway-infra/maas-api-6fdb49dd46-zp66m"
Oct 30 22:09:07.671196 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:07.671204    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/afa4bfff-93da-4cdc-abdf-303292d1656c-openshift-service-ca\") pod \"maas-api-6fdb49dd46-zp66m\" (UID: \"afa4bfff-93da-4cdc-abdf-303292d1656c\") " pod="odh-ai-gateway-infra/maas-api-6fdb49dd46-zp66m"
Oct 30 22:09:07.671488 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:07.671325    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86xkd\" (UniqueName: \"kubernetes.io/projected/afa4bfff-93da-4cdc-abdf-303292d1656c-kube-api-access-86xkd\") pod \"maas-api-6fdb49dd46-zp66m\" (UID: \"afa4bfff-93da-4cdc-abdf-303292d1656c\") " pod="odh-ai-gateway-infra/maas-api-6fdb49dd46-zp66m"
Oct 30 22:09:07.671488 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:07.671437    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/afa4bfff-93da-4cdc-abdf-303292d1656c-maas-api-tls\") pod \"maas-api-6fdb49dd46-zp66m\" (UID: \"afa4bfff-93da-4cdc-abdf-303292d1656c\") " pod="odh-ai-gateway-infra/maas-api-6fdb49dd46-zp66m"
Oct 30 22:09:07.773188 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:07.773102    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afa4bfff-93da-4cdc-abdf-303292d1656c-metrics-certs\") pod \"maas-api-6fdb49dd46-zp66m\" (UID: \"afa4bfff-93da-4cdc-abdf-303292d1656c\") " pod="odh-ai-gateway-infra/maas-api-6fdb49dd46-zp66m"
Oct 30 22:09:07.773188 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:07.773158    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/afa4bfff-93da-4cdc-abdf-303292d1656c-openshift-service-ca\") pod \"maas-api-6fdb49dd46-zp66m\" (UID: \"afa4bfff-93da-4cdc-abdf-303292d1656c\") " pod="odh-ai-gateway-infra/maas-api-6fdb49dd46-zp66m"
Oct 30 22:09:07.773473 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:07.773226    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86xkd\" (UniqueName: \"kubernetes.io/projected/afa4bfff-93da-4cdc-abdf-303292d1656c-kube-api-access-86xkd\") pod \"maas-api-6fdb49dd46-zp66m\" (UID: \"afa4bfff-93da-4cdc-abdf-303292d1656c\") " pod="odh-ai-gateway-infra/maas-api-6fdb49dd46-zp66m"
Oct 30 22:09:07.773473 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:07.773264    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/afa4bfff-93da-4cdc-abdf-303292d1656c-maas-api-tls\") pod \"maas-api-6fdb49dd46-zp66m\" (UID: \"afa4bfff-93da-4cdc-abdf-303292d1656c\") " pod="odh-ai-gateway-infra/maas-api-6fdb49dd46-zp66m"
Oct 30 22:09:07.774070 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:07.774030    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/afa4bfff-93da-4cdc-abdf-303292d1656c-openshift-service-ca\") pod \"maas-api-6fdb49dd46-zp66m\" (UID: \"afa4bfff-93da-4cdc-abdf-303292d1656c\") " pod="odh-ai-gateway-infra/maas-api-6fdb49dd46-zp66m"
Oct 30 22:09:07.776325 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:07.776290    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/afa4bfff-93da-4cdc-abdf-303292d1656c-maas-api-tls\") pod \"maas-api-6fdb49dd46-zp66m\" (UID: \"afa4bfff-93da-4cdc-abdf-303292d1656c\") " pod="odh-ai-gateway-infra/maas-api-6fdb49dd46-zp66m"
Oct 30 22:09:07.776441 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:07.776382    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afa4bfff-93da-4cdc-abdf-303292d1656c-metrics-certs\") pod \"maas-api-6fdb49dd46-zp66m\" (UID: \"afa4bfff-93da-4cdc-abdf-303292d1656c\") " pod="odh-ai-gateway-infra/maas-api-6fdb49dd46-zp66m"
Oct 30 22:09:07.786196 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:07.786168    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-86xkd\" (UniqueName: \"kubernetes.io/projected/afa4bfff-93da-4cdc-abdf-303292d1656c-kube-api-access-86xkd\") pod \"maas-api-6fdb49dd46-zp66m\" (UID: \"afa4bfff-93da-4cdc-abdf-303292d1656c\") " pod="odh-ai-gateway-infra/maas-api-6fdb49dd46-zp66m"
Oct 30 22:09:07.791847 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:07.791822    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6fdb49dd46-zp66m"
Oct 30 22:09:08.075813 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:08.075777    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6fdb49dd46-zp66m"]
Oct 30 22:09:08.077840 ip-10-0-129-195 kubenswrapper[5032]: W1030 22:09:08.077811    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafa4bfff_93da_4cdc_abdf_303292d1656c.slice/crio-f9b97b6e5f791b6ad732b1a0b8d8819c0c5270e582dbe7f3a83238f9b3ea2445 WatchSource:0}: Error finding container f9b97b6e5f791b6ad732b1a0b8d8819c0c5270e582dbe7f3a83238f9b3ea2445: Status 404 returned error can't find the container with id f9b97b6e5f791b6ad732b1a0b8d8819c0c5270e582dbe7f3a83238f9b3ea2445
Oct 30 22:09:08.079496 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:08.079478    5032 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:09:08.942905 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:08.942823    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6fdb49dd46-zp66m" event={"ID":"afa4bfff-93da-4cdc-abdf-303292d1656c","Type":"ContainerStarted","Data":"2fc04e9f44195d3b69ce69aec9354cf1eaa5b30087ccdc262274b13fa42c862d"}
Oct 30 22:09:08.942905 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:08.942875    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6fdb49dd46-zp66m" event={"ID":"afa4bfff-93da-4cdc-abdf-303292d1656c","Type":"ContainerStarted","Data":"f9b97b6e5f791b6ad732b1a0b8d8819c0c5270e582dbe7f3a83238f9b3ea2445"}
Oct 30 22:09:08.943467 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:08.943033    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6fdb49dd46-zp66m"
Oct 30 22:09:08.965612 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:08.965561    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6fdb49dd46-zp66m" podStartSLOduration=1.519527552 podStartE2EDuration="1.965546252s" totalImagesPullingTime="446.0187ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:09:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:09:08.963576896 +0000 UTC m=+2461.765389372" watchObservedRunningTime="2026-10-30 22:09:08.965546252 +0000 UTC m=+2461.767358715"
Oct 30 22:09:14.953548 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:14.953519    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6fdb49dd46-zp66m"
Oct 30 22:09:18.345334 ip-10-0-129-195 kubenswrapper[5032]: E1030 22:09:18.345227    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 22:09:19.837106 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:19.837056    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-83f27219-86849bdb8c-zv722"]
Oct 30 22:09:19.842712 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:19.842683    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-83f27219-86849bdb8c-zv722"
Oct 30 22:09:19.851831 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:19.851803    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-83f27219-serving-cert\""
Oct 30 22:09:19.860269 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:19.860221    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-83f27219-86849bdb8c-zv722"]
Oct 30 22:09:19.875378 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:19.875332    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72pbs\" (UniqueName: \"kubernetes.io/projected/3482a13a-303e-4fc9-af0a-18eb225b2e6f-kube-api-access-72pbs\") pod \"maas-api-e2e-ait-83f27219-86849bdb8c-zv722\" (UID: \"3482a13a-303e-4fc9-af0a-18eb225b2e6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-83f27219-86849bdb8c-zv722"
Oct 30 22:09:19.875532 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:19.875498    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3482a13a-303e-4fc9-af0a-18eb225b2e6f-maas-api-tls\") pod \"maas-api-e2e-ait-83f27219-86849bdb8c-zv722\" (UID: \"3482a13a-303e-4fc9-af0a-18eb225b2e6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-83f27219-86849bdb8c-zv722"
Oct 30 22:09:19.875608 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:19.875590    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3482a13a-303e-4fc9-af0a-18eb225b2e6f-metrics-certs\") pod \"maas-api-e2e-ait-83f27219-86849bdb8c-zv722\" (UID: \"3482a13a-303e-4fc9-af0a-18eb225b2e6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-83f27219-86849bdb8c-zv722"
Oct 30 22:09:19.875654 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:19.875640    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3482a13a-303e-4fc9-af0a-18eb225b2e6f-openshift-service-ca\") pod \"maas-api-e2e-ait-83f27219-86849bdb8c-zv722\" (UID: \"3482a13a-303e-4fc9-af0a-18eb225b2e6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-83f27219-86849bdb8c-zv722"
Oct 30 22:09:19.976500 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:19.976463    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72pbs\" (UniqueName: \"kubernetes.io/projected/3482a13a-303e-4fc9-af0a-18eb225b2e6f-kube-api-access-72pbs\") pod \"maas-api-e2e-ait-83f27219-86849bdb8c-zv722\" (UID: \"3482a13a-303e-4fc9-af0a-18eb225b2e6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-83f27219-86849bdb8c-zv722"
Oct 30 22:09:19.976691 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:19.976532    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3482a13a-303e-4fc9-af0a-18eb225b2e6f-maas-api-tls\") pod \"maas-api-e2e-ait-83f27219-86849bdb8c-zv722\" (UID: \"3482a13a-303e-4fc9-af0a-18eb225b2e6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-83f27219-86849bdb8c-zv722"
Oct 30 22:09:19.976691 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:19.976573    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3482a13a-303e-4fc9-af0a-18eb225b2e6f-metrics-certs\") pod \"maas-api-e2e-ait-83f27219-86849bdb8c-zv722\" (UID: \"3482a13a-303e-4fc9-af0a-18eb225b2e6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-83f27219-86849bdb8c-zv722"
Oct 30 22:09:19.976691 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:19.976628    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3482a13a-303e-4fc9-af0a-18eb225b2e6f-openshift-service-ca\") pod \"maas-api-e2e-ait-83f27219-86849bdb8c-zv722\" (UID: \"3482a13a-303e-4fc9-af0a-18eb225b2e6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-83f27219-86849bdb8c-zv722"
Oct 30 22:09:19.977441 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:19.977410    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3482a13a-303e-4fc9-af0a-18eb225b2e6f-openshift-service-ca\") pod \"maas-api-e2e-ait-83f27219-86849bdb8c-zv722\" (UID: \"3482a13a-303e-4fc9-af0a-18eb225b2e6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-83f27219-86849bdb8c-zv722"
Oct 30 22:09:19.979236 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:19.979213    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3482a13a-303e-4fc9-af0a-18eb225b2e6f-metrics-certs\") pod \"maas-api-e2e-ait-83f27219-86849bdb8c-zv722\" (UID: \"3482a13a-303e-4fc9-af0a-18eb225b2e6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-83f27219-86849bdb8c-zv722"
Oct 30 22:09:19.979323 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:19.979251    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3482a13a-303e-4fc9-af0a-18eb225b2e6f-maas-api-tls\") pod \"maas-api-e2e-ait-83f27219-86849bdb8c-zv722\" (UID: \"3482a13a-303e-4fc9-af0a-18eb225b2e6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-83f27219-86849bdb8c-zv722"
Oct 30 22:09:19.990729 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:19.990702    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-72pbs\" (UniqueName: \"kubernetes.io/projected/3482a13a-303e-4fc9-af0a-18eb225b2e6f-kube-api-access-72pbs\") pod \"maas-api-e2e-ait-83f27219-86849bdb8c-zv722\" (UID: \"3482a13a-303e-4fc9-af0a-18eb225b2e6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-83f27219-86849bdb8c-zv722"
Oct 30 22:09:20.164261 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:20.164165    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-83f27219-86849bdb8c-zv722"
Oct 30 22:09:20.449929 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:20.449902    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-83f27219-86849bdb8c-zv722"]
Oct 30 22:09:20.452307 ip-10-0-129-195 kubenswrapper[5032]: W1030 22:09:20.452280    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3482a13a_303e_4fc9_af0a_18eb225b2e6f.slice/crio-ea75d6081c8c061286462741189c73629a46f4a2df4bf5261ba499c85ff17b96 WatchSource:0}: Error finding container ea75d6081c8c061286462741189c73629a46f4a2df4bf5261ba499c85ff17b96: Status 404 returned error can't find the container with id ea75d6081c8c061286462741189c73629a46f4a2df4bf5261ba499c85ff17b96
Oct 30 22:09:21.010673 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:21.010631    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-83f27219-86849bdb8c-zv722" event={"ID":"3482a13a-303e-4fc9-af0a-18eb225b2e6f","Type":"ContainerStarted","Data":"88e889b6a384f47d37db5596bbe4c4a6fea3ac33a6ef76d5aa44d16075c2bb91"}
Oct 30 22:09:21.011073 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:21.010703    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-83f27219-86849bdb8c-zv722" event={"ID":"3482a13a-303e-4fc9-af0a-18eb225b2e6f","Type":"ContainerStarted","Data":"ea75d6081c8c061286462741189c73629a46f4a2df4bf5261ba499c85ff17b96"}
Oct 30 22:09:21.011073 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:21.010753    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-83f27219-86849bdb8c-zv722"
Oct 30 22:09:21.033518 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:21.033469    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-83f27219-86849bdb8c-zv722" podStartSLOduration=1.6426629689999999 podStartE2EDuration="2.033456068s" totalImagesPullingTime="390.793099ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:09:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:09:21.03283489 +0000 UTC m=+2473.834647389" watchObservedRunningTime="2026-10-30 22:09:21.033456068 +0000 UTC m=+2473.835268531"
Oct 30 22:09:22.443456 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:22.443417    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 22:09:23.349124 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:23.349077    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-6884d9c49d-kr2ch"]
Oct 30 22:09:23.349338 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:23.349308    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-6884d9c49d-kr2ch" podUID="928b268a-2870-48b8-ae93-2480013abe2a" containerName="maas-api" containerID="cri-o://bd7792d8044b5ca19eafc03e5cece417a42c635e9df6fc95a429d04d40ae1e2e" gracePeriod=30
Oct 30 22:09:24.613754 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:24.613725    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6884d9c49d-kr2ch"
Oct 30 22:09:24.729046 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:24.729011    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/928b268a-2870-48b8-ae93-2480013abe2a-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/928b268a-2870-48b8-ae93-2480013abe2a-maas-api-tls\") pod \"928b268a-2870-48b8-ae93-2480013abe2a\" (UID: \"928b268a-2870-48b8-ae93-2480013abe2a\") "
Oct 30 22:09:24.729253 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:24.729129    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/928b268a-2870-48b8-ae93-2480013abe2a-kube-api-access-bb5pt\" (UniqueName: \"kubernetes.io/projected/928b268a-2870-48b8-ae93-2480013abe2a-kube-api-access-bb5pt\") pod \"928b268a-2870-48b8-ae93-2480013abe2a\" (UID: \"928b268a-2870-48b8-ae93-2480013abe2a\") "
Oct 30 22:09:24.729325 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:24.729272    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/928b268a-2870-48b8-ae93-2480013abe2a-metrics-certs\" (UniqueName: \"kubernetes.io/secret/928b268a-2870-48b8-ae93-2480013abe2a-metrics-certs\") pod \"928b268a-2870-48b8-ae93-2480013abe2a\" (UID: \"928b268a-2870-48b8-ae93-2480013abe2a\") "
Oct 30 22:09:24.729400 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:24.729323    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/928b268a-2870-48b8-ae93-2480013abe2a-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/928b268a-2870-48b8-ae93-2480013abe2a-openshift-service-ca\") pod \"928b268a-2870-48b8-ae93-2480013abe2a\" (UID: \"928b268a-2870-48b8-ae93-2480013abe2a\") "
Oct 30 22:09:24.729804 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:24.729769    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/928b268a-2870-48b8-ae93-2480013abe2a-openshift-service-ca" pod "928b268a-2870-48b8-ae93-2480013abe2a" (UID: "928b268a-2870-48b8-ae93-2480013abe2a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:09:24.731419 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:24.731392    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/928b268a-2870-48b8-ae93-2480013abe2a-maas-api-tls" pod "928b268a-2870-48b8-ae93-2480013abe2a" (UID: "928b268a-2870-48b8-ae93-2480013abe2a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:09:24.731654 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:24.731629    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/928b268a-2870-48b8-ae93-2480013abe2a-metrics-certs" pod "928b268a-2870-48b8-ae93-2480013abe2a" (UID: "928b268a-2870-48b8-ae93-2480013abe2a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:09:24.731764 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:24.731631    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/928b268a-2870-48b8-ae93-2480013abe2a-kube-api-access-bb5pt" pod "928b268a-2870-48b8-ae93-2480013abe2a" (UID: "928b268a-2870-48b8-ae93-2480013abe2a"). InnerVolumeSpecName "kube-api-access-bb5pt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:09:24.831297 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:24.831255    5032 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/928b268a-2870-48b8-ae93-2480013abe2a-metrics-certs\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:24.831297 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:24.831290    5032 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/928b268a-2870-48b8-ae93-2480013abe2a-openshift-service-ca\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:24.831297 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:24.831301    5032 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/928b268a-2870-48b8-ae93-2480013abe2a-maas-api-tls\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:24.831584 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:24.831310    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bb5pt\" (UniqueName: \"kubernetes.io/projected/928b268a-2870-48b8-ae93-2480013abe2a-kube-api-access-bb5pt\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:25.037118 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:25.037026    5032 generic.go:356] "Generic (PLEG): container finished" podID="928b268a-2870-48b8-ae93-2480013abe2a" containerID="bd7792d8044b5ca19eafc03e5cece417a42c635e9df6fc95a429d04d40ae1e2e" exitCode=0
Oct 30 22:09:25.037118 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:25.037061    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6884d9c49d-kr2ch" event={"ID":"928b268a-2870-48b8-ae93-2480013abe2a","Type":"ContainerDied","Data":"bd7792d8044b5ca19eafc03e5cece417a42c635e9df6fc95a429d04d40ae1e2e"}
Oct 30 22:09:25.037118 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:25.037083    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6884d9c49d-kr2ch"
Oct 30 22:09:25.037385 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:25.037145    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6884d9c49d-kr2ch" event={"ID":"928b268a-2870-48b8-ae93-2480013abe2a","Type":"ContainerDied","Data":"bac9246b81e6408741bb550f42a5c5579e4fe5de07bb6aa80cf554a840196157"}
Oct 30 22:09:25.037385 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:25.037171    5032 scope.go:122] "RemoveContainer" containerID="bd7792d8044b5ca19eafc03e5cece417a42c635e9df6fc95a429d04d40ae1e2e"
Oct 30 22:09:25.048813 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:25.048708    5032 scope.go:122] "RemoveContainer" containerID="bd7792d8044b5ca19eafc03e5cece417a42c635e9df6fc95a429d04d40ae1e2e"
Oct 30 22:09:25.049093 ip-10-0-129-195 kubenswrapper[5032]: E1030 22:09:25.049065    5032 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd7792d8044b5ca19eafc03e5cece417a42c635e9df6fc95a429d04d40ae1e2e\": container with ID starting with bd7792d8044b5ca19eafc03e5cece417a42c635e9df6fc95a429d04d40ae1e2e not found: ID does not exist" containerID="bd7792d8044b5ca19eafc03e5cece417a42c635e9df6fc95a429d04d40ae1e2e"
Oct 30 22:09:25.049163 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:25.049103    5032 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd7792d8044b5ca19eafc03e5cece417a42c635e9df6fc95a429d04d40ae1e2e"} err="failed to get container status \"bd7792d8044b5ca19eafc03e5cece417a42c635e9df6fc95a429d04d40ae1e2e\": rpc error: code = NotFound desc = could not find container \"bd7792d8044b5ca19eafc03e5cece417a42c635e9df6fc95a429d04d40ae1e2e\": container with ID starting with bd7792d8044b5ca19eafc03e5cece417a42c635e9df6fc95a429d04d40ae1e2e not found: ID does not exist"
Oct 30 22:09:25.077146 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:25.077105    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-6884d9c49d-kr2ch"]
Oct 30 22:09:25.082838 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:25.082810    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-6884d9c49d-kr2ch"]
Oct 30 22:09:25.345393 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:25.345294    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="928b268a-2870-48b8-ae93-2480013abe2a" path="/var/lib/kubelet/pods/928b268a-2870-48b8-ae93-2480013abe2a/volumes"
Oct 30 22:09:27.021655 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:27.021627    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-83f27219-86849bdb8c-zv722"
Oct 30 22:09:44.513154 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:44.513116    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-83f27219-86849bdb8c-zv722"]
Oct 30 22:09:44.514166 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:44.513427    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-83f27219-86849bdb8c-zv722" podUID="3482a13a-303e-4fc9-af0a-18eb225b2e6f" containerName="maas-api" containerID="cri-o://88e889b6a384f47d37db5596bbe4c4a6fea3ac33a6ef76d5aa44d16075c2bb91" gracePeriod=30
Oct 30 22:09:44.796160 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:44.796135    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-83f27219-86849bdb8c-zv722"
Oct 30 22:09:44.961187 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:44.961154    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3482a13a-303e-4fc9-af0a-18eb225b2e6f-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3482a13a-303e-4fc9-af0a-18eb225b2e6f-maas-api-tls\") pod \"3482a13a-303e-4fc9-af0a-18eb225b2e6f\" (UID: \"3482a13a-303e-4fc9-af0a-18eb225b2e6f\") "
Oct 30 22:09:44.961437 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:44.961396    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3482a13a-303e-4fc9-af0a-18eb225b2e6f-metrics-certs\" (UniqueName: \"kubernetes.io/secret/3482a13a-303e-4fc9-af0a-18eb225b2e6f-metrics-certs\") pod \"3482a13a-303e-4fc9-af0a-18eb225b2e6f\" (UID: \"3482a13a-303e-4fc9-af0a-18eb225b2e6f\") "
Oct 30 22:09:44.961527 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:44.961444    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3482a13a-303e-4fc9-af0a-18eb225b2e6f-kube-api-access-72pbs\" (UniqueName: \"kubernetes.io/projected/3482a13a-303e-4fc9-af0a-18eb225b2e6f-kube-api-access-72pbs\") pod \"3482a13a-303e-4fc9-af0a-18eb225b2e6f\" (UID: \"3482a13a-303e-4fc9-af0a-18eb225b2e6f\") "
Oct 30 22:09:44.961589 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:44.961542    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3482a13a-303e-4fc9-af0a-18eb225b2e6f-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3482a13a-303e-4fc9-af0a-18eb225b2e6f-openshift-service-ca\") pod \"3482a13a-303e-4fc9-af0a-18eb225b2e6f\" (UID: \"3482a13a-303e-4fc9-af0a-18eb225b2e6f\") "
Oct 30 22:09:44.961992 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:44.961966    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3482a13a-303e-4fc9-af0a-18eb225b2e6f-openshift-service-ca" pod "3482a13a-303e-4fc9-af0a-18eb225b2e6f" (UID: "3482a13a-303e-4fc9-af0a-18eb225b2e6f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:09:44.963768 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:44.963741    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3482a13a-303e-4fc9-af0a-18eb225b2e6f-kube-api-access-72pbs" pod "3482a13a-303e-4fc9-af0a-18eb225b2e6f" (UID: "3482a13a-303e-4fc9-af0a-18eb225b2e6f"). InnerVolumeSpecName "kube-api-access-72pbs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:09:44.963880 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:44.963869    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3482a13a-303e-4fc9-af0a-18eb225b2e6f-maas-api-tls" pod "3482a13a-303e-4fc9-af0a-18eb225b2e6f" (UID: "3482a13a-303e-4fc9-af0a-18eb225b2e6f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:09:44.963952 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:44.963916    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3482a13a-303e-4fc9-af0a-18eb225b2e6f-metrics-certs" pod "3482a13a-303e-4fc9-af0a-18eb225b2e6f" (UID: "3482a13a-303e-4fc9-af0a-18eb225b2e6f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:09:45.062700 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:45.062618    5032 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3482a13a-303e-4fc9-af0a-18eb225b2e6f-metrics-certs\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:45.062700 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:45.062648    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-72pbs\" (UniqueName: \"kubernetes.io/projected/3482a13a-303e-4fc9-af0a-18eb225b2e6f-kube-api-access-72pbs\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:45.062700 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:45.062658    5032 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3482a13a-303e-4fc9-af0a-18eb225b2e6f-openshift-service-ca\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:45.062700 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:45.062667    5032 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3482a13a-303e-4fc9-af0a-18eb225b2e6f-maas-api-tls\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:45.158152 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:45.158111    5032 generic.go:356] "Generic (PLEG): container finished" podID="3482a13a-303e-4fc9-af0a-18eb225b2e6f" containerID="88e889b6a384f47d37db5596bbe4c4a6fea3ac33a6ef76d5aa44d16075c2bb91" exitCode=0
Oct 30 22:09:45.158330 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:45.158157    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-83f27219-86849bdb8c-zv722"
Oct 30 22:09:45.158330 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:45.158183    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-83f27219-86849bdb8c-zv722" event={"ID":"3482a13a-303e-4fc9-af0a-18eb225b2e6f","Type":"ContainerDied","Data":"88e889b6a384f47d37db5596bbe4c4a6fea3ac33a6ef76d5aa44d16075c2bb91"}
Oct 30 22:09:45.158330 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:45.158237    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-83f27219-86849bdb8c-zv722" event={"ID":"3482a13a-303e-4fc9-af0a-18eb225b2e6f","Type":"ContainerDied","Data":"ea75d6081c8c061286462741189c73629a46f4a2df4bf5261ba499c85ff17b96"}
Oct 30 22:09:45.158330 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:45.158264    5032 scope.go:122] "RemoveContainer" containerID="88e889b6a384f47d37db5596bbe4c4a6fea3ac33a6ef76d5aa44d16075c2bb91"
Oct 30 22:09:45.169540 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:45.169513    5032 scope.go:122] "RemoveContainer" containerID="88e889b6a384f47d37db5596bbe4c4a6fea3ac33a6ef76d5aa44d16075c2bb91"
Oct 30 22:09:45.169856 ip-10-0-129-195 kubenswrapper[5032]: E1030 22:09:45.169829    5032 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88e889b6a384f47d37db5596bbe4c4a6fea3ac33a6ef76d5aa44d16075c2bb91\": container with ID starting with 88e889b6a384f47d37db5596bbe4c4a6fea3ac33a6ef76d5aa44d16075c2bb91 not found: ID does not exist" containerID="88e889b6a384f47d37db5596bbe4c4a6fea3ac33a6ef76d5aa44d16075c2bb91"
Oct 30 22:09:45.169980 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:45.169864    5032 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88e889b6a384f47d37db5596bbe4c4a6fea3ac33a6ef76d5aa44d16075c2bb91"} err="failed to get container status \"88e889b6a384f47d37db5596bbe4c4a6fea3ac33a6ef76d5aa44d16075c2bb91\": rpc error: code = NotFound desc = could not find container \"88e889b6a384f47d37db5596bbe4c4a6fea3ac33a6ef76d5aa44d16075c2bb91\": container with ID starting with 88e889b6a384f47d37db5596bbe4c4a6fea3ac33a6ef76d5aa44d16075c2bb91 not found: ID does not exist"
Oct 30 22:09:45.184371 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:45.184325    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-83f27219-86849bdb8c-zv722"]
Oct 30 22:09:45.189712 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:45.189684    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-83f27219-86849bdb8c-zv722"]
Oct 30 22:09:45.345678 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:45.345601    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3482a13a-303e-4fc9-af0a-18eb225b2e6f" path="/var/lib/kubelet/pods/3482a13a-303e-4fc9-af0a-18eb225b2e6f/volumes"
Oct 30 22:09:46.210875 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:46.210831    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 22:09:48.344138 ip-10-0-129-195 kubenswrapper[5032]: E1030 22:09:48.344097    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 22:09:57.035320 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:57.035279    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-7766bb-6f557f7c78-zwbr4"]
Oct 30 22:09:57.035942 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:57.035920    5032 cpu_manager.go:414] "containerMap: removing container" podUID="3482a13a-303e-4fc9-af0a-18eb225b2e6f" containerName="maas-api"
Oct 30 22:09:57.036055 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:57.035944    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3482a13a-303e-4fc9-af0a-18eb225b2e6f" containerName="maas-api"
Oct 30 22:09:57.036055 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:57.036017    5032 cpu_manager.go:414] "containerMap: removing container" podUID="928b268a-2870-48b8-ae93-2480013abe2a" containerName="maas-api"
Oct 30 22:09:57.036055 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:57.036027    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="928b268a-2870-48b8-ae93-2480013abe2a" containerName="maas-api"
Oct 30 22:09:57.036220 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:57.036149    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="3482a13a-303e-4fc9-af0a-18eb225b2e6f" containerName="maas-api"
Oct 30 22:09:57.036220 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:57.036180    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="928b268a-2870-48b8-ae93-2480013abe2a" containerName="maas-api"
Oct 30 22:09:57.043227 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:57.042043    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-7766bb-6f557f7c78-zwbr4"
Oct 30 22:09:57.053170 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:57.053123    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-7766bb-6f557f7c78-zwbr4"]
Oct 30 22:09:57.077598 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:57.077556    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-authcfg-http-7766bb-serving-cert\""
Oct 30 22:09:57.090293 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:57.090256    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/41659fd3-4cdf-4bcb-9686-4a4c54a305ef-openshift-service-ca\") pod \"maas-api-e2e-authcfg-http-7766bb-6f557f7c78-zwbr4\" (UID: \"41659fd3-4cdf-4bcb-9686-4a4c54a305ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-7766bb-6f557f7c78-zwbr4"
Oct 30 22:09:57.090495 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:57.090451    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxbdz\" (UniqueName: \"kubernetes.io/projected/41659fd3-4cdf-4bcb-9686-4a4c54a305ef-kube-api-access-hxbdz\") pod \"maas-api-e2e-authcfg-http-7766bb-6f557f7c78-zwbr4\" (UID: \"41659fd3-4cdf-4bcb-9686-4a4c54a305ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-7766bb-6f557f7c78-zwbr4"
Oct 30 22:09:57.090594 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:57.090570    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41659fd3-4cdf-4bcb-9686-4a4c54a305ef-metrics-certs\") pod \"maas-api-e2e-authcfg-http-7766bb-6f557f7c78-zwbr4\" (UID: \"41659fd3-4cdf-4bcb-9686-4a4c54a305ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-7766bb-6f557f7c78-zwbr4"
Oct 30 22:09:57.090707 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:57.090685    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/41659fd3-4cdf-4bcb-9686-4a4c54a305ef-maas-api-tls\") pod \"maas-api-e2e-authcfg-http-7766bb-6f557f7c78-zwbr4\" (UID: \"41659fd3-4cdf-4bcb-9686-4a4c54a305ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-7766bb-6f557f7c78-zwbr4"
Oct 30 22:09:57.191940 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:57.191905    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/41659fd3-4cdf-4bcb-9686-4a4c54a305ef-maas-api-tls\") pod \"maas-api-e2e-authcfg-http-7766bb-6f557f7c78-zwbr4\" (UID: \"41659fd3-4cdf-4bcb-9686-4a4c54a305ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-7766bb-6f557f7c78-zwbr4"
Oct 30 22:09:57.192145 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:57.191956    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/41659fd3-4cdf-4bcb-9686-4a4c54a305ef-openshift-service-ca\") pod \"maas-api-e2e-authcfg-http-7766bb-6f557f7c78-zwbr4\" (UID: \"41659fd3-4cdf-4bcb-9686-4a4c54a305ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-7766bb-6f557f7c78-zwbr4"
Oct 30 22:09:57.192145 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:57.192008    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxbdz\" (UniqueName: \"kubernetes.io/projected/41659fd3-4cdf-4bcb-9686-4a4c54a305ef-kube-api-access-hxbdz\") pod \"maas-api-e2e-authcfg-http-7766bb-6f557f7c78-zwbr4\" (UID: \"41659fd3-4cdf-4bcb-9686-4a4c54a305ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-7766bb-6f557f7c78-zwbr4"
Oct 30 22:09:57.192145 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:57.192048    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41659fd3-4cdf-4bcb-9686-4a4c54a305ef-metrics-certs\") pod \"maas-api-e2e-authcfg-http-7766bb-6f557f7c78-zwbr4\" (UID: \"41659fd3-4cdf-4bcb-9686-4a4c54a305ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-7766bb-6f557f7c78-zwbr4"
Oct 30 22:09:57.192711 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:57.192684    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/41659fd3-4cdf-4bcb-9686-4a4c54a305ef-openshift-service-ca\") pod \"maas-api-e2e-authcfg-http-7766bb-6f557f7c78-zwbr4\" (UID: \"41659fd3-4cdf-4bcb-9686-4a4c54a305ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-7766bb-6f557f7c78-zwbr4"
Oct 30 22:09:57.194524 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:57.194502    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/41659fd3-4cdf-4bcb-9686-4a4c54a305ef-maas-api-tls\") pod \"maas-api-e2e-authcfg-http-7766bb-6f557f7c78-zwbr4\" (UID: \"41659fd3-4cdf-4bcb-9686-4a4c54a305ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-7766bb-6f557f7c78-zwbr4"
Oct 30 22:09:57.194677 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:57.194651    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41659fd3-4cdf-4bcb-9686-4a4c54a305ef-metrics-certs\") pod \"maas-api-e2e-authcfg-http-7766bb-6f557f7c78-zwbr4\" (UID: \"41659fd3-4cdf-4bcb-9686-4a4c54a305ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-7766bb-6f557f7c78-zwbr4"
Oct 30 22:09:57.206954 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:57.206921    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxbdz\" (UniqueName: \"kubernetes.io/projected/41659fd3-4cdf-4bcb-9686-4a4c54a305ef-kube-api-access-hxbdz\") pod \"maas-api-e2e-authcfg-http-7766bb-6f557f7c78-zwbr4\" (UID: \"41659fd3-4cdf-4bcb-9686-4a4c54a305ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-7766bb-6f557f7c78-zwbr4"
Oct 30 22:09:57.364644 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:57.364552    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-7766bb-6f557f7c78-zwbr4"
Oct 30 22:09:57.672413 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:57.672379    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-7766bb-6f557f7c78-zwbr4"]
Oct 30 22:09:57.673345 ip-10-0-129-195 kubenswrapper[5032]: W1030 22:09:57.673315    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41659fd3_4cdf_4bcb_9686_4a4c54a305ef.slice/crio-0e9c7877e392b95180f7e6615a4b189430386ea003d856aebd0afa46851df598 WatchSource:0}: Error finding container 0e9c7877e392b95180f7e6615a4b189430386ea003d856aebd0afa46851df598: Status 404 returned error can't find the container with id 0e9c7877e392b95180f7e6615a4b189430386ea003d856aebd0afa46851df598
Oct 30 22:09:58.235565 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:58.235515    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-7766bb-6f557f7c78-zwbr4" event={"ID":"41659fd3-4cdf-4bcb-9686-4a4c54a305ef","Type":"ContainerStarted","Data":"94154f281ecd78c831a38d63380024f0d9aab0c33836ae8bd2696b9b44e9a129"}
Oct 30 22:09:58.236046 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:58.235581    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-7766bb-6f557f7c78-zwbr4" event={"ID":"41659fd3-4cdf-4bcb-9686-4a4c54a305ef","Type":"ContainerStarted","Data":"0e9c7877e392b95180f7e6615a4b189430386ea003d856aebd0afa46851df598"}
Oct 30 22:09:58.236046 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:58.235608    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-7766bb-6f557f7c78-zwbr4"
Oct 30 22:09:58.256880 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:58.256826    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-7766bb-6f557f7c78-zwbr4" podStartSLOduration=0.863937192 podStartE2EDuration="1.25681133s" totalImagesPullingTime="392.874138ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:09:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:09:58.255849423 +0000 UTC m=+2511.057661869" watchObservedRunningTime="2026-10-30 22:09:58.25681133 +0000 UTC m=+2511.058623794"
Oct 30 22:09:59.995296 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:09:59.995260    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 22:10:04.246775 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:10:04.246743    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-7766bb-6f557f7c78-zwbr4"
Oct 30 22:10:18.345115 ip-10-0-129-195 kubenswrapper[5032]: E1030 22:10:18.345078    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 22:10:48.347297 ip-10-0-129-195 kubenswrapper[5032]: E1030 22:10:48.347213    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 22:11:18.344529 ip-10-0-129-195 kubenswrapper[5032]: E1030 22:11:18.344488    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 22:11:46.441130 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:11:46.441082    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-7766bb-6f557f7c78-zwbr4"]
Oct 30 22:11:46.441818 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:11:46.441342    5032 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-7766bb-6f557f7c78-zwbr4" podUID="41659fd3-4cdf-4bcb-9686-4a4c54a305ef" containerName="maas-api" containerID="cri-o://94154f281ecd78c831a38d63380024f0d9aab0c33836ae8bd2696b9b44e9a129" gracePeriod=30
Oct 30 22:11:46.723045 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:11:46.723012    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-7766bb-6f557f7c78-zwbr4"
Oct 30 22:11:46.746879 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:11:46.746832    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/41659fd3-4cdf-4bcb-9686-4a4c54a305ef-metrics-certs\" (UniqueName: \"kubernetes.io/secret/41659fd3-4cdf-4bcb-9686-4a4c54a305ef-metrics-certs\") pod \"41659fd3-4cdf-4bcb-9686-4a4c54a305ef\" (UID: \"41659fd3-4cdf-4bcb-9686-4a4c54a305ef\") "
Oct 30 22:11:46.747087 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:11:46.746939    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/41659fd3-4cdf-4bcb-9686-4a4c54a305ef-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/41659fd3-4cdf-4bcb-9686-4a4c54a305ef-openshift-service-ca\") pod \"41659fd3-4cdf-4bcb-9686-4a4c54a305ef\" (UID: \"41659fd3-4cdf-4bcb-9686-4a4c54a305ef\") "
Oct 30 22:11:46.747087 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:11:46.747010    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/41659fd3-4cdf-4bcb-9686-4a4c54a305ef-kube-api-access-hxbdz\" (UniqueName: \"kubernetes.io/projected/41659fd3-4cdf-4bcb-9686-4a4c54a305ef-kube-api-access-hxbdz\") pod \"41659fd3-4cdf-4bcb-9686-4a4c54a305ef\" (UID: \"41659fd3-4cdf-4bcb-9686-4a4c54a305ef\") "
Oct 30 22:11:46.747087 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:11:46.747051    5032 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/41659fd3-4cdf-4bcb-9686-4a4c54a305ef-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/41659fd3-4cdf-4bcb-9686-4a4c54a305ef-maas-api-tls\") pod \"41659fd3-4cdf-4bcb-9686-4a4c54a305ef\" (UID: \"41659fd3-4cdf-4bcb-9686-4a4c54a305ef\") "
Oct 30 22:11:46.752761 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:11:46.752250    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41659fd3-4cdf-4bcb-9686-4a4c54a305ef-kube-api-access-hxbdz" pod "41659fd3-4cdf-4bcb-9686-4a4c54a305ef" (UID: "41659fd3-4cdf-4bcb-9686-4a4c54a305ef"). InnerVolumeSpecName "kube-api-access-hxbdz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:11:46.759615 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:11:46.759556    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41659fd3-4cdf-4bcb-9686-4a4c54a305ef-openshift-service-ca" pod "41659fd3-4cdf-4bcb-9686-4a4c54a305ef" (UID: "41659fd3-4cdf-4bcb-9686-4a4c54a305ef"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:11:46.764417 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:11:46.761381    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41659fd3-4cdf-4bcb-9686-4a4c54a305ef-metrics-certs" pod "41659fd3-4cdf-4bcb-9686-4a4c54a305ef" (UID: "41659fd3-4cdf-4bcb-9686-4a4c54a305ef"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:11:46.765935 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:11:46.765902    5032 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41659fd3-4cdf-4bcb-9686-4a4c54a305ef-maas-api-tls" pod "41659fd3-4cdf-4bcb-9686-4a4c54a305ef" (UID: "41659fd3-4cdf-4bcb-9686-4a4c54a305ef"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:11:46.848340 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:11:46.848302    5032 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hxbdz\" (UniqueName: \"kubernetes.io/projected/41659fd3-4cdf-4bcb-9686-4a4c54a305ef-kube-api-access-hxbdz\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:11:46.848340 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:11:46.848338    5032 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/41659fd3-4cdf-4bcb-9686-4a4c54a305ef-maas-api-tls\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:11:46.848340 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:11:46.848348    5032 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41659fd3-4cdf-4bcb-9686-4a4c54a305ef-metrics-certs\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:11:46.848592 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:11:46.848372    5032 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/41659fd3-4cdf-4bcb-9686-4a4c54a305ef-openshift-service-ca\") on node \"ip-10-0-129-195.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:11:46.871173 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:11:46.871132    5032 generic.go:356] "Generic (PLEG): container finished" podID="41659fd3-4cdf-4bcb-9686-4a4c54a305ef" containerID="94154f281ecd78c831a38d63380024f0d9aab0c33836ae8bd2696b9b44e9a129" exitCode=0
Oct 30 22:11:46.871173 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:11:46.871173    5032 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-7766bb-6f557f7c78-zwbr4"
Oct 30 22:11:46.871439 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:11:46.871215    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-7766bb-6f557f7c78-zwbr4" event={"ID":"41659fd3-4cdf-4bcb-9686-4a4c54a305ef","Type":"ContainerDied","Data":"94154f281ecd78c831a38d63380024f0d9aab0c33836ae8bd2696b9b44e9a129"}
Oct 30 22:11:46.871439 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:11:46.871270    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-7766bb-6f557f7c78-zwbr4" event={"ID":"41659fd3-4cdf-4bcb-9686-4a4c54a305ef","Type":"ContainerDied","Data":"0e9c7877e392b95180f7e6615a4b189430386ea003d856aebd0afa46851df598"}
Oct 30 22:11:46.871439 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:11:46.871292    5032 scope.go:122] "RemoveContainer" containerID="94154f281ecd78c831a38d63380024f0d9aab0c33836ae8bd2696b9b44e9a129"
Oct 30 22:11:46.881617 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:11:46.881597    5032 scope.go:122] "RemoveContainer" containerID="94154f281ecd78c831a38d63380024f0d9aab0c33836ae8bd2696b9b44e9a129"
Oct 30 22:11:46.881927 ip-10-0-129-195 kubenswrapper[5032]: E1030 22:11:46.881901    5032 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94154f281ecd78c831a38d63380024f0d9aab0c33836ae8bd2696b9b44e9a129\": container with ID starting with 94154f281ecd78c831a38d63380024f0d9aab0c33836ae8bd2696b9b44e9a129 not found: ID does not exist" containerID="94154f281ecd78c831a38d63380024f0d9aab0c33836ae8bd2696b9b44e9a129"
Oct 30 22:11:46.881992 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:11:46.881936    5032 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94154f281ecd78c831a38d63380024f0d9aab0c33836ae8bd2696b9b44e9a129"} err="failed to get container status \"94154f281ecd78c831a38d63380024f0d9aab0c33836ae8bd2696b9b44e9a129\": rpc error: code = NotFound desc = could not find container \"94154f281ecd78c831a38d63380024f0d9aab0c33836ae8bd2696b9b44e9a129\": container with ID starting with 94154f281ecd78c831a38d63380024f0d9aab0c33836ae8bd2696b9b44e9a129 not found: ID does not exist"
Oct 30 22:11:46.895475 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:11:46.895439    5032 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-7766bb-6f557f7c78-zwbr4"]
Oct 30 22:11:46.900147 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:11:46.900115    5032 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-7766bb-6f557f7c78-zwbr4"]
Oct 30 22:11:47.345181 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:11:47.345149    5032 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="41659fd3-4cdf-4bcb-9686-4a4c54a305ef" path="/var/lib/kubelet/pods/41659fd3-4cdf-4bcb-9686-4a4c54a305ef/volumes"
Oct 30 22:11:48.345616 ip-10-0-129-195 kubenswrapper[5032]: E1030 22:11:48.345577    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 22:11:48.520959 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:11:48.520916    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 22:11:55.705796 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:11:55.705762    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 22:12:01.000318 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:12:01.000285    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 22:12:16.106000 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:12:16.105965    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 22:12:18.357565 ip-10-0-129-195 kubenswrapper[5032]: E1030 22:12:18.357462    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 22:12:42.917127 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:12:42.917080    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-jxn67"]
Oct 30 22:12:48.344456 ip-10-0-129-195 kubenswrapper[5032]: E1030 22:12:48.344417    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 22:12:54.321976 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:12:54.321942    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-57889585d4-5rjwf_4dc7efd7-d1c8-48da-b76f-9faae51f01e2/manager/0.log"
Oct 30 22:12:54.532139 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:12:54.532108    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-5mx94_7a16205c-fa5e-4c19-bf95-122f14d59b1d/manager/0.log"
Oct 30 22:12:54.712905 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:12:54.712873    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-6nnzj_c369fb18-590a-4da3-93ff-f0da6ae518e2/manager/0.log"
Oct 30 22:12:55.372547 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:12:55.372517    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-wrl7s_9ef57dfb-240c-42e2-bc9d-c68c63f0a11c/server/0.log"
Oct 30 22:12:55.545548 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:12:55.545519    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-gs699_60408246-d50f-492a-9ade-9f28c3f9ad2c/manager/0.log"
Oct 30 22:12:56.713066 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:12:56.713033    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-6fdb49dd46-zp66m_afa4bfff-93da-4cdc-abdf-303292d1656c/maas-api/0.log"
Oct 30 22:12:58.041893 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:12:58.041858    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-7cdbf6dc44-mlq7r_5d55ecfe-deb9-40d1-bdbb-2ad4c0d67f4b/postgres/0.log"
Oct 30 22:13:00.020149 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:13:00.020105    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-xtcms_bd9151d1-084d-4737-b9ce-cbac709e23c1/manager/0.log"
Oct 30 22:13:00.207431 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:13:00.207404    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-zh266_a70b450b-81a2-4059-aed2-6beebb186a95/kuadrant-console-plugin/0.log"
Oct 30 22:13:00.402878 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:13:00.402773    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-wv78j_4b7d8b7c-acb7-4d75-8377-db8e0a0b1a4b/manager/0.log"
Oct 30 22:13:00.628619 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:13:00.628586    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-jxn67_4b0e32e7-f487-4b95-8557-5302fa36d72b/limitador/0.log"
Oct 30 22:13:00.825294 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:13:00.825253    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5778bb4449-m85ql_0900d338-ad69-49e6-b23d-185def51a7cb/manager/0.log"
Oct 30 22:13:01.626648 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:13:01.626600    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6b748b65852wbfw_3f48d971-e68f-4bb2-8c83-0f2acf541950/istio-proxy/0.log"
Oct 30 22:13:02.392501 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:13:02.392469    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75fd9f5d47-7s44p_457cd47c-9543-4d82-875f-f1e45d6d5b70/istio-proxy/0.log"
Oct 30 22:13:02.971708 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:13:02.971674    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75fd9f5d47-c46gq_e3b353c1-3812-44aa-b77d-d7ddfc3b7057/istio-proxy/0.log"
Oct 30 22:13:03.712146 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:13:03.712119    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75fd9f5d47-mxsln_f77afa6c-51f5-4bc2-b10e-89101bd067c8/istio-proxy/0.log"
Oct 30 22:13:04.316084 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:13:04.316050    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj_6b114f93-6494-464c-bfec-b2f7386f353f/istio-proxy/0.log"
Oct 30 22:13:04.867005 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:13:04.866975    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5bbfd9465-x28lh_9b4f9524-db75-4206-aa76-04400d95706f/router/0.log"
Oct 30 22:13:05.670591 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:13:05.670558    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-cbd4f5586-jx4qg_0c945ece-1ac0-4993-aea4-e853a23a4099/main/0.log"
Oct 30 22:13:05.858076 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:13:05.858049    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-7b998b79d9-xv7qv_4f6d23b8-1364-4b03-96c6-6649f3a0644a/main/0.log"
Oct 30 22:13:06.238783 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:13:06.238752    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-69b588897d-d9q4t_294414e8-4fc5-48ec-9c47-9660381fda80/main/0.log"
Oct 30 22:13:06.596771 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:13:06.596692    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-vg752_8fa31603-e280-4566-bded-aaebc80c2179/main/0.log"
Oct 30 22:13:10.183528 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:13:10.183375    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-sklxd_67292bdc-7fe1-449d-af75-2baa90c16ea9/console-operator/1.log"
Oct 30 22:13:10.323527 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:13:10.323501    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-sklxd_67292bdc-7fe1-449d-af75-2baa90c16ea9/console-operator/1.log"
Oct 30 22:13:17.186934 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:13:17.186903    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-57889585d4-5rjwf_4dc7efd7-d1c8-48da-b76f-9faae51f01e2/manager/0.log"
Oct 30 22:13:17.392102 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:13:17.392068    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-5mx94_7a16205c-fa5e-4c19-bf95-122f14d59b1d/manager/0.log"
Oct 30 22:13:17.577818 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:13:17.577778    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-6nnzj_c369fb18-590a-4da3-93ff-f0da6ae518e2/manager/0.log"
Oct 30 22:13:18.257230 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:13:18.257205    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-wrl7s_9ef57dfb-240c-42e2-bc9d-c68c63f0a11c/server/0.log"
Oct 30 22:13:18.345077 ip-10-0-129-195 kubenswrapper[5032]: E1030 22:13:18.345042    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 22:13:18.443722 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:13:18.443683    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-gs699_60408246-d50f-492a-9ade-9f28c3f9ad2c/manager/0.log"
Oct 30 22:13:19.442116 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:13:19.442091    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-6fdb49dd46-zp66m_afa4bfff-93da-4cdc-abdf-303292d1656c/maas-api/0.log"
Oct 30 22:13:20.768533 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:13:20.768496    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-7cdbf6dc44-mlq7r_5d55ecfe-deb9-40d1-bdbb-2ad4c0d67f4b/postgres/0.log"
Oct 30 22:13:22.370045 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:13:22.370008    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-xtcms_bd9151d1-084d-4737-b9ce-cbac709e23c1/manager/0.log"
Oct 30 22:13:22.550208 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:13:22.550180    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-zh266_a70b450b-81a2-4059-aed2-6beebb186a95/kuadrant-console-plugin/0.log"
Oct 30 22:13:22.742483 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:13:22.742448    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-wv78j_4b7d8b7c-acb7-4d75-8377-db8e0a0b1a4b/manager/0.log"
Oct 30 22:13:22.959593 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:13:22.959563    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-jxn67_4b0e32e7-f487-4b95-8557-5302fa36d72b/limitador/0.log"
Oct 30 22:13:23.145248 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:13:23.145173    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5778bb4449-m85ql_0900d338-ad69-49e6-b23d-185def51a7cb/manager/0.log"
Oct 30 22:13:23.724372 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:13:23.724328    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6b748b65852wbfw_3f48d971-e68f-4bb2-8c83-0f2acf541950/istio-proxy/0.log"
Oct 30 22:13:24.467449 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:13:24.467418    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75fd9f5d47-7s44p_457cd47c-9543-4d82-875f-f1e45d6d5b70/istio-proxy/0.log"
Oct 30 22:13:25.026460 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:13:25.026428    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75fd9f5d47-c46gq_e3b353c1-3812-44aa-b77d-d7ddfc3b7057/istio-proxy/0.log"
Oct 30 22:13:25.768032 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:13:25.767999    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75fd9f5d47-mxsln_f77afa6c-51f5-4bc2-b10e-89101bd067c8/istio-proxy/0.log"
Oct 30 22:13:26.351500 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:13:26.351469    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj_6b114f93-6494-464c-bfec-b2f7386f353f/istio-proxy/0.log"
Oct 30 22:13:26.907626 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:13:26.907596    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5bbfd9465-x28lh_9b4f9524-db75-4206-aa76-04400d95706f/router/0.log"
Oct 30 22:13:27.433954 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:13:27.433927    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-cbd4f5586-jx4qg_0c945ece-1ac0-4993-aea4-e853a23a4099/main/0.log"
Oct 30 22:13:27.621429 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:13:27.621386    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-7b998b79d9-xv7qv_4f6d23b8-1364-4b03-96c6-6649f3a0644a/main/0.log"
Oct 30 22:13:27.971667 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:13:27.971640    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-69b588897d-d9q4t_294414e8-4fc5-48ec-9c47-9660381fda80/main/0.log"
Oct 30 22:13:28.330969 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:13:28.330883    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-vg752_8fa31603-e280-4566-bded-aaebc80c2179/main/0.log"
Oct 30 22:13:48.344921 ip-10-0-129-195 kubenswrapper[5032]: E1030 22:13:48.344842    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 22:14:18.347047 ip-10-0-129-195 kubenswrapper[5032]: E1030 22:14:18.347007    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 22:14:48.345329 ip-10-0-129-195 kubenswrapper[5032]: E1030 22:14:48.345289    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 22:15:02.578608 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:02.578558    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-csbxh_5f171661-33a2-4631-81c7-bc10a5db791f/global-pull-secret-syncer/0.log"
Oct 30 22:15:02.698954 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:02.698923    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-9mt44_3c835b41-25cb-4f0c-8e7c-4ac9e2fee318/connection-checker/0.log"
Oct 30 22:15:02.725119 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:02.725097    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-l4dbk_fb314739-c678-4da8-9d54-147dc0612ee2/connection-checker/0.log"
Oct 30 22:15:02.753006 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:02.752980    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-zvqvt_0d83c70b-fab8-4872-969f-11f74c0ddea0/connection-checker/0.log"
Oct 30 22:15:02.847733 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:02.847629    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-l8g24_ee6de179-7e35-46a5-86e1-c7c31428c149/konnectivity-agent/0.log"
Oct 30 22:15:02.911417 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:02.911353    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-195.us-east-2.compute.internal_7c1602df5a81ac24aa1efba1059dbbe8/haproxy/0.log"
Oct 30 22:15:09.603476 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:09.603441    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-xtcms_bd9151d1-084d-4737-b9ce-cbac709e23c1/manager/0.log"
Oct 30 22:15:09.635508 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:09.635483    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-zh266_a70b450b-81a2-4059-aed2-6beebb186a95/kuadrant-console-plugin/0.log"
Oct 30 22:15:09.698506 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:09.698475    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-wv78j_4b7d8b7c-acb7-4d75-8377-db8e0a0b1a4b/manager/0.log"
Oct 30 22:15:09.733926 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:09.733896    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-jxn67_4b0e32e7-f487-4b95-8557-5302fa36d72b/limitador/0.log"
Oct 30 22:15:09.798131 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:09.798098    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5778bb4449-m85ql_0900d338-ad69-49e6-b23d-185def51a7cb/manager/0.log"
Oct 30 22:15:11.328631 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:11.328590    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-884576c57-kdrrd_90937bc9-0051-407d-9c82-014f0a9c65d4/cluster-monitoring-operator/0.log"
Oct 30 22:15:11.622252 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:11.622159    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ggbl6_88eb2d7e-b5dd-4769-b30c-756140a8cc1d/node-exporter/0.log"
Oct 30 22:15:11.646136 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:11.646112    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ggbl6_88eb2d7e-b5dd-4769-b30c-756140a8cc1d/kube-rbac-proxy/0.log"
Oct 30 22:15:11.672603 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:11.672576    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ggbl6_88eb2d7e-b5dd-4769-b30c-756140a8cc1d/init-textfile/0.log"
Oct 30 22:15:11.756126 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:11.756096    5032 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-vrzqz/perf-node-gather-daemonset-x2mxs"]
Oct 30 22:15:11.756600 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:11.756585    5032 cpu_manager.go:414] "containerMap: removing container" podUID="41659fd3-4cdf-4bcb-9686-4a4c54a305ef" containerName="maas-api"
Oct 30 22:15:11.756652 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:11.756602    5032 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="41659fd3-4cdf-4bcb-9686-4a4c54a305ef" containerName="maas-api"
Oct 30 22:15:11.756708 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:11.756699    5032 memory_manager.go:365] "RemoveStaleState removing state" podUID="41659fd3-4cdf-4bcb-9686-4a4c54a305ef" containerName="maas-api"
Oct 30 22:15:11.759898 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:11.759877    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vrzqz/perf-node-gather-daemonset-x2mxs"
Oct 30 22:15:11.766955 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:11.766933    5032 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-vrzqz\"/\"default-dockercfg-p8sks\""
Oct 30 22:15:11.774523 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:11.774499    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vrzqz/perf-node-gather-daemonset-x2mxs"]
Oct 30 22:15:11.783401 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:11.783341    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vrzqz\"/\"kube-root-ca.crt\""
Oct 30 22:15:11.784810 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:11.784783    5032 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vrzqz\"/\"openshift-service-ca.crt\""
Oct 30 22:15:11.881865 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:11.881784    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3ac84756-dcbc-4152-be66-b89753446385-proc\") pod \"perf-node-gather-daemonset-x2mxs\" (UID: \"3ac84756-dcbc-4152-be66-b89753446385\") " pod="openshift-must-gather-vrzqz/perf-node-gather-daemonset-x2mxs"
Oct 30 22:15:11.881865 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:11.881819    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7t8h\" (UniqueName: \"kubernetes.io/projected/3ac84756-dcbc-4152-be66-b89753446385-kube-api-access-c7t8h\") pod \"perf-node-gather-daemonset-x2mxs\" (UID: \"3ac84756-dcbc-4152-be66-b89753446385\") " pod="openshift-must-gather-vrzqz/perf-node-gather-daemonset-x2mxs"
Oct 30 22:15:11.882051 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:11.881952    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3ac84756-dcbc-4152-be66-b89753446385-sys\") pod \"perf-node-gather-daemonset-x2mxs\" (UID: \"3ac84756-dcbc-4152-be66-b89753446385\") " pod="openshift-must-gather-vrzqz/perf-node-gather-daemonset-x2mxs"
Oct 30 22:15:11.882051 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:11.881974    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3ac84756-dcbc-4152-be66-b89753446385-lib-modules\") pod \"perf-node-gather-daemonset-x2mxs\" (UID: \"3ac84756-dcbc-4152-be66-b89753446385\") " pod="openshift-must-gather-vrzqz/perf-node-gather-daemonset-x2mxs"
Oct 30 22:15:11.882051 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:11.882007    5032 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3ac84756-dcbc-4152-be66-b89753446385-podres\") pod \"perf-node-gather-daemonset-x2mxs\" (UID: \"3ac84756-dcbc-4152-be66-b89753446385\") " pod="openshift-must-gather-vrzqz/perf-node-gather-daemonset-x2mxs"
Oct 30 22:15:11.982675 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:11.982635    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3ac84756-dcbc-4152-be66-b89753446385-sys\") pod \"perf-node-gather-daemonset-x2mxs\" (UID: \"3ac84756-dcbc-4152-be66-b89753446385\") " pod="openshift-must-gather-vrzqz/perf-node-gather-daemonset-x2mxs"
Oct 30 22:15:11.982873 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:11.982684    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3ac84756-dcbc-4152-be66-b89753446385-lib-modules\") pod \"perf-node-gather-daemonset-x2mxs\" (UID: \"3ac84756-dcbc-4152-be66-b89753446385\") " pod="openshift-must-gather-vrzqz/perf-node-gather-daemonset-x2mxs"
Oct 30 22:15:11.982873 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:11.982735    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3ac84756-dcbc-4152-be66-b89753446385-podres\") pod \"perf-node-gather-daemonset-x2mxs\" (UID: \"3ac84756-dcbc-4152-be66-b89753446385\") " pod="openshift-must-gather-vrzqz/perf-node-gather-daemonset-x2mxs"
Oct 30 22:15:11.982873 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:11.982770    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3ac84756-dcbc-4152-be66-b89753446385-sys\") pod \"perf-node-gather-daemonset-x2mxs\" (UID: \"3ac84756-dcbc-4152-be66-b89753446385\") " pod="openshift-must-gather-vrzqz/perf-node-gather-daemonset-x2mxs"
Oct 30 22:15:11.982873 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:11.982789    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3ac84756-dcbc-4152-be66-b89753446385-proc\") pod \"perf-node-gather-daemonset-x2mxs\" (UID: \"3ac84756-dcbc-4152-be66-b89753446385\") " pod="openshift-must-gather-vrzqz/perf-node-gather-daemonset-x2mxs"
Oct 30 22:15:11.982873 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:11.982814    5032 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7t8h\" (UniqueName: \"kubernetes.io/projected/3ac84756-dcbc-4152-be66-b89753446385-kube-api-access-c7t8h\") pod \"perf-node-gather-daemonset-x2mxs\" (UID: \"3ac84756-dcbc-4152-be66-b89753446385\") " pod="openshift-must-gather-vrzqz/perf-node-gather-daemonset-x2mxs"
Oct 30 22:15:11.982873 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:11.982848    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3ac84756-dcbc-4152-be66-b89753446385-podres\") pod \"perf-node-gather-daemonset-x2mxs\" (UID: \"3ac84756-dcbc-4152-be66-b89753446385\") " pod="openshift-must-gather-vrzqz/perf-node-gather-daemonset-x2mxs"
Oct 30 22:15:11.982873 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:11.982852    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3ac84756-dcbc-4152-be66-b89753446385-lib-modules\") pod \"perf-node-gather-daemonset-x2mxs\" (UID: \"3ac84756-dcbc-4152-be66-b89753446385\") " pod="openshift-must-gather-vrzqz/perf-node-gather-daemonset-x2mxs"
Oct 30 22:15:11.983131 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:11.982957    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3ac84756-dcbc-4152-be66-b89753446385-proc\") pod \"perf-node-gather-daemonset-x2mxs\" (UID: \"3ac84756-dcbc-4152-be66-b89753446385\") " pod="openshift-must-gather-vrzqz/perf-node-gather-daemonset-x2mxs"
Oct 30 22:15:11.995611 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:11.995590    5032 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7t8h\" (UniqueName: \"kubernetes.io/projected/3ac84756-dcbc-4152-be66-b89753446385-kube-api-access-c7t8h\") pod \"perf-node-gather-daemonset-x2mxs\" (UID: \"3ac84756-dcbc-4152-be66-b89753446385\") " pod="openshift-must-gather-vrzqz/perf-node-gather-daemonset-x2mxs"
Oct 30 22:15:12.076093 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:12.076059    5032 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vrzqz/perf-node-gather-daemonset-x2mxs"
Oct 30 22:15:12.195109 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:12.195076    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-6489d8bf89-kgc78_40aace49-bddf-4a7f-aaaa-92fec5204b1c/prometheus-operator-admission-webhook/0.log"
Oct 30 22:15:12.355134 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:12.355103    5032 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vrzqz/perf-node-gather-daemonset-x2mxs"]
Oct 30 22:15:12.357523 ip-10-0-129-195 kubenswrapper[5032]: W1030 22:15:12.357495    5032 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod3ac84756_dcbc_4152_be66_b89753446385.slice/crio-493cb156f2658f38e06d3567c38ef2c340baeaecea77f129251a629833654813 WatchSource:0}: Error finding container 493cb156f2658f38e06d3567c38ef2c340baeaecea77f129251a629833654813: Status 404 returned error can't find the container with id 493cb156f2658f38e06d3567c38ef2c340baeaecea77f129251a629833654813
Oct 30 22:15:13.047921 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:13.047881    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vrzqz/perf-node-gather-daemonset-x2mxs" event={"ID":"3ac84756-dcbc-4152-be66-b89753446385","Type":"ContainerStarted","Data":"0dd74a52adabbeadda0e08435325214b4ee99730126115097bb117a955fa8239"}
Oct 30 22:15:13.048392 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:13.047942    5032 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vrzqz/perf-node-gather-daemonset-x2mxs" event={"ID":"3ac84756-dcbc-4152-be66-b89753446385","Type":"ContainerStarted","Data":"493cb156f2658f38e06d3567c38ef2c340baeaecea77f129251a629833654813"}
Oct 30 22:15:13.048392 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:13.048007    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-vrzqz/perf-node-gather-daemonset-x2mxs"
Oct 30 22:15:13.069081 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:13.069034    5032 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-vrzqz/perf-node-gather-daemonset-x2mxs" podStartSLOduration=2.069019758 podStartE2EDuration="2.069019758s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 22:15:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:15:13.066592538 +0000 UTC m=+2825.868405003" watchObservedRunningTime="2026-10-30 22:15:13.069019758 +0000 UTC m=+2825.870832222"
Oct 30 22:15:13.666602 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:13.666569    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-86c478bb88-lwm89_753d4639-09e6-4850-9f4d-ed39b0406439/networking-console-plugin/0.log"
Oct 30 22:15:14.285973 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:14.285934    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-sklxd_67292bdc-7fe1-449d-af75-2baa90c16ea9/console-operator/1.log"
Oct 30 22:15:14.291202 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:14.291182    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-sklxd_67292bdc-7fe1-449d-af75-2baa90c16ea9/console-operator/2.log"
Oct 30 22:15:14.821890 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:14.821863    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-6dd789c8b7-qk429_ea11393a-a054-4a6d-9194-156a7adc6c7e/download-server/0.log"
Oct 30 22:15:15.451283 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:15.451253    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-58c9f5f46d-lbjhv_e9d2f04b-bb7d-422f-b497-5fa61580f95a/volume-data-source-validator/0.log"
Oct 30 22:15:16.691493 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:16.691451    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-sbhch_08498890-fb63-4aea-9cf7-a2ade5397df2/dns/0.log"
Oct 30 22:15:16.719918 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:16.719874    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-sbhch_08498890-fb63-4aea-9cf7-a2ade5397df2/kube-rbac-proxy/0.log"
Oct 30 22:15:16.811769 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:16.811742    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-97tsr_baa8d0c0-d477-4f30-a0f6-375fc84cc448/dns-node-resolver/0.log"
Oct 30 22:15:17.556746 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:17.556708    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-848d9d9bc8-mjlrc_cf2eedc0-044e-46d4-ab33-71c5b6d211b8/registry/0.log"
Oct 30 22:15:17.656893 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:17.656866    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-jgns8_48efa9dc-3c20-4f81-86f3-c87ad697772c/node-ca/0.log"
Oct 30 22:15:18.344005 ip-10-0-129-195 kubenswrapper[5032]: E1030 22:15:18.343968    5032 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-z2dd4" containerName="extractor"
Oct 30 22:15:18.794013 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:18.793983    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6b748b65852wbfw_3f48d971-e68f-4bb2-8c83-0f2acf541950/istio-proxy/0.log"
Oct 30 22:15:19.066913 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:19.066809    5032 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-vrzqz/perf-node-gather-daemonset-x2mxs"
Oct 30 22:15:20.506865 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:20.506838    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75fd9f5d47-7s44p_457cd47c-9543-4d82-875f-f1e45d6d5b70/istio-proxy/0.log"
Oct 30 22:15:20.614493 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:20.614465    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75fd9f5d47-c46gq_e3b353c1-3812-44aa-b77d-d7ddfc3b7057/istio-proxy/0.log"
Oct 30 22:15:20.768336 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:20.768262    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75fd9f5d47-mxsln_f77afa6c-51f5-4bc2-b10e-89101bd067c8/istio-proxy/0.log"
Oct 30 22:15:20.869088 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:20.869062    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj_6b114f93-6494-464c-bfec-b2f7386f353f/istio-proxy/0.log"
Oct 30 22:15:20.989114 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:20.989082    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5bbfd9465-x28lh_9b4f9524-db75-4206-aa76-04400d95706f/router/0.log"
Oct 30 22:15:22.343239 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:22.343193    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6b748b65852wbfw_3f48d971-e68f-4bb2-8c83-0f2acf541950/istio-proxy/0.log"
Oct 30 22:15:22.999056 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:22.999018    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hzvrj_3175ff47-1038-4139-9c0a-87062bb38dc2/serve-healthcheck-canary/0.log"
Oct 30 22:15:23.392493 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:23.392407    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75fd9f5d47-7s44p_457cd47c-9543-4d82-875f-f1e45d6d5b70/istio-proxy/0.log"
Oct 30 22:15:23.472024 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:23.471994    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75fd9f5d47-c46gq_e3b353c1-3812-44aa-b77d-d7ddfc3b7057/istio-proxy/0.log"
Oct 30 22:15:23.579045 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:23.579017    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75fd9f5d47-mxsln_f77afa6c-51f5-4bc2-b10e-89101bd067c8/istio-proxy/0.log"
Oct 30 22:15:23.647780 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:23.647697    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj_6b114f93-6494-464c-bfec-b2f7386f353f/istio-proxy/0.log"
Oct 30 22:15:23.654650 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:23.654627    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-7c4986d449-7cgk9_d6ca21b9-20a4-4887-9d08-19d9c491a09e/insights-operator/0.log"
Oct 30 22:15:23.655276 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:23.655261    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-7c4986d449-7cgk9_d6ca21b9-20a4-4887-9d08-19d9c491a09e/insights-operator/1.log"
Oct 30 22:15:23.701853 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:23.701814    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5bbfd9465-x28lh_9b4f9524-db75-4206-aa76-04400d95706f/router/0.log"
Oct 30 22:15:23.858410 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:23.858349    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-z2dd4_0bbc8570-4947-455a-b7f8-8e9fa8e023b3/kube-rbac-proxy/0.log"
Oct 30 22:15:23.885141 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:23.885115    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-z2dd4_0bbc8570-4947-455a-b7f8-8e9fa8e023b3/exporter/0.log"
Oct 30 22:15:23.911698 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:23.911609    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-z2dd4_0bbc8570-4947-455a-b7f8-8e9fa8e023b3/extractor/0.log"
Oct 30 22:15:23.949786 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:23.949753    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_periodic-gathering-nssh6-9f5sq_2495cf48-596f-4ff0-afa7-4e5a9521d6ab/insights-gathering/0.log"
Oct 30 22:15:26.129261 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:26.129232    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6b748b65852wbfw_3f48d971-e68f-4bb2-8c83-0f2acf541950/istio-proxy/0.log"
Oct 30 22:15:26.427979 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:26.427905    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub-operator-system_opendatahub-operator-controller-manager-6bbc596d7f-fq45v_ed1ba0e9-d432-4674-b52e-7110f348e98a/manager/0.log"
Oct 30 22:15:26.460702 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:26.460670    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub-operator-system_opendatahub-operator-controller-manager-6bbc596d7f-fq45v_ed1ba0e9-d432-4674-b52e-7110f348e98a/copy-manifests/0.log"
Oct 30 22:15:27.183091 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:27.183066    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75fd9f5d47-7s44p_457cd47c-9543-4d82-875f-f1e45d6d5b70/istio-proxy/0.log"
Oct 30 22:15:27.244199 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:27.244166    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75fd9f5d47-c46gq_e3b353c1-3812-44aa-b77d-d7ddfc3b7057/istio-proxy/0.log"
Oct 30 22:15:27.324153 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:27.324126    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75fd9f5d47-mxsln_f77afa6c-51f5-4bc2-b10e-89101bd067c8/istio-proxy/0.log"
Oct 30 22:15:27.386043 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:27.386018    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-ai-inference-6b78c66f8c-dlwbj_6b114f93-6494-464c-bfec-b2f7386f353f/istio-proxy/0.log"
Oct 30 22:15:27.452796 ip-10-0-129-195 kubenswrapper[5032]: I1030 22:15:27.452774    5032 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5bbfd9465-x28lh_9b4f9524-db75-4206-aa76-04400d95706f/router/0.log"