Sep 10 19:15:42.986880 ip-10-0-129-230 systemd[1]: Starting Kubernetes Kubelet...
Sep 10 19:15:43.391652 ip-10-0-129-230 kubenswrapper[2564]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 19:15:43.391652 ip-10-0-129-230 kubenswrapper[2564]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 10 19:15:43.391652 ip-10-0-129-230 kubenswrapper[2564]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 19:15:43.391652 ip-10-0-129-230 kubenswrapper[2564]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 10 19:15:43.391652 ip-10-0-129-230 kubenswrapper[2564]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 19:15:43.392685 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.392576    2564 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 10 19:15:43.397012 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.396993    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 19:15:43.397012 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397008    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 19:15:43.397012 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397013    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 19:15:43.397012 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397017    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 19:15:43.397263 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397022    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 19:15:43.397263 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397027    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 19:15:43.397263 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397031    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 19:15:43.397263 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397044    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 19:15:43.397263 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397049    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 19:15:43.397263 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397053    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 19:15:43.397263 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397057    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 19:15:43.397263 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397061    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 19:15:43.397263 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397066    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 19:15:43.397263 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397070    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 19:15:43.397263 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397075    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 19:15:43.397263 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397079    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 19:15:43.397263 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397083    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 19:15:43.397263 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397087    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 19:15:43.397263 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397092    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 19:15:43.397263 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397097    2564 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 19:15:43.397263 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397101    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 19:15:43.397263 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397109    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 19:15:43.397263 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397114    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 19:15:43.397263 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397118    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 19:15:43.398095 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397123    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 19:15:43.398095 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397128    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 19:15:43.398095 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397132    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 19:15:43.398095 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397136    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 19:15:43.398095 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397140    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 19:15:43.398095 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397144    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 19:15:43.398095 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397148    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 19:15:43.398095 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397152    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 19:15:43.398095 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397159    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 19:15:43.398095 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397165    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 19:15:43.398095 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397170    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 19:15:43.398095 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397176    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 19:15:43.398095 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397184    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 19:15:43.398095 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397190    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 19:15:43.398095 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397197    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 19:15:43.398095 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397203    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 19:15:43.398095 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397207    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 19:15:43.398095 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397212    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 19:15:43.398095 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397216    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 19:15:43.398807 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397221    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 19:15:43.398807 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397225    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 19:15:43.398807 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397229    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 19:15:43.398807 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397233    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 19:15:43.398807 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397238    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 19:15:43.398807 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397242    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 19:15:43.398807 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397246    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 19:15:43.398807 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397251    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 19:15:43.398807 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397255    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 19:15:43.398807 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397260    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 19:15:43.398807 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397264    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 19:15:43.398807 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397269    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 19:15:43.398807 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397273    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 19:15:43.398807 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397278    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 19:15:43.398807 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397282    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 19:15:43.398807 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397286    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 19:15:43.398807 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397290    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 19:15:43.398807 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397295    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 19:15:43.398807 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397300    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 19:15:43.398807 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397304    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 19:15:43.399327 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397308    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 19:15:43.399327 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397312    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 19:15:43.399327 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397317    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 19:15:43.399327 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397321    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 19:15:43.399327 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397325    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 19:15:43.399327 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397330    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 19:15:43.399327 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397337    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 19:15:43.399327 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397341    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 19:15:43.399327 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397346    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 19:15:43.399327 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397351    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 19:15:43.399327 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397355    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 19:15:43.399327 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397360    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 19:15:43.399327 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397364    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 19:15:43.399327 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397368    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 19:15:43.399327 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397372    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 19:15:43.399327 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397376    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 19:15:43.399327 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397380    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 19:15:43.399327 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397385    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 19:15:43.399327 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397389    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 19:15:43.399832 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397393    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 19:15:43.399832 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397397    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 19:15:43.399832 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397401    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 19:15:43.399832 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397405    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 19:15:43.399832 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397409    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 19:15:43.399832 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397413    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 19:15:43.399832 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397417    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 19:15:43.399832 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397422    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 19:15:43.399832 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397426    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 19:15:43.399832 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397430    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 19:15:43.399832 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397434    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 19:15:43.399832 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397438    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 19:15:43.399832 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397443    2564 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 19:15:43.399832 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397448    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 19:15:43.399832 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397452    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 19:15:43.399832 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397456    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 19:15:43.399832 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.397460    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 19:15:43.399832 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399308    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 19:15:43.399832 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399319    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 19:15:43.399832 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399322    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 19:15:43.400339 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399325    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 19:15:43.400339 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399328    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 19:15:43.400339 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399332    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 19:15:43.400339 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399335    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 19:15:43.400339 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399338    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 19:15:43.400339 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399341    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 19:15:43.400339 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399343    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 19:15:43.400339 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399346    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 19:15:43.400339 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399349    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 19:15:43.400339 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399351    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 19:15:43.400339 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399354    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 19:15:43.400339 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399357    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 19:15:43.400339 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399360    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 19:15:43.400339 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399362    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 19:15:43.400339 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399365    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 19:15:43.400339 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399369    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 19:15:43.400339 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399373    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 19:15:43.400339 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399376    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 19:15:43.400339 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399379    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 19:15:43.400339 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399382    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 19:15:43.400885 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399385    2564 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 19:15:43.400885 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399388    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 19:15:43.400885 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399391    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 19:15:43.400885 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399394    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 19:15:43.400885 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399397    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 19:15:43.400885 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399400    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 19:15:43.400885 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399404    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 19:15:43.400885 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399408    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 19:15:43.400885 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399411    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 19:15:43.400885 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399414    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 19:15:43.400885 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399417    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 19:15:43.400885 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399420    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 19:15:43.400885 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399423    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 19:15:43.400885 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399426    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 19:15:43.400885 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399428    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 19:15:43.400885 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399432    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 19:15:43.400885 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399435    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 19:15:43.400885 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399437    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 19:15:43.400885 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399440    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 19:15:43.401354 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399442    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 19:15:43.401354 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399445    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 19:15:43.401354 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399447    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 19:15:43.401354 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399450    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 19:15:43.401354 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399453    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 19:15:43.401354 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399455    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 19:15:43.401354 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399458    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 19:15:43.401354 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399461    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 19:15:43.401354 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399463    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 19:15:43.401354 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399466    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 19:15:43.401354 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399469    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 19:15:43.401354 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399472    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 19:15:43.401354 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399475    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 19:15:43.401354 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399477    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 19:15:43.401354 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399480    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 19:15:43.401354 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399483    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 19:15:43.401354 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399485    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 19:15:43.401354 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399489    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 19:15:43.401354 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399493    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 19:15:43.401840 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399496    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 19:15:43.401840 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399500    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 19:15:43.401840 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399503    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 19:15:43.401840 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399505    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 19:15:43.401840 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399508    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 19:15:43.401840 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399510    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 19:15:43.401840 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399513    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 19:15:43.401840 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399515    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 19:15:43.401840 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399518    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 19:15:43.401840 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399521    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 19:15:43.401840 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399523    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 19:15:43.401840 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399525    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 19:15:43.401840 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399528    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 19:15:43.401840 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399531    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 19:15:43.401840 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399533    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 19:15:43.401840 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399536    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 19:15:43.401840 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399539    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 19:15:43.401840 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399542    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 19:15:43.401840 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399544    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 19:15:43.401840 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399547    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 19:15:43.402325 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399549    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 19:15:43.402325 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399552    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 19:15:43.402325 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399556    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 19:15:43.402325 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399559    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 19:15:43.402325 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399561    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 19:15:43.402325 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399563    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 19:15:43.402325 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399566    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 19:15:43.402325 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399569    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 19:15:43.402325 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399571    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 19:15:43.402325 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399574    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 19:15:43.402325 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399577    2564 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 19:15:43.402325 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399579    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 19:15:43.402325 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399582    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 19:15:43.402325 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399585    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 19:15:43.402325 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399587    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 19:15:43.402325 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399590    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 19:15:43.402325 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399592    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 19:15:43.402325 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399595    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 19:15:43.402325 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.399994    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 19:15:43.402325 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400000    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 19:15:43.402909 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400002    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 19:15:43.402909 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400005    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 19:15:43.402909 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400008    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 19:15:43.402909 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400011    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 19:15:43.402909 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400014    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 19:15:43.402909 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400016    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 19:15:43.402909 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400019    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 19:15:43.402909 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400021    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 19:15:43.402909 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400025    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 19:15:43.402909 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400028    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 19:15:43.402909 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400031    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 19:15:43.402909 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400033    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 19:15:43.402909 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400036    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 19:15:43.402909 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400039    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 19:15:43.402909 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400042    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 19:15:43.402909 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400044    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 19:15:43.402909 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400047    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 19:15:43.402909 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400050    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 19:15:43.402909 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400052    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 19:15:43.403380 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400057    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 19:15:43.403380 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400061    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 19:15:43.403380 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400064    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 19:15:43.403380 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400067    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 19:15:43.403380 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400070    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 19:15:43.403380 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400074    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 19:15:43.403380 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400077    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 19:15:43.403380 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400079    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 19:15:43.403380 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400082    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 19:15:43.403380 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400085    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 19:15:43.403380 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400087    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 19:15:43.403380 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400090    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 19:15:43.403380 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400092    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 19:15:43.403380 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400094    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 19:15:43.403380 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400097    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 19:15:43.403380 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400100    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 19:15:43.403380 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400102    2564 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 19:15:43.403380 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400105    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 19:15:43.403380 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400107    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 19:15:43.403860 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400109    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 19:15:43.403860 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400112    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 19:15:43.403860 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400115    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 19:15:43.403860 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400117    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 19:15:43.403860 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400120    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 19:15:43.403860 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400123    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 19:15:43.403860 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400125    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 19:15:43.403860 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400128    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 19:15:43.403860 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400130    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 19:15:43.403860 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400133    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 19:15:43.403860 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400136    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 19:15:43.403860 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400139    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 19:15:43.403860 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400141    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 19:15:43.403860 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400144    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 19:15:43.403860 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400147    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 19:15:43.403860 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400149    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 19:15:43.403860 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400152    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 19:15:43.403860 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400155    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 19:15:43.403860 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400158    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 19:15:43.403860 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400160    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 19:15:43.404354 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400163    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 19:15:43.404354 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400165    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 19:15:43.404354 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400168    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 19:15:43.404354 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400170    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 19:15:43.404354 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400173    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 19:15:43.404354 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400177    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 19:15:43.404354 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400180    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 19:15:43.404354 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400183    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 19:15:43.404354 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400186    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 19:15:43.404354 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400189    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 19:15:43.404354 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400191    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 19:15:43.404354 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400194    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 19:15:43.404354 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400196    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 19:15:43.404354 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400199    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 19:15:43.404354 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400201    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 19:15:43.404354 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400204    2564 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 19:15:43.404354 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400206    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 19:15:43.404354 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400209    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 19:15:43.404354 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400212    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 19:15:43.404354 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400215    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 19:15:43.404867 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400218    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 19:15:43.404867 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400220    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 19:15:43.404867 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400224    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 19:15:43.404867 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400226    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 19:15:43.404867 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400229    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 19:15:43.404867 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400232    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 19:15:43.404867 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400236    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 19:15:43.404867 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400239    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 19:15:43.404867 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400242    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 19:15:43.404867 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400244    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 19:15:43.404867 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400247    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 19:15:43.404867 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400249    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 19:15:43.404867 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400252    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 19:15:43.404867 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400254    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 19:15:43.404867 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400257    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 19:15:43.404867 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400259    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 19:15:43.404867 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400262    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 19:15:43.404867 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400265    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 19:15:43.404867 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.400267    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 19:15:43.405337 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400338    2564 flags.go:64] FLAG: --address="0.0.0.0"
Sep 10 19:15:43.405337 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400377    2564 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 10 19:15:43.405337 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400384    2564 flags.go:64] FLAG: --anonymous-auth="true"
Sep 10 19:15:43.405337 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400389    2564 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 10 19:15:43.405337 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400393    2564 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 10 19:15:43.405337 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400396    2564 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 10 19:15:43.405337 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400400    2564 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 10 19:15:43.405337 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400405    2564 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 10 19:15:43.405337 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400408    2564 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 10 19:15:43.405337 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400411    2564 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 10 19:15:43.405337 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400414    2564 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 10 19:15:43.405337 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400418    2564 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 10 19:15:43.405337 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400421    2564 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 10 19:15:43.405337 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400424    2564 flags.go:64] FLAG: --cgroup-root=""
Sep 10 19:15:43.405337 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400427    2564 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 10 19:15:43.405337 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400430    2564 flags.go:64] FLAG: --client-ca-file=""
Sep 10 19:15:43.405337 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400435    2564 flags.go:64] FLAG: --cloud-provider="external"
Sep 10 19:15:43.405337 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400438    2564 flags.go:64] FLAG: --cluster-dns="[]"
Sep 10 19:15:43.405337 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400442    2564 flags.go:64] FLAG: --cluster-domain=""
Sep 10 19:15:43.405337 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400445    2564 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 10 19:15:43.405337 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400448    2564 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 10 19:15:43.405337 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400451    2564 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 10 19:15:43.405337 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400455    2564 flags.go:64] FLAG: --container-log-max-files="5"
Sep 10 19:15:43.405337 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400459    2564 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 10 19:15:43.405941 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400462    2564 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 10 19:15:43.405941 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400465    2564 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 10 19:15:43.405941 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400468    2564 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 10 19:15:43.405941 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400471    2564 flags.go:64] FLAG: --contention-profiling="false"
Sep 10 19:15:43.405941 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400474    2564 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 10 19:15:43.405941 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400477    2564 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 10 19:15:43.405941 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400480    2564 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 10 19:15:43.405941 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400483    2564 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 10 19:15:43.405941 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400487    2564 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 10 19:15:43.405941 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400490    2564 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 10 19:15:43.405941 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400493    2564 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 10 19:15:43.405941 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400496    2564 flags.go:64] FLAG: --enable-load-reader="false"
Sep 10 19:15:43.405941 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400499    2564 flags.go:64] FLAG: --enable-server="true"
Sep 10 19:15:43.405941 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400502    2564 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 10 19:15:43.405941 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400506    2564 flags.go:64] FLAG: --event-burst="100"
Sep 10 19:15:43.405941 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400509    2564 flags.go:64] FLAG: --event-qps="50"
Sep 10 19:15:43.405941 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400512    2564 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 10 19:15:43.405941 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400515    2564 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 10 19:15:43.405941 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400518    2564 flags.go:64] FLAG: --eviction-hard=""
Sep 10 19:15:43.405941 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400522    2564 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 10 19:15:43.405941 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400525    2564 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 10 19:15:43.405941 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400528    2564 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 10 19:15:43.405941 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400530    2564 flags.go:64] FLAG: --eviction-soft=""
Sep 10 19:15:43.405941 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400536    2564 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 10 19:15:43.405941 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400539    2564 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 10 19:15:43.406605 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400542    2564 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 10 19:15:43.406605 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400546    2564 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 10 19:15:43.406605 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400549    2564 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 10 19:15:43.406605 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400552    2564 flags.go:64] FLAG: --fail-swap-on="true"
Sep 10 19:15:43.406605 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400555    2564 flags.go:64] FLAG: --feature-gates=""
Sep 10 19:15:43.406605 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400559    2564 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 10 19:15:43.406605 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400562    2564 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 10 19:15:43.406605 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400565    2564 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 10 19:15:43.406605 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400568    2564 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 10 19:15:43.406605 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400571    2564 flags.go:64] FLAG: --healthz-port="10248"
Sep 10 19:15:43.406605 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400574    2564 flags.go:64] FLAG: --help="false"
Sep 10 19:15:43.406605 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400577    2564 flags.go:64] FLAG: --hostname-override="ip-10-0-129-230.ec2.internal"
Sep 10 19:15:43.406605 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400580    2564 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 10 19:15:43.406605 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400583    2564 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 10 19:15:43.406605 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400586    2564 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 10 19:15:43.406605 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400589    2564 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 10 19:15:43.406605 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400593    2564 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 10 19:15:43.406605 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400596    2564 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 10 19:15:43.406605 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400598    2564 flags.go:64] FLAG: --image-service-endpoint=""
Sep 10 19:15:43.406605 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400601    2564 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 10 19:15:43.406605 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400604    2564 flags.go:64] FLAG: --kube-api-burst="100"
Sep 10 19:15:43.406605 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400607    2564 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 10 19:15:43.406605 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400623    2564 flags.go:64] FLAG: --kube-api-qps="50"
Sep 10 19:15:43.406605 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400626    2564 flags.go:64] FLAG: --kube-reserved=""
Sep 10 19:15:43.407216 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400629    2564 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 10 19:15:43.407216 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400632    2564 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 10 19:15:43.407216 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400635    2564 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 10 19:15:43.407216 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400638    2564 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 10 19:15:43.407216 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400641    2564 flags.go:64] FLAG: --lock-file=""
Sep 10 19:15:43.407216 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400644    2564 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 10 19:15:43.407216 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400649    2564 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 10 19:15:43.407216 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400652    2564 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 10 19:15:43.407216 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400657    2564 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 10 19:15:43.407216 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400661    2564 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 10 19:15:43.407216 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400663    2564 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 10 19:15:43.407216 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400667    2564 flags.go:64] FLAG: --logging-format="text"
Sep 10 19:15:43.407216 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400669    2564 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 10 19:15:43.407216 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400673    2564 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 10 19:15:43.407216 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400675    2564 flags.go:64] FLAG: --manifest-url=""
Sep 10 19:15:43.407216 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400678    2564 flags.go:64] FLAG: --manifest-url-header=""
Sep 10 19:15:43.407216 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400699    2564 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 10 19:15:43.407216 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400703    2564 flags.go:64] FLAG: --max-open-files="1000000"
Sep 10 19:15:43.407216 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400707    2564 flags.go:64] FLAG: --max-pods="110"
Sep 10 19:15:43.407216 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400711    2564 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 10 19:15:43.407216 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400714    2564 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 10 19:15:43.407216 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400717    2564 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 10 19:15:43.407216 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400720    2564 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 10 19:15:43.407216 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400723    2564 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 10 19:15:43.407216 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400726    2564 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 10 19:15:43.407901 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400729    2564 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 10 19:15:43.407901 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400736    2564 flags.go:64] FLAG: --node-status-max-images="50"
Sep 10 19:15:43.407901 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400740    2564 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 10 19:15:43.407901 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400743    2564 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 10 19:15:43.407901 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400746    2564 flags.go:64] FLAG: --pod-cidr=""
Sep 10 19:15:43.407901 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400748    2564 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 10 19:15:43.407901 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400755    2564 flags.go:64] FLAG: --pod-manifest-path=""
Sep 10 19:15:43.407901 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400757    2564 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 10 19:15:43.407901 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400761    2564 flags.go:64] FLAG: --pods-per-core="0"
Sep 10 19:15:43.407901 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400763    2564 flags.go:64] FLAG: --port="10250"
Sep 10 19:15:43.407901 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400766    2564 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 10 19:15:43.407901 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400769    2564 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-02888366181062fb8"
Sep 10 19:15:43.407901 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400772    2564 flags.go:64] FLAG: --qos-reserved=""
Sep 10 19:15:43.407901 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400775    2564 flags.go:64] FLAG: --read-only-port="10255"
Sep 10 19:15:43.407901 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400779    2564 flags.go:64] FLAG: --register-node="true"
Sep 10 19:15:43.407901 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400782    2564 flags.go:64] FLAG: --register-with-taints=""
Sep 10 19:15:43.407901 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400787    2564 flags.go:64] FLAG: --registry-burst="10"
Sep 10 19:15:43.407901 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400791    2564 flags.go:64] FLAG: --registry-qps="5"
Sep 10 19:15:43.407901 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400794    2564 flags.go:64] FLAG: --reserved-cpus=""
Sep 10 19:15:43.407901 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400797    2564 flags.go:64] FLAG: --reserved-memory=""
Sep 10 19:15:43.407901 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400800    2564 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 10 19:15:43.407901 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400803    2564 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 10 19:15:43.407901 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400806    2564 flags.go:64] FLAG: --rotate-certificates="false"
Sep 10 19:15:43.407901 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400809    2564 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 10 19:15:43.407901 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400812    2564 flags.go:64] FLAG: --runonce="false"
Sep 10 19:15:43.408517 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400815    2564 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 10 19:15:43.408517 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400818    2564 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 10 19:15:43.408517 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400821    2564 flags.go:64] FLAG: --seccomp-default="false"
Sep 10 19:15:43.408517 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400824    2564 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 10 19:15:43.408517 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400827    2564 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 10 19:15:43.408517 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400830    2564 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 10 19:15:43.408517 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400833    2564 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 10 19:15:43.408517 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400836    2564 flags.go:64] FLAG: --storage-driver-password="root"
Sep 10 19:15:43.408517 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400839    2564 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 10 19:15:43.408517 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400842    2564 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 10 19:15:43.408517 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400845    2564 flags.go:64] FLAG: --storage-driver-user="root"
Sep 10 19:15:43.408517 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400847    2564 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 10 19:15:43.408517 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400850    2564 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 10 19:15:43.408517 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400853    2564 flags.go:64] FLAG: --system-cgroups=""
Sep 10 19:15:43.408517 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400856    2564 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 10 19:15:43.408517 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400861    2564 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 10 19:15:43.408517 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400864    2564 flags.go:64] FLAG: --tls-cert-file=""
Sep 10 19:15:43.408517 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400866    2564 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 10 19:15:43.408517 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400871    2564 flags.go:64] FLAG: --tls-min-version=""
Sep 10 19:15:43.408517 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400874    2564 flags.go:64] FLAG: --tls-private-key-file=""
Sep 10 19:15:43.408517 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400876    2564 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 10 19:15:43.408517 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400880    2564 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 10 19:15:43.408517 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400883    2564 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 10 19:15:43.408517 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400888    2564 flags.go:64] FLAG: --v="2"
Sep 10 19:15:43.408517 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400893    2564 flags.go:64] FLAG: --version="false"
Sep 10 19:15:43.409148 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400898    2564 flags.go:64] FLAG: --vmodule=""
Sep 10 19:15:43.409148 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400902    2564 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 10 19:15:43.409148 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.400905    2564 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 10 19:15:43.409148 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401023    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 19:15:43.409148 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401027    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 19:15:43.409148 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401031    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 19:15:43.409148 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401033    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 19:15:43.409148 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401036    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 19:15:43.409148 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401039    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 19:15:43.409148 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401042    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 19:15:43.409148 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401044    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 19:15:43.409148 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401047    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 19:15:43.409148 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401051    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 19:15:43.409148 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401055    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 19:15:43.409148 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401058    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 19:15:43.409148 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401061    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 19:15:43.409148 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401064    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 19:15:43.409148 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401067    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 19:15:43.409148 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401070    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 19:15:43.409148 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401072    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 19:15:43.409651 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401075    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 19:15:43.409651 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401077    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 19:15:43.409651 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401080    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 19:15:43.409651 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401082    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 19:15:43.409651 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401085    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 19:15:43.409651 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401088    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 19:15:43.409651 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401091    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 19:15:43.409651 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401093    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 19:15:43.409651 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401097    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 19:15:43.409651 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401100    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 19:15:43.409651 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401104    2564 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 19:15:43.409651 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401107    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 19:15:43.409651 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401110    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 19:15:43.409651 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401113    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 19:15:43.409651 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401116    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 19:15:43.409651 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401119    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 19:15:43.409651 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401121    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 19:15:43.409651 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401124    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 19:15:43.409651 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401126    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 19:15:43.409651 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401129    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 19:15:43.410192 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401132    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 19:15:43.410192 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401135    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 19:15:43.410192 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401137    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 19:15:43.410192 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401140    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 19:15:43.410192 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401142    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 19:15:43.410192 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401145    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 19:15:43.410192 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401148    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 19:15:43.410192 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401150    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 19:15:43.410192 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401153    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 19:15:43.410192 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401157    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 19:15:43.410192 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401160    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 19:15:43.410192 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401163    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 19:15:43.410192 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401166    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 19:15:43.410192 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401168    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 19:15:43.410192 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401171    2564 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 19:15:43.410192 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401174    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 19:15:43.410192 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401177    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 19:15:43.410192 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401181    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 19:15:43.410192 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401184    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 19:15:43.410673 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401186    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 19:15:43.410673 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401191    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 19:15:43.410673 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401193    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 19:15:43.410673 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401197    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 19:15:43.410673 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401200    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 19:15:43.410673 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401202    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 19:15:43.410673 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401205    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 19:15:43.410673 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401208    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 19:15:43.410673 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401211    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 19:15:43.410673 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401214    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 19:15:43.410673 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401216    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 19:15:43.410673 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401219    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 19:15:43.410673 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401222    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 19:15:43.410673 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401224    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 19:15:43.410673 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401227    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 19:15:43.410673 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401230    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 19:15:43.410673 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401233    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 19:15:43.410673 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401237    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 19:15:43.410673 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401241    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 19:15:43.410673 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401245    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 19:15:43.411171 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401249    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 19:15:43.411171 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401252    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 19:15:43.411171 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401255    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 19:15:43.411171 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401257    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 19:15:43.411171 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401260    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 19:15:43.411171 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401262    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 19:15:43.411171 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401265    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 19:15:43.411171 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401267    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 19:15:43.411171 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401270    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 19:15:43.411171 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401273    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 19:15:43.411171 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401275    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 19:15:43.411171 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401278    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 19:15:43.411171 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401280    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 19:15:43.411171 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401284    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 19:15:43.411171 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401287    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 19:15:43.411171 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401291    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 19:15:43.411171 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401293    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 19:15:43.411171 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401296    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 19:15:43.411171 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401299    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 19:15:43.411171 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401301    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 19:15:43.411782 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401305    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 19:15:43.411782 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401308    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 19:15:43.411782 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.401310    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 19:15:43.411782 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.402108    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 19:15:43.411782 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.410128    2564 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 10 19:15:43.411782 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.410145    2564 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 10 19:15:43.411782 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.410168    2564 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 10 19:15:43.411782 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.410174    2564 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 10 19:15:43.411782 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410232    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 19:15:43.411782 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410238    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 19:15:43.411782 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410242    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 19:15:43.411782 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410245    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 19:15:43.411782 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410248    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 19:15:43.411782 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410251    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 19:15:43.411782 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410254    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 19:15:43.412166 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410256    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 19:15:43.412166 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410259    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 19:15:43.412166 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410262    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 19:15:43.412166 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410265    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 19:15:43.412166 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410267    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 19:15:43.412166 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410270    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 19:15:43.412166 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410273    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 19:15:43.412166 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410276    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 19:15:43.412166 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410278    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 19:15:43.412166 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410281    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 19:15:43.412166 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410284    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 19:15:43.412166 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410287    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 19:15:43.412166 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410290    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 19:15:43.412166 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410293    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 19:15:43.412166 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410295    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 19:15:43.412166 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410299    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 19:15:43.412166 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410304    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 19:15:43.412166 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410308    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 19:15:43.412166 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410311    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 19:15:43.412648 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410314    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 19:15:43.412648 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410317    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 19:15:43.412648 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410320    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 19:15:43.412648 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410323    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 19:15:43.412648 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410327    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 19:15:43.412648 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410331    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 19:15:43.412648 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410334    2564 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 19:15:43.412648 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410337    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 19:15:43.412648 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410340    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 19:15:43.412648 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410343    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 19:15:43.412648 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410346    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 19:15:43.412648 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410349    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 19:15:43.412648 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410351    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 19:15:43.412648 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410354    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 19:15:43.412648 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410357    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 19:15:43.412648 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410360    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 19:15:43.412648 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410362    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 19:15:43.412648 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410365    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 19:15:43.412648 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410368    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 19:15:43.412648 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410370    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 19:15:43.413128 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410373    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 19:15:43.413128 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410375    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 19:15:43.413128 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410378    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 19:15:43.413128 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410380    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 19:15:43.413128 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410383    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 19:15:43.413128 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410386    2564 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 19:15:43.413128 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410389    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 19:15:43.413128 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410391    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 19:15:43.413128 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410394    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 19:15:43.413128 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410397    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 19:15:43.413128 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410400    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 19:15:43.413128 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410403    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 19:15:43.413128 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410406    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 19:15:43.413128 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410408    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 19:15:43.413128 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410411    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 19:15:43.413128 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410414    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 19:15:43.413128 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410417    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 19:15:43.413128 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410419    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 19:15:43.413128 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410422    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 19:15:43.413128 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410424    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 19:15:43.413626 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410427    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 19:15:43.413626 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410429    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 19:15:43.413626 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410432    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 19:15:43.413626 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410435    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 19:15:43.413626 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410437    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 19:15:43.413626 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410440    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 19:15:43.413626 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410442    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 19:15:43.413626 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410445    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 19:15:43.413626 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410448    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 19:15:43.413626 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410450    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 19:15:43.413626 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410453    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 19:15:43.413626 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410455    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 19:15:43.413626 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410459    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 19:15:43.413626 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410463    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 19:15:43.413626 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410466    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 19:15:43.413626 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410469    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 19:15:43.413626 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410472    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 19:15:43.413626 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410475    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 19:15:43.413626 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410478    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 19:15:43.414122 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410481    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 19:15:43.414122 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410484    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 19:15:43.414122 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410486    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 19:15:43.414122 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410489    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 19:15:43.414122 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410492    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 19:15:43.414122 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410495    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 19:15:43.414122 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410497    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 19:15:43.414122 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410500    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 19:15:43.414122 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410503    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 19:15:43.414122 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410506    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 19:15:43.414122 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410509    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 19:15:43.414122 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410511    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 19:15:43.414122 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410514    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 19:15:43.414122 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410516    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 19:15:43.414122 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.410521    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 19:15:43.414494 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410649    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 19:15:43.414494 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410655    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 19:15:43.414494 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410658    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 19:15:43.414494 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410661    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 19:15:43.414494 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410663    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 19:15:43.414494 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410666    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 19:15:43.414494 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410669    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 19:15:43.414494 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410671    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 19:15:43.414494 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410674    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 19:15:43.414494 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410677    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 19:15:43.414494 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410680    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 19:15:43.414494 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410682    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 19:15:43.414494 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410685    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 19:15:43.414494 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410688    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 19:15:43.414494 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410691    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 19:15:43.414494 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410693    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 19:15:43.414494 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410696    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 19:15:43.414494 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410699    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 19:15:43.414494 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410702    2564 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 19:15:43.414494 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410704    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 19:15:43.414990 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410707    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 19:15:43.414990 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410709    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 19:15:43.414990 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410712    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 19:15:43.414990 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410717    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 19:15:43.414990 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410720    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 19:15:43.414990 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410723    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 19:15:43.414990 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410726    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 19:15:43.414990 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410728    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 19:15:43.414990 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410731    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 19:15:43.414990 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410733    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 19:15:43.414990 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410736    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 19:15:43.414990 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410738    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 19:15:43.414990 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410741    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 19:15:43.414990 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410744    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 19:15:43.414990 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410746    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 19:15:43.414990 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410749    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 19:15:43.414990 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410751    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 19:15:43.414990 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410754    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 19:15:43.414990 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410756    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 19:15:43.414990 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410758    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 19:15:43.415477 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410761    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 19:15:43.415477 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410764    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 19:15:43.415477 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410766    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 19:15:43.415477 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410769    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 19:15:43.415477 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410772    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 19:15:43.415477 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410774    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 19:15:43.415477 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410777    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 19:15:43.415477 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410779    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 19:15:43.415477 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410782    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 19:15:43.415477 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410785    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 19:15:43.415477 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410788    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 19:15:43.415477 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410790    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 19:15:43.415477 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410793    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 19:15:43.415477 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410795    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 19:15:43.415477 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410798    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 19:15:43.415477 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410802    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 19:15:43.415477 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410807    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 19:15:43.415477 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410810    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 19:15:43.415477 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410814    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 19:15:43.415477 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410816    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 19:15:43.415993 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410820    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 19:15:43.415993 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410822    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 19:15:43.415993 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410825    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 19:15:43.415993 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410828    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 19:15:43.415993 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410830    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 19:15:43.415993 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410833    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 19:15:43.415993 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410835    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 19:15:43.415993 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410838    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 19:15:43.415993 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410840    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 19:15:43.415993 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410843    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 19:15:43.415993 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410845    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 19:15:43.415993 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410849    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 19:15:43.415993 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410852    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 19:15:43.415993 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410854    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 19:15:43.415993 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410857    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 19:15:43.415993 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410859    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 19:15:43.415993 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410863    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 19:15:43.415993 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410866    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 19:15:43.415993 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410869    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 19:15:43.416460 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410872    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 19:15:43.416460 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410876    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 19:15:43.416460 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410878    2564 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 19:15:43.416460 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410881    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 19:15:43.416460 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410884    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 19:15:43.416460 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410887    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 19:15:43.416460 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410890    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 19:15:43.416460 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410893    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 19:15:43.416460 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410895    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 19:15:43.416460 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410898    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 19:15:43.416460 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410902    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 19:15:43.416460 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410905    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 19:15:43.416460 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410907    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 19:15:43.416460 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410910    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 19:15:43.416460 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410913    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 19:15:43.416460 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410915    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 19:15:43.416460 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410918    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 19:15:43.416460 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410920    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 19:15:43.416460 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410923    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 19:15:43.416971 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:43.410925    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 19:15:43.416971 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.410930    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 19:15:43.416971 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.411043    2564 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 10 19:15:43.416971 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.413650    2564 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 10 19:15:43.416971 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.414741    2564 server.go:1019] "Starting client certificate rotation"
Sep 10 19:15:43.416971 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.414841    2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 19:15:43.416971 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.414878    2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 19:15:43.417967 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.417953    2564 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 19:15:43.419134 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.419114    2564 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 19:15:43.435115 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.435093    2564 log.go:25] "Validated CRI v1 runtime API"
Sep 10 19:15:43.439144 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.439125    2564 log.go:25] "Validated CRI v1 image API"
Sep 10 19:15:43.440082 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.440066    2564 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 10 19:15:43.443629 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.443597    2564 fs.go:135] Filesystem UUIDs: map[50de4492-e663-4589-8c3e-4c85c7ec063b:/dev/nvme0n1p3 5e64ca80-935c-4d87-a1e6-f329c53b1325:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Sep 10 19:15:43.443736 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.443631    2564 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 10 19:15:43.446324 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.446301    2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 19:15:43.449051 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.448926    2564 manager.go:216] Machine: {Timestamp:2026-09-10 19:15:43.447400593 +0000 UTC m=+0.370317623 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3097459 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2fb83a113935c629abd747ac897cd8 SystemUUID:ec2fb83a-1139-35c6-29ab-d747ac897cd8 BootID:643bbc4c-060c-469a-8cbe-58edb922d33f Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ec:ed:cd:59:f5 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ec:ed:cd:59:f5 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ee:f9:c8:f7:85:61 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 10 19:15:43.449051 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.449039    2564 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 10 19:15:43.449220 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.449157    2564 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 10 19:15:43.452386 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.452360    2564 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 10 19:15:43.452529 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.452390    2564 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-230.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 10 19:15:43.454159 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.454149    2564 topology_manager.go:138] "Creating topology manager with none policy"
Sep 10 19:15:43.454198 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.454162    2564 container_manager_linux.go:309] "Creating device plugin manager"
Sep 10 19:15:43.454198 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.454171    2564 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 19:15:43.454898 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.454888    2564 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 19:15:43.456628 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.456606    2564 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 10 19:15:43.460942 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.460931    2564 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 19:15:43.461067 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.461057    2564 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 10 19:15:43.461099 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.461091    2564 kubelet.go:486] "Attempting to sync node with API server"
Sep 10 19:15:43.461135 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.461104    2564 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 10 19:15:43.461135 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.461122    2564 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 10 19:15:43.461135 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.461132    2564 kubelet.go:398] "Adding apiserver pod source"
Sep 10 19:15:43.461244 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.461141    2564 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 10 19:15:43.461512 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.461491    2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 19:15:43.461512 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.461511    2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 19:15:43.469501 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.469483    2564 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 10 19:15:43.471677 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:43.471647    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 10 19:15:43.471787 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:43.471746    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-230.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 10 19:15:43.471899 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.471880    2564 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 10 19:15:43.471929 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.471917    2564 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 10 19:15:43.472680 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.472668    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 10 19:15:43.472715 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.472691    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 10 19:15:43.472715 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.472698    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 10 19:15:43.472715 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.472704    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 10 19:15:43.472715 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.472712    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 10 19:15:43.472830 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.472718    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 10 19:15:43.472830 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.472731    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 10 19:15:43.472830 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.472736    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 10 19:15:43.472830 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.472743    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 10 19:15:43.472830 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.472749    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 10 19:15:43.472830 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.472757    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 10 19:15:43.472830 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.472762    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 10 19:15:43.473681 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.473670    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 10 19:15:43.473714 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.473682    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 10 19:15:43.476406 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.476388    2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rn5cl"
Sep 10 19:15:43.477536 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.477521    2564 server.go:1268] "Started kubelet"
Sep 10 19:15:43.477636 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.477595    2564 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 10 19:15:43.477681 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.477634    2564 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 10 19:15:43.477717 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.477706    2564 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 10 19:15:43.478413 ip-10-0-129-230 systemd[1]: Started Kubernetes Kubelet.
Sep 10 19:15:43.482077 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.482052    2564 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 10 19:15:43.482518 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.482495    2564 server.go:310] "Adding debug handlers to kubelet server"
Sep 10 19:15:43.483461 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:43.483407    2564 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 10 19:15:43.485000 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.484917    2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rn5cl"
Sep 10 19:15:43.485592 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.485571    2564 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-230.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 19:15:43.488375 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.488355    2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 10 19:15:43.488984 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.488966    2564 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 10 19:15:43.489155 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:43.489115    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-230.ec2.internal\" not found"
Sep 10 19:15:43.489788 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.489772    2564 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 10 19:15:43.489862 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.489798    2564 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 10 19:15:43.489862 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.489772    2564 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 10 19:15:43.489862 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.489774    2564 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 10 19:15:43.489862 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.489857    2564 factory.go:55] Registering systemd factory
Sep 10 19:15:43.490010 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.489865    2564 factory.go:223] Registration of the systemd container factory successfully
Sep 10 19:15:43.490010 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.489876    2564 reconstruct.go:97] "Volume reconstruction finished"
Sep 10 19:15:43.490010 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.489884    2564 reconciler.go:29] "Reconciler: start to sync state"
Sep 10 19:15:43.490153 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.490139    2564 factory.go:153] Registering CRI-O factory
Sep 10 19:15:43.490187 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.490157    2564 factory.go:223] Registration of the crio container factory successfully
Sep 10 19:15:43.490187 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.490176    2564 factory.go:103] Registering Raw factory
Sep 10 19:15:43.490241 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.490189    2564 manager.go:1196] Started watching for new ooms in manager
Sep 10 19:15:43.490590 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.490578    2564 manager.go:349] Starting recovery of all containers
Sep 10 19:15:43.498716 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.498693    2564 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 19:15:43.501988 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.501970    2564 manager.go:354] Recovery completed
Sep 10 19:15:43.503406 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:43.503389    2564 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-129-230.ec2.internal\" not found" node="ip-10-0-129-230.ec2.internal"
Sep 10 19:15:43.506700 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.506687    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 19:15:43.508958 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.508943    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-230.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 19:15:43.509041 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.508979    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-230.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 19:15:43.509041 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.508995    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-230.ec2.internal" event="NodeHasSufficientPID"
Sep 10 19:15:43.509581 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.509569    2564 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 10 19:15:43.509705 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.509582    2564 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 10 19:15:43.509705 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.509587    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 19:15:43.509705 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.509598    2564 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 19:15:43.510409 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.510392    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-230.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 19:15:43.510480 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.510424    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-230.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 19:15:43.510480 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.510439    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-230.ec2.internal" event="NodeHasSufficientPID"
Sep 10 19:15:43.511792 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.511775    2564 policy_none.go:49] "None policy: Start"
Sep 10 19:15:43.511792 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.511795    2564 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 10 19:15:43.511923 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.511809    2564 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 10 19:15:43.513911 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.513897    2564 policy_none.go:47] "Start"
Sep 10 19:15:43.542198 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.542180    2564 manager.go:337] "Starting Device Plugin manager"
Sep 10 19:15:43.566575 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:43.542218    2564 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 10 19:15:43.566575 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.542228    2564 server.go:87] "Starting device plugin registration server"
Sep 10 19:15:43.566575 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.542554    2564 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 10 19:15:43.566575 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.542566    2564 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 10 19:15:43.566575 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.542691    2564 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 10 19:15:43.566575 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.542773    2564 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 10 19:15:43.566575 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.542783    2564 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 10 19:15:43.566575 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:43.543528    2564 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 10 19:15:43.566575 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:43.543580    2564 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-230.ec2.internal\" not found"
Sep 10 19:15:43.618948 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.618911    2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 10 19:15:43.620126 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.620110    2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 10 19:15:43.620197 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.620165    2564 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 10 19:15:43.620197 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.620188    2564 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 10 19:15:43.620329 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:43.620309    2564 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 10 19:15:43.622855 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.622836    2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 19:15:43.643028 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.642932    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 19:15:43.643914 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.643892    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-230.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 19:15:43.644023 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.643920    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-230.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 19:15:43.644023 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.643929    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-230.ec2.internal" event="NodeHasSufficientPID"
Sep 10 19:15:43.644023 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.643954    2564 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-230.ec2.internal"
Sep 10 19:15:43.653388 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.653371    2564 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-230.ec2.internal"
Sep 10 19:15:43.653451 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:43.653396    2564 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-230.ec2.internal\": node \"ip-10-0-129-230.ec2.internal\" not found"
Sep 10 19:15:43.670865 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:43.670848    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-230.ec2.internal\" not found"
Sep 10 19:15:43.721324 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.721294    2564 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-230.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-230.ec2.internal"]
Sep 10 19:15:43.721438 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.721361    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 19:15:43.723476 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.723460    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-230.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 19:15:43.723549 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.723487    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-230.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 19:15:43.723549 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.723497    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-230.ec2.internal" event="NodeHasSufficientPID"
Sep 10 19:15:43.725894 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.725882    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 19:15:43.726032 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.726017    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-230.ec2.internal"
Sep 10 19:15:43.726068 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.726050    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 19:15:43.726896 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.726877    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-230.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 19:15:43.726896 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.726891    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-230.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 19:15:43.727019 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.726908    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-230.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 19:15:43.727019 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.726907    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-230.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 19:15:43.727019 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.726918    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-230.ec2.internal" event="NodeHasSufficientPID"
Sep 10 19:15:43.727019 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.726925    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-230.ec2.internal" event="NodeHasSufficientPID"
Sep 10 19:15:43.729271 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.729255    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-230.ec2.internal"
Sep 10 19:15:43.729338 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.729285    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 19:15:43.731166 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.731153    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-230.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 19:15:43.731236 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.731174    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-230.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 19:15:43.731236 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.731183    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-230.ec2.internal" event="NodeHasSufficientPID"
Sep 10 19:15:43.752194 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:43.752172    2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-230.ec2.internal\" not found" node="ip-10-0-129-230.ec2.internal"
Sep 10 19:15:43.756649 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:43.756631    2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-230.ec2.internal\" not found" node="ip-10-0-129-230.ec2.internal"
Sep 10 19:15:43.771565 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:43.771546    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-230.ec2.internal\" not found"
Sep 10 19:15:43.791861 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.791842    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8daf278338a7c357076b7909cbd675ea-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-230.ec2.internal\" (UID: \"8daf278338a7c357076b7909cbd675ea\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-230.ec2.internal"
Sep 10 19:15:43.791972 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.791871    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/099d282a8bec5dd8dd074b5f725a2d0c-config\") pod \"kube-apiserver-proxy-ip-10-0-129-230.ec2.internal\" (UID: \"099d282a8bec5dd8dd074b5f725a2d0c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-230.ec2.internal"
Sep 10 19:15:43.791972 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.791895    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8daf278338a7c357076b7909cbd675ea-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-230.ec2.internal\" (UID: \"8daf278338a7c357076b7909cbd675ea\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-230.ec2.internal"
Sep 10 19:15:43.872391 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:43.872361    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-230.ec2.internal\" not found"
Sep 10 19:15:43.892751 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.892732    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8daf278338a7c357076b7909cbd675ea-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-230.ec2.internal\" (UID: \"8daf278338a7c357076b7909cbd675ea\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-230.ec2.internal"
Sep 10 19:15:43.892824 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.892756    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8daf278338a7c357076b7909cbd675ea-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-230.ec2.internal\" (UID: \"8daf278338a7c357076b7909cbd675ea\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-230.ec2.internal"
Sep 10 19:15:43.892824 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.892773    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/099d282a8bec5dd8dd074b5f725a2d0c-config\") pod \"kube-apiserver-proxy-ip-10-0-129-230.ec2.internal\" (UID: \"099d282a8bec5dd8dd074b5f725a2d0c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-230.ec2.internal"
Sep 10 19:15:43.892919 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.892839    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/099d282a8bec5dd8dd074b5f725a2d0c-config\") pod \"kube-apiserver-proxy-ip-10-0-129-230.ec2.internal\" (UID: \"099d282a8bec5dd8dd074b5f725a2d0c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-230.ec2.internal"
Sep 10 19:15:43.892919 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.892849    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8daf278338a7c357076b7909cbd675ea-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-230.ec2.internal\" (UID: \"8daf278338a7c357076b7909cbd675ea\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-230.ec2.internal"
Sep 10 19:15:43.892919 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:43.892861    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8daf278338a7c357076b7909cbd675ea-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-230.ec2.internal\" (UID: \"8daf278338a7c357076b7909cbd675ea\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-230.ec2.internal"
Sep 10 19:15:43.973216 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:43.973135    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-230.ec2.internal\" not found"
Sep 10 19:15:44.054690 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:44.054657    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-230.ec2.internal"
Sep 10 19:15:44.059523 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:44.059508    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-230.ec2.internal"
Sep 10 19:15:44.074119 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:44.074099    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-230.ec2.internal\" not found"
Sep 10 19:15:45.496716 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:45.496672    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-230.ec2.internal\" not found"
Sep 10 19:15:45.597254 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:45.597182    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-230.ec2.internal\" not found"
Sep 10 19:15:45.697830 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:45.697798    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-230.ec2.internal\" not found"
Sep 10 19:15:45.736286 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:45.736250    2564 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 10 19:15:45.736719 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:45.736416    2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 10 19:15:45.736719 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:45.736440    2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 10 19:15:45.796418 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:45.796393    2564 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 19:15:45.797883 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:45.797862    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-230.ec2.internal\" not found"
Sep 10 19:15:45.812382 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:45.812342    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-09 19:10:44 +0000 UTC" deadline="2028-03-11 08:26:47.315910279 +0000 UTC"
Sep 10 19:15:45.812501 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:45.812381    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13141h11m1.503538218s"
Sep 10 19:15:45.898452 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:45.898392    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-230.ec2.internal\" not found"
Sep 10 19:15:45.999252 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:45.999219    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-230.ec2.internal\" not found"
Sep 10 19:15:46.028367 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:46.028339    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod099d282a8bec5dd8dd074b5f725a2d0c.slice/crio-11a97bf9ac01412ac9b2690e4d00143abcaccbfea2d8f095ebecc9426cd17db2 WatchSource:0}: Error finding container 11a97bf9ac01412ac9b2690e4d00143abcaccbfea2d8f095ebecc9426cd17db2: Status 404 returned error can't find the container with id 11a97bf9ac01412ac9b2690e4d00143abcaccbfea2d8f095ebecc9426cd17db2
Sep 10 19:15:46.032123 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.032109    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 19:15:46.099846 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:46.099823    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-230.ec2.internal\" not found"
Sep 10 19:15:46.135871 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.135854    2564 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 19:15:46.211868 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.211807    2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-230.ec2.internal"
Sep 10 19:15:46.223462 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.223443    2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 19:15:46.224320 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.224308    2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-230.ec2.internal"
Sep 10 19:15:46.233277 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.233252    2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 19:15:46.447173 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:46.447140    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8daf278338a7c357076b7909cbd675ea.slice/crio-46692b3c3470a1f4bbebace3fdb269f2693e13d41c2af4ea0dde30139f65dc38 WatchSource:0}: Error finding container 46692b3c3470a1f4bbebace3fdb269f2693e13d41c2af4ea0dde30139f65dc38: Status 404 returned error can't find the container with id 46692b3c3470a1f4bbebace3fdb269f2693e13d41c2af4ea0dde30139f65dc38
Sep 10 19:15:46.665454 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.665368    2564 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 19:15:46.784386 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.784357    2564 apiserver.go:52] "Watching apiserver"
Sep 10 19:15:46.794009 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.793986    2564 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 10 19:15:46.794879 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.794854    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-pvhzx","openshift-multus/network-metrics-daemon-tvlcw","openshift-network-diagnostics/network-check-target-9stjj","openshift-network-operator/iptables-alerter-5jjtz","kube-system/global-pull-secret-syncer-vcs72","kube-system/kube-apiserver-proxy-ip-10-0-129-230.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-klvjs","openshift-cluster-node-tuning-operator/tuned-wv24c","openshift-image-registry/node-ca-7s7hj","openshift-multus/multus-additional-cni-plugins-mzwh9","openshift-ovn-kubernetes/ovnkube-node-82crq","kube-system/konnectivity-agent-lhhr2","openshift-dns/node-resolver-l8b8c","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-230.ec2.internal"]
Sep 10 19:15:46.797417 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.797396    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-pvhzx"
Sep 10 19:15:46.799469 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.799444    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvlcw"
Sep 10 19:15:46.799572 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:46.799531    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvlcw" podUID="94b7fd86-07b1-4d2c-869f-787a7fcc6dca"
Sep 10 19:15:46.799886 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.799867    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 10 19:15:46.799986 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.799882    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 10 19:15:46.799986 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.799887    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 10 19:15:46.799986 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.799937    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-gjcr8\""
Sep 10 19:15:46.800142 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.799986    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 10 19:15:46.801653 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.801616    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-klvjs"
Sep 10 19:15:46.803836 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.803819    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-5jjtz"
Sep 10 19:15:46.804359 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.804340    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 10 19:15:46.804789 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.804773    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-v6xb6\""
Sep 10 19:15:46.804860 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.804795    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 10 19:15:46.804860 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.804777    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 10 19:15:46.806063 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.806043    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 10 19:15:46.806152 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.806071    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-pcj9f\""
Sep 10 19:15:46.806152 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.806071    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 10 19:15:46.806364 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.806349    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcs72"
Sep 10 19:15:46.806437 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:46.806410    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcs72" podUID="80a423f3-63d3-42fa-a233-166ab675777e"
Sep 10 19:15:46.806844 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.806829    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 10 19:15:46.808422 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.808403    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9stjj"
Sep 10 19:15:46.808520 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:46.808479    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9stjj" podUID="24f84abd-cd7a-40fa-b23f-72e60cc247b8"
Sep 10 19:15:46.810751 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.810732    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-wv24c"
Sep 10 19:15:46.811299 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.811285    2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 10 19:15:46.812877 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.812859    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7s7hj"
Sep 10 19:15:46.813225 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.813200    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 10 19:15:46.813302 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.813234    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-djhw8\""
Sep 10 19:15:46.813535 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.813522    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 10 19:15:46.815145 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.815124    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mzwh9"
Sep 10 19:15:46.815551 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.815522    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 10 19:15:46.815651 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.815566    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 10 19:15:46.815651 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.815604    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 10 19:15:46.815651 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.815570    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-r45qd\""
Sep 10 19:15:46.817609 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.817591    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-wngt5\""
Sep 10 19:15:46.817724 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.817593    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 10 19:15:46.817867 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.817853    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 10 19:15:46.818322 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.818305    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:15:46.820251 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.820236    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 10 19:15:46.820711 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.820691    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-lhhr2"
Sep 10 19:15:46.821752 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.821571    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 10 19:15:46.821752 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.821594    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 10 19:15:46.821888 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.821780    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 10 19:15:46.821888 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.821808    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 10 19:15:46.823515 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.823492    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-tj896\""
Sep 10 19:15:46.823764 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.823744    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 10 19:15:46.823936 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.823914    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-l8b8c"
Sep 10 19:15:46.824273 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.824253    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-4mkp5\""
Sep 10 19:15:46.824661 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.824640    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 10 19:15:46.825951 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.825930    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 10 19:15:46.826262 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.826240    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/10449b06-ca4d-4ee1-8d45-d33f3bd9aea1-socket-dir\") pod \"aws-ebs-csi-driver-node-klvjs\" (UID: \"10449b06-ca4d-4ee1-8d45-d33f3bd9aea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-klvjs"
Sep 10 19:15:46.826379 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.826271    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/10449b06-ca4d-4ee1-8d45-d33f3bd9aea1-device-dir\") pod \"aws-ebs-csi-driver-node-klvjs\" (UID: \"10449b06-ca4d-4ee1-8d45-d33f3bd9aea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-klvjs"
Sep 10 19:15:46.826444 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.826382    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/80a423f3-63d3-42fa-a233-166ab675777e-original-pull-secret\") pod \"global-pull-secret-syncer-vcs72\" (UID: \"80a423f3-63d3-42fa-a233-166ab675777e\") " pod="kube-system/global-pull-secret-syncer-vcs72"
Sep 10 19:15:46.826444 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.826424    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/09feb671-e7d9-467f-ac39-5ad97eb572fa-etc-modprobe-d\") pod \"tuned-wv24c\" (UID: \"09feb671-e7d9-467f-ac39-5ad97eb572fa\") " pod="openshift-cluster-node-tuning-operator/tuned-wv24c"
Sep 10 19:15:46.826558 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.826478    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/09feb671-e7d9-467f-ac39-5ad97eb572fa-etc-systemd\") pod \"tuned-wv24c\" (UID: \"09feb671-e7d9-467f-ac39-5ad97eb572fa\") " pod="openshift-cluster-node-tuning-operator/tuned-wv24c"
Sep 10 19:15:46.826558 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.826498    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/09feb671-e7d9-467f-ac39-5ad97eb572fa-host\") pod \"tuned-wv24c\" (UID: \"09feb671-e7d9-467f-ac39-5ad97eb572fa\") " pod="openshift-cluster-node-tuning-operator/tuned-wv24c"
Sep 10 19:15:46.826558 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.826521    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2-system-cni-dir\") pod \"multus-pvhzx\" (UID: \"c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2\") " pod="openshift-multus/multus-pvhzx"
Sep 10 19:15:46.826777 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.826578    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2-cni-binary-copy\") pod \"multus-pvhzx\" (UID: \"c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2\") " pod="openshift-multus/multus-pvhzx"
Sep 10 19:15:46.826777 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.826599    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2-multus-socket-dir-parent\") pod \"multus-pvhzx\" (UID: \"c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2\") " pod="openshift-multus/multus-pvhzx"
Sep 10 19:15:46.826777 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.826635    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/80a423f3-63d3-42fa-a233-166ab675777e-kubelet-config\") pod \"global-pull-secret-syncer-vcs72\" (UID: \"80a423f3-63d3-42fa-a233-166ab675777e\") " pod="kube-system/global-pull-secret-syncer-vcs72"
Sep 10 19:15:46.826777 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.826687    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/09feb671-e7d9-467f-ac39-5ad97eb572fa-lib-modules\") pod \"tuned-wv24c\" (UID: \"09feb671-e7d9-467f-ac39-5ad97eb572fa\") " pod="openshift-cluster-node-tuning-operator/tuned-wv24c"
Sep 10 19:15:46.826777 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.826717    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/09feb671-e7d9-467f-ac39-5ad97eb572fa-tmp\") pod \"tuned-wv24c\" (UID: \"09feb671-e7d9-467f-ac39-5ad97eb572fa\") " pod="openshift-cluster-node-tuning-operator/tuned-wv24c"
Sep 10 19:15:46.826777 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.826758    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 10 19:15:46.826956 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.826767    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2-multus-cni-dir\") pod \"multus-pvhzx\" (UID: \"c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2\") " pod="openshift-multus/multus-pvhzx"
Sep 10 19:15:46.826956 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.826820    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2-host-run-k8s-cni-cncf-io\") pod \"multus-pvhzx\" (UID: \"c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2\") " pod="openshift-multus/multus-pvhzx"
Sep 10 19:15:46.826956 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.826842    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2-host-var-lib-kubelet\") pod \"multus-pvhzx\" (UID: \"c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2\") " pod="openshift-multus/multus-pvhzx"
Sep 10 19:15:46.826956 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.826859    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2-host-run-multus-certs\") pod \"multus-pvhzx\" (UID: \"c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2\") " pod="openshift-multus/multus-pvhzx"
Sep 10 19:15:46.826956 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.826885    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/58cdd25d-25ab-403e-abcc-65f251ee8482-iptables-alerter-script\") pod \"iptables-alerter-5jjtz\" (UID: \"58cdd25d-25ab-403e-abcc-65f251ee8482\") " pod="openshift-network-operator/iptables-alerter-5jjtz"
Sep 10 19:15:46.826956 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.826908    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/80a423f3-63d3-42fa-a233-166ab675777e-dbus\") pod \"global-pull-secret-syncer-vcs72\" (UID: \"80a423f3-63d3-42fa-a233-166ab675777e\") " pod="kube-system/global-pull-secret-syncer-vcs72"
Sep 10 19:15:46.826956 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.826925    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/09feb671-e7d9-467f-ac39-5ad97eb572fa-etc-kubernetes\") pod \"tuned-wv24c\" (UID: \"09feb671-e7d9-467f-ac39-5ad97eb572fa\") " pod="openshift-cluster-node-tuning-operator/tuned-wv24c"
Sep 10 19:15:46.827179 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.826973    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/09feb671-e7d9-467f-ac39-5ad97eb572fa-etc-sysctl-conf\") pod \"tuned-wv24c\" (UID: \"09feb671-e7d9-467f-ac39-5ad97eb572fa\") " pod="openshift-cluster-node-tuning-operator/tuned-wv24c"
Sep 10 19:15:46.827179 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.827012    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ae8c5b13-1925-4df6-b3a2-d4ace9a51801-host\") pod \"node-ca-7s7hj\" (UID: \"ae8c5b13-1925-4df6-b3a2-d4ace9a51801\") " pod="openshift-image-registry/node-ca-7s7hj"
Sep 10 19:15:46.827179 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.827045    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2-host-var-lib-cni-bin\") pod \"multus-pvhzx\" (UID: \"c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2\") " pod="openshift-multus/multus-pvhzx"
Sep 10 19:15:46.827179 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.827075    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2-multus-daemon-config\") pod \"multus-pvhzx\" (UID: \"c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2\") " pod="openshift-multus/multus-pvhzx"
Sep 10 19:15:46.827179 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.827105    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/10449b06-ca4d-4ee1-8d45-d33f3bd9aea1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-klvjs\" (UID: \"10449b06-ca4d-4ee1-8d45-d33f3bd9aea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-klvjs"
Sep 10 19:15:46.827378 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.827184    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tfb7\" (UniqueName: \"kubernetes.io/projected/94b7fd86-07b1-4d2c-869f-787a7fcc6dca-kube-api-access-4tfb7\") pod \"network-metrics-daemon-tvlcw\" (UID: \"94b7fd86-07b1-4d2c-869f-787a7fcc6dca\") " pod="openshift-multus/network-metrics-daemon-tvlcw"
Sep 10 19:15:46.827378 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.827217    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/09feb671-e7d9-467f-ac39-5ad97eb572fa-etc-sysctl-d\") pod \"tuned-wv24c\" (UID: \"09feb671-e7d9-467f-ac39-5ad97eb572fa\") " pod="openshift-cluster-node-tuning-operator/tuned-wv24c"
Sep 10 19:15:46.827378 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.827243    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/09feb671-e7d9-467f-ac39-5ad97eb572fa-var-lib-kubelet\") pod \"tuned-wv24c\" (UID: \"09feb671-e7d9-467f-ac39-5ad97eb572fa\") " pod="openshift-cluster-node-tuning-operator/tuned-wv24c"
Sep 10 19:15:46.827378 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.827272    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/09feb671-e7d9-467f-ac39-5ad97eb572fa-etc-tuned\") pod \"tuned-wv24c\" (UID: \"09feb671-e7d9-467f-ac39-5ad97eb572fa\") " pod="openshift-cluster-node-tuning-operator/tuned-wv24c"
Sep 10 19:15:46.827378 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.827353    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2-hostroot\") pod \"multus-pvhzx\" (UID: \"c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2\") " pod="openshift-multus/multus-pvhzx"
Sep 10 19:15:46.827607 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.827395    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/10449b06-ca4d-4ee1-8d45-d33f3bd9aea1-registration-dir\") pod \"aws-ebs-csi-driver-node-klvjs\" (UID: \"10449b06-ca4d-4ee1-8d45-d33f3bd9aea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-klvjs"
Sep 10 19:15:46.827607 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.827428    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvdsl\" (UniqueName: \"kubernetes.io/projected/10449b06-ca4d-4ee1-8d45-d33f3bd9aea1-kube-api-access-mvdsl\") pod \"aws-ebs-csi-driver-node-klvjs\" (UID: \"10449b06-ca4d-4ee1-8d45-d33f3bd9aea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-klvjs"
Sep 10 19:15:46.827607 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.827504    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpzqd\" (UniqueName: \"kubernetes.io/projected/58cdd25d-25ab-403e-abcc-65f251ee8482-kube-api-access-rpzqd\") pod \"iptables-alerter-5jjtz\" (UID: \"58cdd25d-25ab-403e-abcc-65f251ee8482\") " pod="openshift-network-operator/iptables-alerter-5jjtz"
Sep 10 19:15:46.827607 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.827535    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94b7fd86-07b1-4d2c-869f-787a7fcc6dca-metrics-certs\") pod \"network-metrics-daemon-tvlcw\" (UID: \"94b7fd86-07b1-4d2c-869f-787a7fcc6dca\") " pod="openshift-multus/network-metrics-daemon-tvlcw"
Sep 10 19:15:46.827607 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.827569    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqrjw\" (UniqueName: \"kubernetes.io/projected/24f84abd-cd7a-40fa-b23f-72e60cc247b8-kube-api-access-qqrjw\") pod \"network-check-target-9stjj\" (UID: \"24f84abd-cd7a-40fa-b23f-72e60cc247b8\") " pod="openshift-network-diagnostics/network-check-target-9stjj"
Sep 10 19:15:46.827607 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.827596    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/09feb671-e7d9-467f-ac39-5ad97eb572fa-sys\") pod \"tuned-wv24c\" (UID: \"09feb671-e7d9-467f-ac39-5ad97eb572fa\") " pod="openshift-cluster-node-tuning-operator/tuned-wv24c"
Sep 10 19:15:46.827882 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.827661    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9qpw\" (UniqueName: \"kubernetes.io/projected/ae8c5b13-1925-4df6-b3a2-d4ace9a51801-kube-api-access-h9qpw\") pod \"node-ca-7s7hj\" (UID: \"ae8c5b13-1925-4df6-b3a2-d4ace9a51801\") " pod="openshift-image-registry/node-ca-7s7hj"
Sep 10 19:15:46.827882 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.827689    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2-host-var-lib-cni-multus\") pod \"multus-pvhzx\" (UID: \"c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2\") " pod="openshift-multus/multus-pvhzx"
Sep 10 19:15:46.827882 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.827717    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkxd4\" (UniqueName: \"kubernetes.io/projected/c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2-kube-api-access-xkxd4\") pod \"multus-pvhzx\" (UID: \"c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2\") " pod="openshift-multus/multus-pvhzx"
Sep 10 19:15:46.827882 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.827744    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/10449b06-ca4d-4ee1-8d45-d33f3bd9aea1-etc-selinux\") pod \"aws-ebs-csi-driver-node-klvjs\" (UID: \"10449b06-ca4d-4ee1-8d45-d33f3bd9aea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-klvjs"
Sep 10 19:15:46.827882 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.827773    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/09feb671-e7d9-467f-ac39-5ad97eb572fa-etc-sysconfig\") pod \"tuned-wv24c\" (UID: \"09feb671-e7d9-467f-ac39-5ad97eb572fa\") " pod="openshift-cluster-node-tuning-operator/tuned-wv24c"
Sep 10 19:15:46.827882 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.827797    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ae8c5b13-1925-4df6-b3a2-d4ace9a51801-serviceca\") pod \"node-ca-7s7hj\" (UID: \"ae8c5b13-1925-4df6-b3a2-d4ace9a51801\") " pod="openshift-image-registry/node-ca-7s7hj"
Sep 10 19:15:46.828082 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.827892    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2-host-run-netns\") pod \"multus-pvhzx\" (UID: \"c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2\") " pod="openshift-multus/multus-pvhzx"
Sep 10 19:15:46.828082 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.827922    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/58cdd25d-25ab-403e-abcc-65f251ee8482-host-slash\") pod \"iptables-alerter-5jjtz\" (UID: \"58cdd25d-25ab-403e-abcc-65f251ee8482\") " pod="openshift-network-operator/iptables-alerter-5jjtz"
Sep 10 19:15:46.828082 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.827947    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kv67r\" (UniqueName: \"kubernetes.io/projected/09feb671-e7d9-467f-ac39-5ad97eb572fa-kube-api-access-kv67r\") pod \"tuned-wv24c\" (UID: \"09feb671-e7d9-467f-ac39-5ad97eb572fa\") " pod="openshift-cluster-node-tuning-operator/tuned-wv24c"
Sep 10 19:15:46.828082 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.828000    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2-cnibin\") pod \"multus-pvhzx\" (UID: \"c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2\") " pod="openshift-multus/multus-pvhzx"
Sep 10 19:15:46.828082 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.828028    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2-multus-conf-dir\") pod \"multus-pvhzx\" (UID: \"c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2\") " pod="openshift-multus/multus-pvhzx"
Sep 10 19:15:46.828082 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.828048    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-4zrjm\""
Sep 10 19:15:46.828082 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.828052    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2-etc-kubernetes\") pod \"multus-pvhzx\" (UID: \"c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2\") " pod="openshift-multus/multus-pvhzx"
Sep 10 19:15:46.828082 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.828080    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/10449b06-ca4d-4ee1-8d45-d33f3bd9aea1-sys-fs\") pod \"aws-ebs-csi-driver-node-klvjs\" (UID: \"10449b06-ca4d-4ee1-8d45-d33f3bd9aea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-klvjs"
Sep 10 19:15:46.828335 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.828108    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/09feb671-e7d9-467f-ac39-5ad97eb572fa-run\") pod \"tuned-wv24c\" (UID: \"09feb671-e7d9-467f-ac39-5ad97eb572fa\") " pod="openshift-cluster-node-tuning-operator/tuned-wv24c"
Sep 10 19:15:46.829068 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.828881    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 10 19:15:46.830047 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.830025    2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 19:15:46.846579 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.846555    2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-fhf69"
Sep 10 19:15:46.855392 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.855370    2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-fhf69"
Sep 10 19:15:46.912752 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.912721    2564 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 10 19:15:46.929410 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.929326    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ac0ef973-45eb-4f42-8c5d-d86695e32507-cnibin\") pod \"multus-additional-cni-plugins-mzwh9\" (UID: \"ac0ef973-45eb-4f42-8c5d-d86695e32507\") " pod="openshift-multus/multus-additional-cni-plugins-mzwh9"
Sep 10 19:15:46.929410 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.929365    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ae8c5b13-1925-4df6-b3a2-d4ace9a51801-host\") pod \"node-ca-7s7hj\" (UID: \"ae8c5b13-1925-4df6-b3a2-d4ace9a51801\") " pod="openshift-image-registry/node-ca-7s7hj"
Sep 10 19:15:46.929410 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.929391    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2-host-var-lib-cni-bin\") pod \"multus-pvhzx\" (UID: \"c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2\") " pod="openshift-multus/multus-pvhzx"
Sep 10 19:15:46.929698 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.929449    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/10449b06-ca4d-4ee1-8d45-d33f3bd9aea1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-klvjs\" (UID: \"10449b06-ca4d-4ee1-8d45-d33f3bd9aea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-klvjs"
Sep 10 19:15:46.929698 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.929464    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ae8c5b13-1925-4df6-b3a2-d4ace9a51801-host\") pod \"node-ca-7s7hj\" (UID: \"ae8c5b13-1925-4df6-b3a2-d4ace9a51801\") " pod="openshift-image-registry/node-ca-7s7hj"
Sep 10 19:15:46.929698 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.929479    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvdsl\" (UniqueName: \"kubernetes.io/projected/10449b06-ca4d-4ee1-8d45-d33f3bd9aea1-kube-api-access-mvdsl\") pod \"aws-ebs-csi-driver-node-klvjs\" (UID: \"10449b06-ca4d-4ee1-8d45-d33f3bd9aea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-klvjs"
Sep 10 19:15:46.929698 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.929513    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpzqd\" (UniqueName: \"kubernetes.io/projected/58cdd25d-25ab-403e-abcc-65f251ee8482-kube-api-access-rpzqd\") pod \"iptables-alerter-5jjtz\" (UID: \"58cdd25d-25ab-403e-abcc-65f251ee8482\") " pod="openshift-network-operator/iptables-alerter-5jjtz"
Sep 10 19:15:46.929698 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.929533    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/299c1d38-48fb-48bd-a701-d89a74c1cecb-systemd-units\") pod \"ovnkube-node-82crq\" (UID: \"299c1d38-48fb-48bd-a701-d89a74c1cecb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:15:46.929698 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.929541    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2-host-var-lib-cni-bin\") pod \"multus-pvhzx\" (UID: \"c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2\") " pod="openshift-multus/multus-pvhzx"
Sep 10 19:15:46.929698 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.929551    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/299c1d38-48fb-48bd-a701-d89a74c1cecb-env-overrides\") pod \"ovnkube-node-82crq\" (UID: \"299c1d38-48fb-48bd-a701-d89a74c1cecb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:15:46.929698 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.929564    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/10449b06-ca4d-4ee1-8d45-d33f3bd9aea1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-klvjs\" (UID: \"10449b06-ca4d-4ee1-8d45-d33f3bd9aea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-klvjs"
Sep 10 19:15:46.929698 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.929610    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ac0ef973-45eb-4f42-8c5d-d86695e32507-system-cni-dir\") pod \"multus-additional-cni-plugins-mzwh9\" (UID: \"ac0ef973-45eb-4f42-8c5d-d86695e32507\") " pod="openshift-multus/multus-additional-cni-plugins-mzwh9"
Sep 10 19:15:46.929698 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.929685    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/09feb671-e7d9-467f-ac39-5ad97eb572fa-var-lib-kubelet\") pod \"tuned-wv24c\" (UID: \"09feb671-e7d9-467f-ac39-5ad97eb572fa\") " pod="openshift-cluster-node-tuning-operator/tuned-wv24c"
Sep 10 19:15:46.930109 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.929724    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/299c1d38-48fb-48bd-a701-d89a74c1cecb-run-systemd\") pod \"ovnkube-node-82crq\" (UID: \"299c1d38-48fb-48bd-a701-d89a74c1cecb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:15:46.930109 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.929763    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/299c1d38-48fb-48bd-a701-d89a74c1cecb-run-ovn\") pod \"ovnkube-node-82crq\" (UID: \"299c1d38-48fb-48bd-a701-d89a74c1cecb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:15:46.930109 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.929800    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/09feb671-e7d9-467f-ac39-5ad97eb572fa-var-lib-kubelet\") pod \"tuned-wv24c\" (UID: \"09feb671-e7d9-467f-ac39-5ad97eb572fa\") " pod="openshift-cluster-node-tuning-operator/tuned-wv24c"
Sep 10 19:15:46.930109 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.929836    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/09feb671-e7d9-467f-ac39-5ad97eb572fa-sys\") pod \"tuned-wv24c\" (UID: \"09feb671-e7d9-467f-ac39-5ad97eb572fa\") " pod="openshift-cluster-node-tuning-operator/tuned-wv24c"
Sep 10 19:15:46.930109 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.929862    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ac0ef973-45eb-4f42-8c5d-d86695e32507-cni-binary-copy\") pod \"multus-additional-cni-plugins-mzwh9\" (UID: \"ac0ef973-45eb-4f42-8c5d-d86695e32507\") " pod="openshift-multus/multus-additional-cni-plugins-mzwh9"
Sep 10 19:15:46.930109 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.929934    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/09feb671-e7d9-467f-ac39-5ad97eb572fa-sys\") pod \"tuned-wv24c\" (UID: \"09feb671-e7d9-467f-ac39-5ad97eb572fa\") " pod="openshift-cluster-node-tuning-operator/tuned-wv24c"
Sep 10 19:15:46.930109 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.929971    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/299c1d38-48fb-48bd-a701-d89a74c1cecb-host-cni-bin\") pod \"ovnkube-node-82crq\" (UID: \"299c1d38-48fb-48bd-a701-d89a74c1cecb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:15:46.930109 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.930003    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7lj9\" (UniqueName: \"kubernetes.io/projected/299c1d38-48fb-48bd-a701-d89a74c1cecb-kube-api-access-z7lj9\") pod \"ovnkube-node-82crq\" (UID: \"299c1d38-48fb-48bd-a701-d89a74c1cecb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:15:46.930109 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.930044    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ae8c5b13-1925-4df6-b3a2-d4ace9a51801-serviceca\") pod \"node-ca-7s7hj\" (UID: \"ae8c5b13-1925-4df6-b3a2-d4ace9a51801\") " pod="openshift-image-registry/node-ca-7s7hj"
Sep 10 19:15:46.930109 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.930067    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2-host-run-netns\") pod \"multus-pvhzx\" (UID: \"c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2\") " pod="openshift-multus/multus-pvhzx"
Sep 10 19:15:46.930109 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.930090    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/58cdd25d-25ab-403e-abcc-65f251ee8482-host-slash\") pod \"iptables-alerter-5jjtz\" (UID: \"58cdd25d-25ab-403e-abcc-65f251ee8482\") " pod="openshift-network-operator/iptables-alerter-5jjtz"
Sep 10 19:15:46.930467 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.930112    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ac0ef973-45eb-4f42-8c5d-d86695e32507-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mzwh9\" (UID: \"ac0ef973-45eb-4f42-8c5d-d86695e32507\") " pod="openshift-multus/multus-additional-cni-plugins-mzwh9"
Sep 10 19:15:46.930467 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.930188    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/299c1d38-48fb-48bd-a701-d89a74c1cecb-host-slash\") pod \"ovnkube-node-82crq\" (UID: \"299c1d38-48fb-48bd-a701-d89a74c1cecb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:15:46.930467 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.930206    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/299c1d38-48fb-48bd-a701-d89a74c1cecb-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-82crq\" (UID: \"299c1d38-48fb-48bd-a701-d89a74c1cecb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:15:46.930467 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.930221    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/299c1d38-48fb-48bd-a701-d89a74c1cecb-ovnkube-config\") pod \"ovnkube-node-82crq\" (UID: \"299c1d38-48fb-48bd-a701-d89a74c1cecb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:15:46.930467 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.930246    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2-multus-conf-dir\") pod \"multus-pvhzx\" (UID: \"c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2\") " pod="openshift-multus/multus-pvhzx"
Sep 10 19:15:46.930467 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.930267    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2-etc-kubernetes\") pod \"multus-pvhzx\" (UID: \"c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2\") " pod="openshift-multus/multus-pvhzx"
Sep 10 19:15:46.930467 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.930300    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/10449b06-ca4d-4ee1-8d45-d33f3bd9aea1-sys-fs\") pod \"aws-ebs-csi-driver-node-klvjs\" (UID: \"10449b06-ca4d-4ee1-8d45-d33f3bd9aea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-klvjs"
Sep 10 19:15:46.930467 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.930329    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/299c1d38-48fb-48bd-a701-d89a74c1cecb-ovnkube-script-lib\") pod \"ovnkube-node-82crq\" (UID: \"299c1d38-48fb-48bd-a701-d89a74c1cecb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:15:46.930467 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.930351    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/09feb671-e7d9-467f-ac39-5ad97eb572fa-run\") pod \"tuned-wv24c\" (UID: \"09feb671-e7d9-467f-ac39-5ad97eb572fa\") " pod="openshift-cluster-node-tuning-operator/tuned-wv24c"
Sep 10 19:15:46.930467 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.930402    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/10449b06-ca4d-4ee1-8d45-d33f3bd9aea1-socket-dir\") pod \"aws-ebs-csi-driver-node-klvjs\" (UID: \"10449b06-ca4d-4ee1-8d45-d33f3bd9aea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-klvjs"
Sep 10 19:15:46.930467 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.930423    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/09feb671-e7d9-467f-ac39-5ad97eb572fa-etc-modprobe-d\") pod \"tuned-wv24c\" (UID: \"09feb671-e7d9-467f-ac39-5ad97eb572fa\") " pod="openshift-cluster-node-tuning-operator/tuned-wv24c"
Sep 10 19:15:46.930467 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.930440    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/09feb671-e7d9-467f-ac39-5ad97eb572fa-etc-systemd\") pod \"tuned-wv24c\" (UID: \"09feb671-e7d9-467f-ac39-5ad97eb572fa\") " pod="openshift-cluster-node-tuning-operator/tuned-wv24c"
Sep 10 19:15:46.930467 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.930459    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2-multus-socket-dir-parent\") pod \"multus-pvhzx\" (UID: \"c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2\") " pod="openshift-multus/multus-pvhzx"
Sep 10 19:15:46.931015 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.930478    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ac0ef973-45eb-4f42-8c5d-d86695e32507-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mzwh9\" (UID: \"ac0ef973-45eb-4f42-8c5d-d86695e32507\") " pod="openshift-multus/multus-additional-cni-plugins-mzwh9"
Sep 10 19:15:46.931015 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.930495    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/299c1d38-48fb-48bd-a701-d89a74c1cecb-var-lib-openvswitch\") pod \"ovnkube-node-82crq\" (UID: \"299c1d38-48fb-48bd-a701-d89a74c1cecb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:15:46.931015 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.930508    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/299c1d38-48fb-48bd-a701-d89a74c1cecb-log-socket\") pod \"ovnkube-node-82crq\" (UID: \"299c1d38-48fb-48bd-a701-d89a74c1cecb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:15:46.931015 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.930530    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6e38a77e-c0f7-43c1-a915-fd6a036cbb0e-agent-certs\") pod \"konnectivity-agent-lhhr2\" (UID: \"6e38a77e-c0f7-43c1-a915-fd6a036cbb0e\") " pod="kube-system/konnectivity-agent-lhhr2"
Sep 10 19:15:46.931015 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.930546    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/80a423f3-63d3-42fa-a233-166ab675777e-kubelet-config\") pod \"global-pull-secret-syncer-vcs72\" (UID: \"80a423f3-63d3-42fa-a233-166ab675777e\") " pod="kube-system/global-pull-secret-syncer-vcs72"
Sep 10 19:15:46.931015 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.930565    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/09feb671-e7d9-467f-ac39-5ad97eb572fa-lib-modules\") pod \"tuned-wv24c\" (UID: \"09feb671-e7d9-467f-ac39-5ad97eb572fa\") " pod="openshift-cluster-node-tuning-operator/tuned-wv24c"
Sep 10 19:15:46.931015 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.930579    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2-multus-cni-dir\") pod \"multus-pvhzx\" (UID: \"c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2\") " pod="openshift-multus/multus-pvhzx"
Sep 10 19:15:46.931015 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.930602    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jl4xv\" (UniqueName: \"kubernetes.io/projected/ac0ef973-45eb-4f42-8c5d-d86695e32507-kube-api-access-jl4xv\") pod \"multus-additional-cni-plugins-mzwh9\" (UID: \"ac0ef973-45eb-4f42-8c5d-d86695e32507\") " pod="openshift-multus/multus-additional-cni-plugins-mzwh9"
Sep 10 19:15:46.931015 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.930644    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/299c1d38-48fb-48bd-a701-d89a74c1cecb-run-openvswitch\") pod \"ovnkube-node-82crq\" (UID: \"299c1d38-48fb-48bd-a701-d89a74c1cecb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:15:46.931015 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.930672    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/80a423f3-63d3-42fa-a233-166ab675777e-dbus\") pod \"global-pull-secret-syncer-vcs72\" (UID: \"80a423f3-63d3-42fa-a233-166ab675777e\") " pod="kube-system/global-pull-secret-syncer-vcs72"
Sep 10 19:15:46.931015 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.930682    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2-etc-kubernetes\") pod \"multus-pvhzx\" (UID: \"c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2\") " pod="openshift-multus/multus-pvhzx"
Sep 10 19:15:46.931015 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.930748    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/58cdd25d-25ab-403e-abcc-65f251ee8482-host-slash\") pod \"iptables-alerter-5jjtz\" (UID: \"58cdd25d-25ab-403e-abcc-65f251ee8482\") " pod="openshift-network-operator/iptables-alerter-5jjtz"
Sep 10 19:15:46.931015 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.930749    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2-host-run-netns\") pod \"multus-pvhzx\" (UID: \"c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2\") " pod="openshift-multus/multus-pvhzx"
Sep 10 19:15:46.931015 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.930761    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2-multus-socket-dir-parent\") pod \"multus-pvhzx\" (UID: \"c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2\") " pod="openshift-multus/multus-pvhzx"
Sep 10 19:15:46.931015 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.930818    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/09feb671-e7d9-467f-ac39-5ad97eb572fa-etc-modprobe-d\") pod \"tuned-wv24c\" (UID: \"09feb671-e7d9-467f-ac39-5ad97eb572fa\") " pod="openshift-cluster-node-tuning-operator/tuned-wv24c"
Sep 10 19:15:46.931015 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.930838    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/10449b06-ca4d-4ee1-8d45-d33f3bd9aea1-sys-fs\") pod \"aws-ebs-csi-driver-node-klvjs\" (UID: \"10449b06-ca4d-4ee1-8d45-d33f3bd9aea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-klvjs"
Sep 10 19:15:46.931015 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.930864    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/80a423f3-63d3-42fa-a233-166ab675777e-kubelet-config\") pod \"global-pull-secret-syncer-vcs72\" (UID: \"80a423f3-63d3-42fa-a233-166ab675777e\") " pod="kube-system/global-pull-secret-syncer-vcs72"
Sep 10 19:15:46.931752 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.930893    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/10449b06-ca4d-4ee1-8d45-d33f3bd9aea1-socket-dir\") pod \"aws-ebs-csi-driver-node-klvjs\" (UID: \"10449b06-ca4d-4ee1-8d45-d33f3bd9aea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-klvjs"
Sep 10 19:15:46.931752 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.930900    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/80a423f3-63d3-42fa-a233-166ab675777e-dbus\") pod \"global-pull-secret-syncer-vcs72\" (UID: \"80a423f3-63d3-42fa-a233-166ab675777e\") " pod="kube-system/global-pull-secret-syncer-vcs72"
Sep 10 19:15:46.931752 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.930921    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/09feb671-e7d9-467f-ac39-5ad97eb572fa-etc-systemd\") pod \"tuned-wv24c\" (UID: \"09feb671-e7d9-467f-ac39-5ad97eb572fa\") " pod="openshift-cluster-node-tuning-operator/tuned-wv24c"
Sep 10 19:15:46.931752 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.930930    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/09feb671-e7d9-467f-ac39-5ad97eb572fa-etc-kubernetes\") pod \"tuned-wv24c\" (UID: \"09feb671-e7d9-467f-ac39-5ad97eb572fa\") " pod="openshift-cluster-node-tuning-operator/tuned-wv24c"
Sep 10 19:15:46.931752 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.930951    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/09feb671-e7d9-467f-ac39-5ad97eb572fa-run\") pod \"tuned-wv24c\" (UID: \"09feb671-e7d9-467f-ac39-5ad97eb572fa\") " pod="openshift-cluster-node-tuning-operator/tuned-wv24c"
Sep 10 19:15:46.931752 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.930965    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/09feb671-e7d9-467f-ac39-5ad97eb572fa-etc-sysctl-conf\") pod \"tuned-wv24c\" (UID: \"09feb671-e7d9-467f-ac39-5ad97eb572fa\") " pod="openshift-cluster-node-tuning-operator/tuned-wv24c"
Sep 10 19:15:46.931752 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.930976    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2-multus-cni-dir\") pod \"multus-pvhzx\" (UID: \"c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2\") " pod="openshift-multus/multus-pvhzx"
Sep 10 19:15:46.931752 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.930999    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2-multus-daemon-config\") pod \"multus-pvhzx\" (UID: \"c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2\") " pod="openshift-multus/multus-pvhzx"
Sep 10 19:15:46.931752 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.931004    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2-multus-conf-dir\") pod \"multus-pvhzx\" (UID: \"c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2\") " pod="openshift-multus/multus-pvhzx"
Sep 10 19:15:46.931752 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.931022    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/10449b06-ca4d-4ee1-8d45-d33f3bd9aea1-registration-dir\") pod \"aws-ebs-csi-driver-node-klvjs\" (UID: \"10449b06-ca4d-4ee1-8d45-d33f3bd9aea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-klvjs"
Sep 10 19:15:46.931752 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.931062    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/09feb671-e7d9-467f-ac39-5ad97eb572fa-etc-kubernetes\") pod \"tuned-wv24c\" (UID: \"09feb671-e7d9-467f-ac39-5ad97eb572fa\") " pod="openshift-cluster-node-tuning-operator/tuned-wv24c"
Sep 10 19:15:46.931752 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.931061    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e08684c6-1cc0-4f4c-a4cd-1fea1e46339a-tmp-dir\") pod \"node-resolver-l8b8c\" (UID: \"e08684c6-1cc0-4f4c-a4cd-1fea1e46339a\") " pod="openshift-dns/node-resolver-l8b8c"
Sep 10 19:15:46.931752 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.931174    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/09feb671-e7d9-467f-ac39-5ad97eb572fa-lib-modules\") pod \"tuned-wv24c\" (UID: \"09feb671-e7d9-467f-ac39-5ad97eb572fa\") " pod="openshift-cluster-node-tuning-operator/tuned-wv24c"
Sep 10 19:15:46.931752 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.931185    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/10449b06-ca4d-4ee1-8d45-d33f3bd9aea1-registration-dir\") pod \"aws-ebs-csi-driver-node-klvjs\" (UID: \"10449b06-ca4d-4ee1-8d45-d33f3bd9aea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-klvjs"
Sep 10 19:15:46.931752 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.931185    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tfb7\" (UniqueName: \"kubernetes.io/projected/94b7fd86-07b1-4d2c-869f-787a7fcc6dca-kube-api-access-4tfb7\") pod \"network-metrics-daemon-tvlcw\" (UID: \"94b7fd86-07b1-4d2c-869f-787a7fcc6dca\") " pod="openshift-multus/network-metrics-daemon-tvlcw"
Sep 10 19:15:46.931752 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.931198    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/09feb671-e7d9-467f-ac39-5ad97eb572fa-etc-sysctl-conf\") pod \"tuned-wv24c\" (UID: \"09feb671-e7d9-467f-ac39-5ad97eb572fa\") " pod="openshift-cluster-node-tuning-operator/tuned-wv24c"
Sep 10 19:15:46.931752 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.931229    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/09feb671-e7d9-467f-ac39-5ad97eb572fa-etc-sysctl-d\") pod \"tuned-wv24c\" (UID: \"09feb671-e7d9-467f-ac39-5ad97eb572fa\") " pod="openshift-cluster-node-tuning-operator/tuned-wv24c"
Sep 10 19:15:46.931752 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.931276    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ae8c5b13-1925-4df6-b3a2-d4ace9a51801-serviceca\") pod \"node-ca-7s7hj\" (UID: \"ae8c5b13-1925-4df6-b3a2-d4ace9a51801\") " pod="openshift-image-registry/node-ca-7s7hj"
Sep 10 19:15:46.932520 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.931376    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/09feb671-e7d9-467f-ac39-5ad97eb572fa-etc-tuned\") pod \"tuned-wv24c\" (UID: \"09feb671-e7d9-467f-ac39-5ad97eb572fa\") " pod="openshift-cluster-node-tuning-operator/tuned-wv24c"
Sep 10 19:15:46.932520 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.931440    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2-hostroot\") pod \"multus-pvhzx\" (UID: \"c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2\") " pod="openshift-multus/multus-pvhzx"
Sep 10 19:15:46.932520 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.931452    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/09feb671-e7d9-467f-ac39-5ad97eb572fa-etc-sysctl-d\") pod \"tuned-wv24c\" (UID: \"09feb671-e7d9-467f-ac39-5ad97eb572fa\") " pod="openshift-cluster-node-tuning-operator/tuned-wv24c"
Sep 10 19:15:46.932520 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.931479    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e08684c6-1cc0-4f4c-a4cd-1fea1e46339a-hosts-file\") pod \"node-resolver-l8b8c\" (UID: \"e08684c6-1cc0-4f4c-a4cd-1fea1e46339a\") " pod="openshift-dns/node-resolver-l8b8c"
Sep 10 19:15:46.932520 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.931494    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2-hostroot\") pod \"multus-pvhzx\" (UID: \"c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2\") " pod="openshift-multus/multus-pvhzx"
Sep 10 19:15:46.932520 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.931530    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/299c1d38-48fb-48bd-a701-d89a74c1cecb-host-kubelet\") pod \"ovnkube-node-82crq\" (UID: \"299c1d38-48fb-48bd-a701-d89a74c1cecb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:15:46.932520 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.931555    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/299c1d38-48fb-48bd-a701-d89a74c1cecb-host-cni-netd\") pod \"ovnkube-node-82crq\" (UID: \"299c1d38-48fb-48bd-a701-d89a74c1cecb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:15:46.932520 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.931594    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94b7fd86-07b1-4d2c-869f-787a7fcc6dca-metrics-certs\") pod \"network-metrics-daemon-tvlcw\" (UID: \"94b7fd86-07b1-4d2c-869f-787a7fcc6dca\") " pod="openshift-multus/network-metrics-daemon-tvlcw"
Sep 10 19:15:46.932520 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.931597    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2-multus-daemon-config\") pod \"multus-pvhzx\" (UID: \"c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2\") " pod="openshift-multus/multus-pvhzx"
Sep 10 19:15:46.932520 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.931657    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqrjw\" (UniqueName: \"kubernetes.io/projected/24f84abd-cd7a-40fa-b23f-72e60cc247b8-kube-api-access-qqrjw\") pod \"network-check-target-9stjj\" (UID: \"24f84abd-cd7a-40fa-b23f-72e60cc247b8\") " pod="openshift-network-diagnostics/network-check-target-9stjj"
Sep 10 19:15:46.932520 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.931717    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9qpw\" (UniqueName: \"kubernetes.io/projected/ae8c5b13-1925-4df6-b3a2-d4ace9a51801-kube-api-access-h9qpw\") pod \"node-ca-7s7hj\" (UID: \"ae8c5b13-1925-4df6-b3a2-d4ace9a51801\") " pod="openshift-image-registry/node-ca-7s7hj"
Sep 10 19:15:46.932520 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:46.931744    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:15:46.932520 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.931795    2564 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 10 19:15:46.932520 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.931815    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2-host-var-lib-cni-multus\") pod \"multus-pvhzx\" (UID: \"c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2\") " pod="openshift-multus/multus-pvhzx"
Sep 10 19:15:46.932520 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.931834    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkxd4\" (UniqueName: \"kubernetes.io/projected/c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2-kube-api-access-xkxd4\") pod \"multus-pvhzx\" (UID: \"c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2\") " pod="openshift-multus/multus-pvhzx"
Sep 10 19:15:46.932520 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.931867    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/10449b06-ca4d-4ee1-8d45-d33f3bd9aea1-etc-selinux\") pod \"aws-ebs-csi-driver-node-klvjs\" (UID: \"10449b06-ca4d-4ee1-8d45-d33f3bd9aea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-klvjs"
Sep 10 19:15:46.932520 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.931900    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2-host-var-lib-cni-multus\") pod \"multus-pvhzx\" (UID: \"c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2\") " pod="openshift-multus/multus-pvhzx"
Sep 10 19:15:46.932520 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.931922    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/10449b06-ca4d-4ee1-8d45-d33f3bd9aea1-etc-selinux\") pod \"aws-ebs-csi-driver-node-klvjs\" (UID: \"10449b06-ca4d-4ee1-8d45-d33f3bd9aea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-klvjs"
Sep 10 19:15:46.933219 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:46.931930    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94b7fd86-07b1-4d2c-869f-787a7fcc6dca-metrics-certs podName:94b7fd86-07b1-4d2c-869f-787a7fcc6dca nodeName:}" failed. No retries permitted until 2026-09-10 19:15:47.431908982 +0000 UTC m=+3.032816222 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/94b7fd86-07b1-4d2c-869f-787a7fcc6dca-metrics-certs") pod "network-metrics-daemon-tvlcw" (UID: "94b7fd86-07b1-4d2c-869f-787a7fcc6dca") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:15:46.933219 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.931955    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/299c1d38-48fb-48bd-a701-d89a74c1cecb-node-log\") pod \"ovnkube-node-82crq\" (UID: \"299c1d38-48fb-48bd-a701-d89a74c1cecb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:15:46.933219 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.931992    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/09feb671-e7d9-467f-ac39-5ad97eb572fa-etc-sysconfig\") pod \"tuned-wv24c\" (UID: \"09feb671-e7d9-467f-ac39-5ad97eb572fa\") " pod="openshift-cluster-node-tuning-operator/tuned-wv24c"
Sep 10 19:15:46.933219 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.932032    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ac0ef973-45eb-4f42-8c5d-d86695e32507-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mzwh9\" (UID: \"ac0ef973-45eb-4f42-8c5d-d86695e32507\") " pod="openshift-multus/multus-additional-cni-plugins-mzwh9"
Sep 10 19:15:46.933219 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.932048    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/09feb671-e7d9-467f-ac39-5ad97eb572fa-etc-sysconfig\") pod \"tuned-wv24c\" (UID: \"09feb671-e7d9-467f-ac39-5ad97eb572fa\") " pod="openshift-cluster-node-tuning-operator/tuned-wv24c"
Sep 10 19:15:46.933219 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.932069    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kv67r\" (UniqueName: \"kubernetes.io/projected/09feb671-e7d9-467f-ac39-5ad97eb572fa-kube-api-access-kv67r\") pod \"tuned-wv24c\" (UID: \"09feb671-e7d9-467f-ac39-5ad97eb572fa\") " pod="openshift-cluster-node-tuning-operator/tuned-wv24c"
Sep 10 19:15:46.933219 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.932106    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2-cnibin\") pod \"multus-pvhzx\" (UID: \"c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2\") " pod="openshift-multus/multus-pvhzx"
Sep 10 19:15:46.933219 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.932142    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/10449b06-ca4d-4ee1-8d45-d33f3bd9aea1-device-dir\") pod \"aws-ebs-csi-driver-node-klvjs\" (UID: \"10449b06-ca4d-4ee1-8d45-d33f3bd9aea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-klvjs"
Sep 10 19:15:46.933219 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.932166    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2-cnibin\") pod \"multus-pvhzx\" (UID: \"c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2\") " pod="openshift-multus/multus-pvhzx"
Sep 10 19:15:46.933219 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.932171    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpvjv\" (UniqueName: \"kubernetes.io/projected/e08684c6-1cc0-4f4c-a4cd-1fea1e46339a-kube-api-access-jpvjv\") pod \"node-resolver-l8b8c\" (UID: \"e08684c6-1cc0-4f4c-a4cd-1fea1e46339a\") " pod="openshift-dns/node-resolver-l8b8c"
Sep 10 19:15:46.933219 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.932196    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/299c1d38-48fb-48bd-a701-d89a74c1cecb-host-run-ovn-kubernetes\") pod \"ovnkube-node-82crq\" (UID: \"299c1d38-48fb-48bd-a701-d89a74c1cecb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:15:46.933219 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.932208    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/10449b06-ca4d-4ee1-8d45-d33f3bd9aea1-device-dir\") pod \"aws-ebs-csi-driver-node-klvjs\" (UID: \"10449b06-ca4d-4ee1-8d45-d33f3bd9aea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-klvjs"
Sep 10 19:15:46.933219 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.932214    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6e38a77e-c0f7-43c1-a915-fd6a036cbb0e-konnectivity-ca\") pod \"konnectivity-agent-lhhr2\" (UID: \"6e38a77e-c0f7-43c1-a915-fd6a036cbb0e\") " pod="kube-system/konnectivity-agent-lhhr2"
Sep 10 19:15:46.933219 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.932251    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/80a423f3-63d3-42fa-a233-166ab675777e-original-pull-secret\") pod \"global-pull-secret-syncer-vcs72\" (UID: \"80a423f3-63d3-42fa-a233-166ab675777e\") " pod="kube-system/global-pull-secret-syncer-vcs72"
Sep 10 19:15:46.933219 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.932277    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/09feb671-e7d9-467f-ac39-5ad97eb572fa-host\") pod \"tuned-wv24c\" (UID: \"09feb671-e7d9-467f-ac39-5ad97eb572fa\") " pod="openshift-cluster-node-tuning-operator/tuned-wv24c"
Sep 10 19:15:46.933219 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.932291    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2-system-cni-dir\") pod \"multus-pvhzx\" (UID: \"c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2\") " pod="openshift-multus/multus-pvhzx"
Sep 10 19:15:46.934035 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.932311    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2-cni-binary-copy\") pod \"multus-pvhzx\" (UID: \"c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2\") " pod="openshift-multus/multus-pvhzx"
Sep 10 19:15:46.934035 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.932327    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2-host-run-k8s-cni-cncf-io\") pod \"multus-pvhzx\" (UID: \"c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2\") " pod="openshift-multus/multus-pvhzx"
Sep 10 19:15:46.934035 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.932346    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2-host-var-lib-kubelet\") pod \"multus-pvhzx\" (UID: \"c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2\") " pod="openshift-multus/multus-pvhzx"
Sep 10 19:15:46.934035 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:46.932364    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 19:15:46.934035 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.932391    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2-host-run-multus-certs\") pod \"multus-pvhzx\" (UID: \"c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2\") " pod="openshift-multus/multus-pvhzx"
Sep 10 19:15:46.934035 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:46.932413    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80a423f3-63d3-42fa-a233-166ab675777e-original-pull-secret podName:80a423f3-63d3-42fa-a233-166ab675777e nodeName:}" failed. No retries permitted until 2026-09-10 19:15:47.432399193 +0000 UTC m=+3.033306425 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/80a423f3-63d3-42fa-a233-166ab675777e-original-pull-secret") pod "global-pull-secret-syncer-vcs72" (UID: "80a423f3-63d3-42fa-a233-166ab675777e") : object "kube-system"/"original-pull-secret" not registered
Sep 10 19:15:46.934035 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.932368    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2-host-run-multus-certs\") pod \"multus-pvhzx\" (UID: \"c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2\") " pod="openshift-multus/multus-pvhzx"
Sep 10 19:15:46.934035 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.932427    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/09feb671-e7d9-467f-ac39-5ad97eb572fa-host\") pod \"tuned-wv24c\" (UID: \"09feb671-e7d9-467f-ac39-5ad97eb572fa\") " pod="openshift-cluster-node-tuning-operator/tuned-wv24c"
Sep 10 19:15:46.934035 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.932445    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/09feb671-e7d9-467f-ac39-5ad97eb572fa-tmp\") pod \"tuned-wv24c\" (UID: \"09feb671-e7d9-467f-ac39-5ad97eb572fa\") " pod="openshift-cluster-node-tuning-operator/tuned-wv24c"
Sep 10 19:15:46.934035 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.932474    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/58cdd25d-25ab-403e-abcc-65f251ee8482-iptables-alerter-script\") pod \"iptables-alerter-5jjtz\" (UID: \"58cdd25d-25ab-403e-abcc-65f251ee8482\") " pod="openshift-network-operator/iptables-alerter-5jjtz"
Sep 10 19:15:46.934035 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.932502    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/299c1d38-48fb-48bd-a701-d89a74c1cecb-host-run-netns\") pod \"ovnkube-node-82crq\" (UID: \"299c1d38-48fb-48bd-a701-d89a74c1cecb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:15:46.934035 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.932464    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2-system-cni-dir\") pod \"multus-pvhzx\" (UID: \"c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2\") " pod="openshift-multus/multus-pvhzx"
Sep 10 19:15:46.934035 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.932534    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/299c1d38-48fb-48bd-a701-d89a74c1cecb-etc-openvswitch\") pod \"ovnkube-node-82crq\" (UID: \"299c1d38-48fb-48bd-a701-d89a74c1cecb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:15:46.934035 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.932555    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/299c1d38-48fb-48bd-a701-d89a74c1cecb-ovn-node-metrics-cert\") pod \"ovnkube-node-82crq\" (UID: \"299c1d38-48fb-48bd-a701-d89a74c1cecb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:15:46.934035 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.932832    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2-host-var-lib-kubelet\") pod \"multus-pvhzx\" (UID: \"c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2\") " pod="openshift-multus/multus-pvhzx"
Sep 10 19:15:46.934035 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.932864    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2-host-run-k8s-cni-cncf-io\") pod \"multus-pvhzx\" (UID: \"c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2\") " pod="openshift-multus/multus-pvhzx"
Sep 10 19:15:46.934035 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.933035    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2-cni-binary-copy\") pod \"multus-pvhzx\" (UID: \"c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2\") " pod="openshift-multus/multus-pvhzx"
Sep 10 19:15:46.935351 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.933287    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/58cdd25d-25ab-403e-abcc-65f251ee8482-iptables-alerter-script\") pod \"iptables-alerter-5jjtz\" (UID: \"58cdd25d-25ab-403e-abcc-65f251ee8482\") " pod="openshift-network-operator/iptables-alerter-5jjtz"
Sep 10 19:15:46.935500 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.935477    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/09feb671-e7d9-467f-ac39-5ad97eb572fa-etc-tuned\") pod \"tuned-wv24c\" (UID: \"09feb671-e7d9-467f-ac39-5ad97eb572fa\") " pod="openshift-cluster-node-tuning-operator/tuned-wv24c"
Sep 10 19:15:46.935558 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.935529    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/09feb671-e7d9-467f-ac39-5ad97eb572fa-tmp\") pod \"tuned-wv24c\" (UID: \"09feb671-e7d9-467f-ac39-5ad97eb572fa\") " pod="openshift-cluster-node-tuning-operator/tuned-wv24c"
Sep 10 19:15:46.939900 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:46.939875    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 19:15:46.940011 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:46.939906    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 19:15:46.940011 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:46.939924    2564 projected.go:196] Error preparing data for projected volume kube-api-access-qqrjw for pod openshift-network-diagnostics/network-check-target-9stjj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:15:46.940011 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:46.939990    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/24f84abd-cd7a-40fa-b23f-72e60cc247b8-kube-api-access-qqrjw podName:24f84abd-cd7a-40fa-b23f-72e60cc247b8 nodeName:}" failed. No retries permitted until 2026-09-10 19:15:47.439968038 +0000 UTC m=+3.040875278 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-qqrjw" (UniqueName: "kubernetes.io/projected/24f84abd-cd7a-40fa-b23f-72e60cc247b8-kube-api-access-qqrjw") pod "network-check-target-9stjj" (UID: "24f84abd-cd7a-40fa-b23f-72e60cc247b8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:15:46.943144 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.943117    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpzqd\" (UniqueName: \"kubernetes.io/projected/58cdd25d-25ab-403e-abcc-65f251ee8482-kube-api-access-rpzqd\") pod \"iptables-alerter-5jjtz\" (UID: \"58cdd25d-25ab-403e-abcc-65f251ee8482\") " pod="openshift-network-operator/iptables-alerter-5jjtz"
Sep 10 19:15:46.943262 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.943211    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvdsl\" (UniqueName: \"kubernetes.io/projected/10449b06-ca4d-4ee1-8d45-d33f3bd9aea1-kube-api-access-mvdsl\") pod \"aws-ebs-csi-driver-node-klvjs\" (UID: \"10449b06-ca4d-4ee1-8d45-d33f3bd9aea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-klvjs"
Sep 10 19:15:46.944293 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.944272    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tfb7\" (UniqueName: \"kubernetes.io/projected/94b7fd86-07b1-4d2c-869f-787a7fcc6dca-kube-api-access-4tfb7\") pod \"network-metrics-daemon-tvlcw\" (UID: \"94b7fd86-07b1-4d2c-869f-787a7fcc6dca\") " pod="openshift-multus/network-metrics-daemon-tvlcw"
Sep 10 19:15:46.946019 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.945996    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kv67r\" (UniqueName: \"kubernetes.io/projected/09feb671-e7d9-467f-ac39-5ad97eb572fa-kube-api-access-kv67r\") pod \"tuned-wv24c\" (UID: \"09feb671-e7d9-467f-ac39-5ad97eb572fa\") " pod="openshift-cluster-node-tuning-operator/tuned-wv24c"
Sep 10 19:15:46.946326 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.946293    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkxd4\" (UniqueName: \"kubernetes.io/projected/c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2-kube-api-access-xkxd4\") pod \"multus-pvhzx\" (UID: \"c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2\") " pod="openshift-multus/multus-pvhzx"
Sep 10 19:15:46.946852 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.946808    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9qpw\" (UniqueName: \"kubernetes.io/projected/ae8c5b13-1925-4df6-b3a2-d4ace9a51801-kube-api-access-h9qpw\") pod \"node-ca-7s7hj\" (UID: \"ae8c5b13-1925-4df6-b3a2-d4ace9a51801\") " pod="openshift-image-registry/node-ca-7s7hj"
Sep 10 19:15:46.950643 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.949156    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-230.ec2.internal" event={"ID":"8daf278338a7c357076b7909cbd675ea","Type":"ContainerStarted","Data":"46692b3c3470a1f4bbebace3fdb269f2693e13d41c2af4ea0dde30139f65dc38"}
Sep 10 19:15:46.952738 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:46.952711    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-230.ec2.internal" event={"ID":"099d282a8bec5dd8dd074b5f725a2d0c","Type":"ContainerStarted","Data":"11a97bf9ac01412ac9b2690e4d00143abcaccbfea2d8f095ebecc9426cd17db2"}
Sep 10 19:15:47.033750 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.033713    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/299c1d38-48fb-48bd-a701-d89a74c1cecb-node-log\") pod \"ovnkube-node-82crq\" (UID: \"299c1d38-48fb-48bd-a701-d89a74c1cecb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:15:47.033893 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.033759    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ac0ef973-45eb-4f42-8c5d-d86695e32507-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mzwh9\" (UID: \"ac0ef973-45eb-4f42-8c5d-d86695e32507\") " pod="openshift-multus/multus-additional-cni-plugins-mzwh9"
Sep 10 19:15:47.033893 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.033824    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/299c1d38-48fb-48bd-a701-d89a74c1cecb-node-log\") pod \"ovnkube-node-82crq\" (UID: \"299c1d38-48fb-48bd-a701-d89a74c1cecb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:15:47.033893 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.033875    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpvjv\" (UniqueName: \"kubernetes.io/projected/e08684c6-1cc0-4f4c-a4cd-1fea1e46339a-kube-api-access-jpvjv\") pod \"node-resolver-l8b8c\" (UID: \"e08684c6-1cc0-4f4c-a4cd-1fea1e46339a\") " pod="openshift-dns/node-resolver-l8b8c"
Sep 10 19:15:47.034046 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.033899    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/299c1d38-48fb-48bd-a701-d89a74c1cecb-host-run-ovn-kubernetes\") pod \"ovnkube-node-82crq\" (UID: \"299c1d38-48fb-48bd-a701-d89a74c1cecb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:15:47.034046 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.033916    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6e38a77e-c0f7-43c1-a915-fd6a036cbb0e-konnectivity-ca\") pod \"konnectivity-agent-lhhr2\" (UID: \"6e38a77e-c0f7-43c1-a915-fd6a036cbb0e\") " pod="kube-system/konnectivity-agent-lhhr2"
Sep 10 19:15:47.034046 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.033983    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/299c1d38-48fb-48bd-a701-d89a74c1cecb-host-run-ovn-kubernetes\") pod \"ovnkube-node-82crq\" (UID: \"299c1d38-48fb-48bd-a701-d89a74c1cecb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:15:47.034202 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.034049    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/299c1d38-48fb-48bd-a701-d89a74c1cecb-host-run-netns\") pod \"ovnkube-node-82crq\" (UID: \"299c1d38-48fb-48bd-a701-d89a74c1cecb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:15:47.034202 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.034088    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/299c1d38-48fb-48bd-a701-d89a74c1cecb-etc-openvswitch\") pod \"ovnkube-node-82crq\" (UID: \"299c1d38-48fb-48bd-a701-d89a74c1cecb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:15:47.034202 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.034113    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/299c1d38-48fb-48bd-a701-d89a74c1cecb-ovn-node-metrics-cert\") pod \"ovnkube-node-82crq\" (UID: \"299c1d38-48fb-48bd-a701-d89a74c1cecb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:15:47.034202 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.034140    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ac0ef973-45eb-4f42-8c5d-d86695e32507-cnibin\") pod \"multus-additional-cni-plugins-mzwh9\" (UID: \"ac0ef973-45eb-4f42-8c5d-d86695e32507\") " pod="openshift-multus/multus-additional-cni-plugins-mzwh9"
Sep 10 19:15:47.034202 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.034169    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/299c1d38-48fb-48bd-a701-d89a74c1cecb-systemd-units\") pod \"ovnkube-node-82crq\" (UID: \"299c1d38-48fb-48bd-a701-d89a74c1cecb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:15:47.034202 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.034192    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/299c1d38-48fb-48bd-a701-d89a74c1cecb-env-overrides\") pod \"ovnkube-node-82crq\" (UID: \"299c1d38-48fb-48bd-a701-d89a74c1cecb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:15:47.034474 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.034218    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ac0ef973-45eb-4f42-8c5d-d86695e32507-system-cni-dir\") pod \"multus-additional-cni-plugins-mzwh9\" (UID: \"ac0ef973-45eb-4f42-8c5d-d86695e32507\") " pod="openshift-multus/multus-additional-cni-plugins-mzwh9"
Sep 10 19:15:47.034474 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.034246    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/299c1d38-48fb-48bd-a701-d89a74c1cecb-run-systemd\") pod \"ovnkube-node-82crq\" (UID: \"299c1d38-48fb-48bd-a701-d89a74c1cecb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:15:47.034474 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.034268    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/299c1d38-48fb-48bd-a701-d89a74c1cecb-run-ovn\") pod \"ovnkube-node-82crq\" (UID: \"299c1d38-48fb-48bd-a701-d89a74c1cecb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:15:47.034474 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.034294    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ac0ef973-45eb-4f42-8c5d-d86695e32507-cni-binary-copy\") pod \"multus-additional-cni-plugins-mzwh9\" (UID: \"ac0ef973-45eb-4f42-8c5d-d86695e32507\") " pod="openshift-multus/multus-additional-cni-plugins-mzwh9"
Sep 10 19:15:47.034474 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.034319    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/299c1d38-48fb-48bd-a701-d89a74c1cecb-host-cni-bin\") pod \"ovnkube-node-82crq\" (UID: \"299c1d38-48fb-48bd-a701-d89a74c1cecb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:15:47.034474 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.034343    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7lj9\" (UniqueName: \"kubernetes.io/projected/299c1d38-48fb-48bd-a701-d89a74c1cecb-kube-api-access-z7lj9\") pod \"ovnkube-node-82crq\" (UID: \"299c1d38-48fb-48bd-a701-d89a74c1cecb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:15:47.034474 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.034359    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ac0ef973-45eb-4f42-8c5d-d86695e32507-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mzwh9\" (UID: \"ac0ef973-45eb-4f42-8c5d-d86695e32507\") " pod="openshift-multus/multus-additional-cni-plugins-mzwh9"
Sep 10 19:15:47.034474 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.034369    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ac0ef973-45eb-4f42-8c5d-d86695e32507-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mzwh9\" (UID: \"ac0ef973-45eb-4f42-8c5d-d86695e32507\") " pod="openshift-multus/multus-additional-cni-plugins-mzwh9"
Sep 10 19:15:47.034474 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.034415    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/299c1d38-48fb-48bd-a701-d89a74c1cecb-host-slash\") pod \"ovnkube-node-82crq\" (UID: \"299c1d38-48fb-48bd-a701-d89a74c1cecb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:15:47.034474 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.034443    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/299c1d38-48fb-48bd-a701-d89a74c1cecb-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-82crq\" (UID: \"299c1d38-48fb-48bd-a701-d89a74c1cecb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:15:47.034474 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.034464    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6e38a77e-c0f7-43c1-a915-fd6a036cbb0e-konnectivity-ca\") pod \"konnectivity-agent-lhhr2\" (UID: \"6e38a77e-c0f7-43c1-a915-fd6a036cbb0e\") " pod="kube-system/konnectivity-agent-lhhr2"
Sep 10 19:15:47.034474 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.034468    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/299c1d38-48fb-48bd-a701-d89a74c1cecb-ovnkube-config\") pod \"ovnkube-node-82crq\" (UID: \"299c1d38-48fb-48bd-a701-d89a74c1cecb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:15:47.035064 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.034506    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/299c1d38-48fb-48bd-a701-d89a74c1cecb-ovnkube-script-lib\") pod \"ovnkube-node-82crq\" (UID: \"299c1d38-48fb-48bd-a701-d89a74c1cecb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:15:47.035064 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.034526    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ac0ef973-45eb-4f42-8c5d-d86695e32507-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mzwh9\" (UID: \"ac0ef973-45eb-4f42-8c5d-d86695e32507\") " pod="openshift-multus/multus-additional-cni-plugins-mzwh9"
Sep 10 19:15:47.035064 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.034537    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/299c1d38-48fb-48bd-a701-d89a74c1cecb-host-run-netns\") pod \"ovnkube-node-82crq\" (UID: \"299c1d38-48fb-48bd-a701-d89a74c1cecb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:15:47.035064 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.034540    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ac0ef973-45eb-4f42-8c5d-d86695e32507-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mzwh9\" (UID: \"ac0ef973-45eb-4f42-8c5d-d86695e32507\") " pod="openshift-multus/multus-additional-cni-plugins-mzwh9"
Sep 10 19:15:47.035064 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.034578    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ac0ef973-45eb-4f42-8c5d-d86695e32507-cnibin\") pod \"multus-additional-cni-plugins-mzwh9\" (UID: \"ac0ef973-45eb-4f42-8c5d-d86695e32507\") " pod="openshift-multus/multus-additional-cni-plugins-mzwh9"
Sep 10 19:15:47.035064 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.034614    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/299c1d38-48fb-48bd-a701-d89a74c1cecb-var-lib-openvswitch\") pod \"ovnkube-node-82crq\" (UID: \"299c1d38-48fb-48bd-a701-d89a74c1cecb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:15:47.035064 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.034643    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/299c1d38-48fb-48bd-a701-d89a74c1cecb-systemd-units\") pod \"ovnkube-node-82crq\" (UID: \"299c1d38-48fb-48bd-a701-d89a74c1cecb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:15:47.035064 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.034655    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/299c1d38-48fb-48bd-a701-d89a74c1cecb-log-socket\") pod \"ovnkube-node-82crq\" (UID: \"299c1d38-48fb-48bd-a701-d89a74c1cecb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:15:47.035064 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.034678    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6e38a77e-c0f7-43c1-a915-fd6a036cbb0e-agent-certs\") pod \"konnectivity-agent-lhhr2\" (UID: \"6e38a77e-c0f7-43c1-a915-fd6a036cbb0e\") " pod="kube-system/konnectivity-agent-lhhr2"
Sep 10 19:15:47.035064 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.034705    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jl4xv\" (UniqueName: \"kubernetes.io/projected/ac0ef973-45eb-4f42-8c5d-d86695e32507-kube-api-access-jl4xv\") pod \"multus-additional-cni-plugins-mzwh9\" (UID: \"ac0ef973-45eb-4f42-8c5d-d86695e32507\") " pod="openshift-multus/multus-additional-cni-plugins-mzwh9"
Sep 10 19:15:47.035064 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.034730    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/299c1d38-48fb-48bd-a701-d89a74c1cecb-run-openvswitch\") pod \"ovnkube-node-82crq\" (UID: \"299c1d38-48fb-48bd-a701-d89a74c1cecb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:15:47.035064 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.034759    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e08684c6-1cc0-4f4c-a4cd-1fea1e46339a-tmp-dir\") pod \"node-resolver-l8b8c\" (UID: \"e08684c6-1cc0-4f4c-a4cd-1fea1e46339a\") " pod="openshift-dns/node-resolver-l8b8c"
Sep 10 19:15:47.035064 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.034788    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e08684c6-1cc0-4f4c-a4cd-1fea1e46339a-hosts-file\") pod \"node-resolver-l8b8c\" (UID: \"e08684c6-1cc0-4f4c-a4cd-1fea1e46339a\") " pod="openshift-dns/node-resolver-l8b8c"
Sep 10 19:15:47.035064 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.034811    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/299c1d38-48fb-48bd-a701-d89a74c1cecb-host-kubelet\") pod \"ovnkube-node-82crq\" (UID: \"299c1d38-48fb-48bd-a701-d89a74c1cecb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:15:47.035064 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.034845    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/299c1d38-48fb-48bd-a701-d89a74c1cecb-host-cni-netd\") pod \"ovnkube-node-82crq\" (UID: \"299c1d38-48fb-48bd-a701-d89a74c1cecb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:15:47.035064 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.034936    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/299c1d38-48fb-48bd-a701-d89a74c1cecb-var-lib-openvswitch\") pod \"ovnkube-node-82crq\" (UID: \"299c1d38-48fb-48bd-a701-d89a74c1cecb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:15:47.035064 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.034951    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/299c1d38-48fb-48bd-a701-d89a74c1cecb-run-openvswitch\") pod \"ovnkube-node-82crq\" (UID: \"299c1d38-48fb-48bd-a701-d89a74c1cecb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:15:47.035871 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.034976    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/299c1d38-48fb-48bd-a701-d89a74c1cecb-etc-openvswitch\") pod \"ovnkube-node-82crq\" (UID: \"299c1d38-48fb-48bd-a701-d89a74c1cecb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:15:47.035871 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.034987    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ac0ef973-45eb-4f42-8c5d-d86695e32507-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mzwh9\" (UID: \"ac0ef973-45eb-4f42-8c5d-d86695e32507\") " pod="openshift-multus/multus-additional-cni-plugins-mzwh9"
Sep 10 19:15:47.035871 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.035000    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/299c1d38-48fb-48bd-a701-d89a74c1cecb-log-socket\") pod \"ovnkube-node-82crq\" (UID: \"299c1d38-48fb-48bd-a701-d89a74c1cecb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:15:47.035871 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.034991    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/299c1d38-48fb-48bd-a701-d89a74c1cecb-ovnkube-config\") pod \"ovnkube-node-82crq\" (UID: \"299c1d38-48fb-48bd-a701-d89a74c1cecb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:15:47.035871 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.035018    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/299c1d38-48fb-48bd-a701-d89a74c1cecb-host-cni-netd\") pod \"ovnkube-node-82crq\" (UID: \"299c1d38-48fb-48bd-a701-d89a74c1cecb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:15:47.035871 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.035031    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/299c1d38-48fb-48bd-a701-d89a74c1cecb-env-overrides\") pod \"ovnkube-node-82crq\" (UID: \"299c1d38-48fb-48bd-a701-d89a74c1cecb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:15:47.035871 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.035046    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/299c1d38-48fb-48bd-a701-d89a74c1cecb-host-slash\") pod \"ovnkube-node-82crq\" (UID: \"299c1d38-48fb-48bd-a701-d89a74c1cecb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:15:47.035871 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.035127    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/299c1d38-48fb-48bd-a701-d89a74c1cecb-host-kubelet\") pod \"ovnkube-node-82crq\" (UID: \"299c1d38-48fb-48bd-a701-d89a74c1cecb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:15:47.035871 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.035150    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e08684c6-1cc0-4f4c-a4cd-1fea1e46339a-hosts-file\") pod \"node-resolver-l8b8c\" (UID: \"e08684c6-1cc0-4f4c-a4cd-1fea1e46339a\") " pod="openshift-dns/node-resolver-l8b8c"
Sep 10 19:15:47.035871 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.035229    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/299c1d38-48fb-48bd-a701-d89a74c1cecb-run-systemd\") pod \"ovnkube-node-82crq\" (UID: \"299c1d38-48fb-48bd-a701-d89a74c1cecb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:15:47.035871 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.035272    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/299c1d38-48fb-48bd-a701-d89a74c1cecb-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-82crq\" (UID: \"299c1d38-48fb-48bd-a701-d89a74c1cecb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:15:47.035871 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.035306    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ac0ef973-45eb-4f42-8c5d-d86695e32507-system-cni-dir\") pod \"multus-additional-cni-plugins-mzwh9\" (UID: \"ac0ef973-45eb-4f42-8c5d-d86695e32507\") " pod="openshift-multus/multus-additional-cni-plugins-mzwh9"
Sep 10 19:15:47.035871 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.035329    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/299c1d38-48fb-48bd-a701-d89a74c1cecb-host-cni-bin\") pod \"ovnkube-node-82crq\" (UID: \"299c1d38-48fb-48bd-a701-d89a74c1cecb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:15:47.035871 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.035356    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/299c1d38-48fb-48bd-a701-d89a74c1cecb-run-ovn\") pod \"ovnkube-node-82crq\" (UID: \"299c1d38-48fb-48bd-a701-d89a74c1cecb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:15:47.035871 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.035455    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/299c1d38-48fb-48bd-a701-d89a74c1cecb-ovnkube-script-lib\") pod \"ovnkube-node-82crq\" (UID: \"299c1d38-48fb-48bd-a701-d89a74c1cecb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:15:47.035871 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.035599    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ac0ef973-45eb-4f42-8c5d-d86695e32507-cni-binary-copy\") pod \"multus-additional-cni-plugins-mzwh9\" (UID: \"ac0ef973-45eb-4f42-8c5d-d86695e32507\") " pod="openshift-multus/multus-additional-cni-plugins-mzwh9"
Sep 10 19:15:47.035871 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.035658    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e08684c6-1cc0-4f4c-a4cd-1fea1e46339a-tmp-dir\") pod \"node-resolver-l8b8c\" (UID: \"e08684c6-1cc0-4f4c-a4cd-1fea1e46339a\") " pod="openshift-dns/node-resolver-l8b8c"
Sep 10 19:15:47.036985 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.036948    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/299c1d38-48fb-48bd-a701-d89a74c1cecb-ovn-node-metrics-cert\") pod \"ovnkube-node-82crq\" (UID: \"299c1d38-48fb-48bd-a701-d89a74c1cecb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:15:47.037581 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.037563    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6e38a77e-c0f7-43c1-a915-fd6a036cbb0e-agent-certs\") pod \"konnectivity-agent-lhhr2\" (UID: \"6e38a77e-c0f7-43c1-a915-fd6a036cbb0e\") " pod="kube-system/konnectivity-agent-lhhr2"
Sep 10 19:15:47.043474 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.043454    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpvjv\" (UniqueName: \"kubernetes.io/projected/e08684c6-1cc0-4f4c-a4cd-1fea1e46339a-kube-api-access-jpvjv\") pod \"node-resolver-l8b8c\" (UID: \"e08684c6-1cc0-4f4c-a4cd-1fea1e46339a\") " pod="openshift-dns/node-resolver-l8b8c"
Sep 10 19:15:47.046039 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.046013    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7lj9\" (UniqueName: \"kubernetes.io/projected/299c1d38-48fb-48bd-a701-d89a74c1cecb-kube-api-access-z7lj9\") pod \"ovnkube-node-82crq\" (UID: \"299c1d38-48fb-48bd-a701-d89a74c1cecb\") " pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:15:47.046135 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.046043    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jl4xv\" (UniqueName: \"kubernetes.io/projected/ac0ef973-45eb-4f42-8c5d-d86695e32507-kube-api-access-jl4xv\") pod \"multus-additional-cni-plugins-mzwh9\" (UID: \"ac0ef973-45eb-4f42-8c5d-d86695e32507\") " pod="openshift-multus/multus-additional-cni-plugins-mzwh9"
Sep 10 19:15:47.066654 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.066614    2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 19:15:47.108340 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.108294    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-pvhzx"
Sep 10 19:15:47.116355 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.116336    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-klvjs"
Sep 10 19:15:47.127008 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.126983    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-5jjtz"
Sep 10 19:15:47.132733 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.132710    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-wv24c"
Sep 10 19:15:47.138654 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.138634    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7s7hj"
Sep 10 19:15:47.145276 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.145259    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mzwh9"
Sep 10 19:15:47.152155 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.152136    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:15:47.158172 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.158151    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-lhhr2"
Sep 10 19:15:47.162850 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.162833    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-l8b8c"
Sep 10 19:15:47.419234 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:47.419198    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09feb671_e7d9_467f_ac39_5ad97eb572fa.slice/crio-c72d0afc5eaee9a48e378506d4fd2c94d3dbcbcd9e5d2083cc0550136c525d10 WatchSource:0}: Error finding container c72d0afc5eaee9a48e378506d4fd2c94d3dbcbcd9e5d2083cc0550136c525d10: Status 404 returned error can't find the container with id c72d0afc5eaee9a48e378506d4fd2c94d3dbcbcd9e5d2083cc0550136c525d10
Sep 10 19:15:47.420798 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:47.420721    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58cdd25d_25ab_403e_abcc_65f251ee8482.slice/crio-cb667ec040e4efb28d2f900f86e62de10afe3d5e43f7b5a83253d37c836ad014 WatchSource:0}: Error finding container cb667ec040e4efb28d2f900f86e62de10afe3d5e43f7b5a83253d37c836ad014: Status 404 returned error can't find the container with id cb667ec040e4efb28d2f900f86e62de10afe3d5e43f7b5a83253d37c836ad014
Sep 10 19:15:47.422237 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:47.421825    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc89c4c0a_726d_4d2d_ab72_d3cfc4c907c2.slice/crio-50f8045ef9427fee02d27d00fce1deb13708ee53d2bd0f8fb3f62a9d9220de69 WatchSource:0}: Error finding container 50f8045ef9427fee02d27d00fce1deb13708ee53d2bd0f8fb3f62a9d9220de69: Status 404 returned error can't find the container with id 50f8045ef9427fee02d27d00fce1deb13708ee53d2bd0f8fb3f62a9d9220de69
Sep 10 19:15:47.425264 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:47.425242    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e38a77e_c0f7_43c1_a915_fd6a036cbb0e.slice/crio-2629678efba014592d407c0c943e8ec9469f1b9f48cd5e5b9dccbb79941d167f WatchSource:0}: Error finding container 2629678efba014592d407c0c943e8ec9469f1b9f48cd5e5b9dccbb79941d167f: Status 404 returned error can't find the container with id 2629678efba014592d407c0c943e8ec9469f1b9f48cd5e5b9dccbb79941d167f
Sep 10 19:15:47.426610 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:47.426578    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac0ef973_45eb_4f42_8c5d_d86695e32507.slice/crio-b2b45d9676137c4c7eb309a6f101f4bb0ed33acb4054ab98802f9c40d06d75cf WatchSource:0}: Error finding container b2b45d9676137c4c7eb309a6f101f4bb0ed33acb4054ab98802f9c40d06d75cf: Status 404 returned error can't find the container with id b2b45d9676137c4c7eb309a6f101f4bb0ed33acb4054ab98802f9c40d06d75cf
Sep 10 19:15:47.438218 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.438194    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/80a423f3-63d3-42fa-a233-166ab675777e-original-pull-secret\") pod \"global-pull-secret-syncer-vcs72\" (UID: \"80a423f3-63d3-42fa-a233-166ab675777e\") " pod="kube-system/global-pull-secret-syncer-vcs72"
Sep 10 19:15:47.438297 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.438252    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94b7fd86-07b1-4d2c-869f-787a7fcc6dca-metrics-certs\") pod \"network-metrics-daemon-tvlcw\" (UID: \"94b7fd86-07b1-4d2c-869f-787a7fcc6dca\") " pod="openshift-multus/network-metrics-daemon-tvlcw"
Sep 10 19:15:47.438297 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:47.438278    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 19:15:47.438381 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:47.438323    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80a423f3-63d3-42fa-a233-166ab675777e-original-pull-secret podName:80a423f3-63d3-42fa-a233-166ab675777e nodeName:}" failed. No retries permitted until 2026-09-10 19:15:48.438309418 +0000 UTC m=+4.039216639 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/80a423f3-63d3-42fa-a233-166ab675777e-original-pull-secret") pod "global-pull-secret-syncer-vcs72" (UID: "80a423f3-63d3-42fa-a233-166ab675777e") : object "kube-system"/"original-pull-secret" not registered
Sep 10 19:15:47.438381 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:47.438339    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:15:47.438381 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:47.438377    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94b7fd86-07b1-4d2c-869f-787a7fcc6dca-metrics-certs podName:94b7fd86-07b1-4d2c-869f-787a7fcc6dca nodeName:}" failed. No retries permitted until 2026-09-10 19:15:48.438366163 +0000 UTC m=+4.039273393 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/94b7fd86-07b1-4d2c-869f-787a7fcc6dca-metrics-certs") pod "network-metrics-daemon-tvlcw" (UID: "94b7fd86-07b1-4d2c-869f-787a7fcc6dca") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:15:47.448592 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:47.448571    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae8c5b13_1925_4df6_b3a2_d4ace9a51801.slice/crio-b70708f662eb99389675f62b0c29bad00d29f2408066d8a00f810f79cf59d95d WatchSource:0}: Error finding container b70708f662eb99389675f62b0c29bad00d29f2408066d8a00f810f79cf59d95d: Status 404 returned error can't find the container with id b70708f662eb99389675f62b0c29bad00d29f2408066d8a00f810f79cf59d95d
Sep 10 19:15:47.449592 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:47.449325    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod299c1d38_48fb_48bd_a701_d89a74c1cecb.slice/crio-79ffe5d02431b5256ffeb8cc5a14a0c4eccc70f65c430b122bbfef2b9d7e2fc3 WatchSource:0}: Error finding container 79ffe5d02431b5256ffeb8cc5a14a0c4eccc70f65c430b122bbfef2b9d7e2fc3: Status 404 returned error can't find the container with id 79ffe5d02431b5256ffeb8cc5a14a0c4eccc70f65c430b122bbfef2b9d7e2fc3
Sep 10 19:15:47.450368 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:47.450339    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode08684c6_1cc0_4f4c_a4cd_1fea1e46339a.slice/crio-448a69a6ce6f09d4fc8f09410437d76277707e282ab4807e6b97ad0fc8368d75 WatchSource:0}: Error finding container 448a69a6ce6f09d4fc8f09410437d76277707e282ab4807e6b97ad0fc8368d75: Status 404 returned error can't find the container with id 448a69a6ce6f09d4fc8f09410437d76277707e282ab4807e6b97ad0fc8368d75
Sep 10 19:15:47.450996 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:15:47.450976    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10449b06_ca4d_4ee1_8d45_d33f3bd9aea1.slice/crio-7cf09f130374edc9f5baee0d0b2dea0880c4e000003b375dc4bfbb6c95bfe43d WatchSource:0}: Error finding container 7cf09f130374edc9f5baee0d0b2dea0880c4e000003b375dc4bfbb6c95bfe43d: Status 404 returned error can't find the container with id 7cf09f130374edc9f5baee0d0b2dea0880c4e000003b375dc4bfbb6c95bfe43d
Sep 10 19:15:47.538688 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.538540    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqrjw\" (UniqueName: \"kubernetes.io/projected/24f84abd-cd7a-40fa-b23f-72e60cc247b8-kube-api-access-qqrjw\") pod \"network-check-target-9stjj\" (UID: \"24f84abd-cd7a-40fa-b23f-72e60cc247b8\") " pod="openshift-network-diagnostics/network-check-target-9stjj"
Sep 10 19:15:47.538795 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:47.538707    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 19:15:47.538795 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:47.538727    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 19:15:47.538795 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:47.538737    2564 projected.go:196] Error preparing data for projected volume kube-api-access-qqrjw for pod openshift-network-diagnostics/network-check-target-9stjj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:15:47.538795 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:47.538778    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/24f84abd-cd7a-40fa-b23f-72e60cc247b8-kube-api-access-qqrjw podName:24f84abd-cd7a-40fa-b23f-72e60cc247b8 nodeName:}" failed. No retries permitted until 2026-09-10 19:15:48.538765391 +0000 UTC m=+4.139672609 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-qqrjw" (UniqueName: "kubernetes.io/projected/24f84abd-cd7a-40fa-b23f-72e60cc247b8-kube-api-access-qqrjw") pod "network-check-target-9stjj" (UID: "24f84abd-cd7a-40fa-b23f-72e60cc247b8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:15:47.858568 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.858482    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 19:10:46 +0000 UTC" deadline="2028-03-27 22:51:13.4067295 +0000 UTC"
Sep 10 19:15:47.858568 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.858512    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13539h35m25.548220887s"
Sep 10 19:15:47.943118 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.943085    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9stjj"
Sep 10 19:15:47.943281 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:47.943213    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9stjj" podUID="24f84abd-cd7a-40fa-b23f-72e60cc247b8"
Sep 10 19:15:47.959007 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.958945    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-230.ec2.internal" event={"ID":"099d282a8bec5dd8dd074b5f725a2d0c","Type":"ContainerStarted","Data":"8a0d47ed10339012c9f0e308d5581e9898957c19fba1a0e7604bb2df759cb86c"}
Sep 10 19:15:47.960503 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.960474    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mzwh9" event={"ID":"ac0ef973-45eb-4f42-8c5d-d86695e32507","Type":"ContainerStarted","Data":"b2b45d9676137c4c7eb309a6f101f4bb0ed33acb4054ab98802f9c40d06d75cf"}
Sep 10 19:15:47.962578 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.962550    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-klvjs" event={"ID":"10449b06-ca4d-4ee1-8d45-d33f3bd9aea1","Type":"ContainerStarted","Data":"7cf09f130374edc9f5baee0d0b2dea0880c4e000003b375dc4bfbb6c95bfe43d"}
Sep 10 19:15:47.964719 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.964643    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-lhhr2" event={"ID":"6e38a77e-c0f7-43c1-a915-fd6a036cbb0e","Type":"ContainerStarted","Data":"2629678efba014592d407c0c943e8ec9469f1b9f48cd5e5b9dccbb79941d167f"}
Sep 10 19:15:47.966655 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.966569    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-pvhzx" event={"ID":"c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2","Type":"ContainerStarted","Data":"50f8045ef9427fee02d27d00fce1deb13708ee53d2bd0f8fb3f62a9d9220de69"}
Sep 10 19:15:47.968570 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.968520    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-wv24c" event={"ID":"09feb671-e7d9-467f-ac39-5ad97eb572fa","Type":"ContainerStarted","Data":"c72d0afc5eaee9a48e378506d4fd2c94d3dbcbcd9e5d2083cc0550136c525d10"}
Sep 10 19:15:47.972906 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.972846    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-l8b8c" event={"ID":"e08684c6-1cc0-4f4c-a4cd-1fea1e46339a","Type":"ContainerStarted","Data":"448a69a6ce6f09d4fc8f09410437d76277707e282ab4807e6b97ad0fc8368d75"}
Sep 10 19:15:47.975591 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.975489    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-82crq" event={"ID":"299c1d38-48fb-48bd-a701-d89a74c1cecb","Type":"ContainerStarted","Data":"79ffe5d02431b5256ffeb8cc5a14a0c4eccc70f65c430b122bbfef2b9d7e2fc3"}
Sep 10 19:15:47.979778 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.979748    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7s7hj" event={"ID":"ae8c5b13-1925-4df6-b3a2-d4ace9a51801","Type":"ContainerStarted","Data":"b70708f662eb99389675f62b0c29bad00d29f2408066d8a00f810f79cf59d95d"}
Sep 10 19:15:47.981720 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:47.981655    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-5jjtz" event={"ID":"58cdd25d-25ab-403e-abcc-65f251ee8482","Type":"ContainerStarted","Data":"cb667ec040e4efb28d2f900f86e62de10afe3d5e43f7b5a83253d37c836ad014"}
Sep 10 19:15:48.445007 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:48.444973    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94b7fd86-07b1-4d2c-869f-787a7fcc6dca-metrics-certs\") pod \"network-metrics-daemon-tvlcw\" (UID: \"94b7fd86-07b1-4d2c-869f-787a7fcc6dca\") " pod="openshift-multus/network-metrics-daemon-tvlcw"
Sep 10 19:15:48.445206 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:48.445036    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/80a423f3-63d3-42fa-a233-166ab675777e-original-pull-secret\") pod \"global-pull-secret-syncer-vcs72\" (UID: \"80a423f3-63d3-42fa-a233-166ab675777e\") " pod="kube-system/global-pull-secret-syncer-vcs72"
Sep 10 19:15:48.445206 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:48.445177    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 19:15:48.445325 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:48.445239    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80a423f3-63d3-42fa-a233-166ab675777e-original-pull-secret podName:80a423f3-63d3-42fa-a233-166ab675777e nodeName:}" failed. No retries permitted until 2026-09-10 19:15:50.445221546 +0000 UTC m=+6.046128768 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/80a423f3-63d3-42fa-a233-166ab675777e-original-pull-secret") pod "global-pull-secret-syncer-vcs72" (UID: "80a423f3-63d3-42fa-a233-166ab675777e") : object "kube-system"/"original-pull-secret" not registered
Sep 10 19:15:48.446352 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:48.445644    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:15:48.446352 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:48.445707    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94b7fd86-07b1-4d2c-869f-787a7fcc6dca-metrics-certs podName:94b7fd86-07b1-4d2c-869f-787a7fcc6dca nodeName:}" failed. No retries permitted until 2026-09-10 19:15:50.445689927 +0000 UTC m=+6.046597148 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/94b7fd86-07b1-4d2c-869f-787a7fcc6dca-metrics-certs") pod "network-metrics-daemon-tvlcw" (UID: "94b7fd86-07b1-4d2c-869f-787a7fcc6dca") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:15:48.546326 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:48.546288    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqrjw\" (UniqueName: \"kubernetes.io/projected/24f84abd-cd7a-40fa-b23f-72e60cc247b8-kube-api-access-qqrjw\") pod \"network-check-target-9stjj\" (UID: \"24f84abd-cd7a-40fa-b23f-72e60cc247b8\") " pod="openshift-network-diagnostics/network-check-target-9stjj"
Sep 10 19:15:48.546511 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:48.546475    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 19:15:48.546511 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:48.546494    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 19:15:48.546511 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:48.546506    2564 projected.go:196] Error preparing data for projected volume kube-api-access-qqrjw for pod openshift-network-diagnostics/network-check-target-9stjj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:15:48.546688 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:48.546563    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/24f84abd-cd7a-40fa-b23f-72e60cc247b8-kube-api-access-qqrjw podName:24f84abd-cd7a-40fa-b23f-72e60cc247b8 nodeName:}" failed. No retries permitted until 2026-09-10 19:15:50.546544763 +0000 UTC m=+6.147451995 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-qqrjw" (UniqueName: "kubernetes.io/projected/24f84abd-cd7a-40fa-b23f-72e60cc247b8-kube-api-access-qqrjw") pod "network-check-target-9stjj" (UID: "24f84abd-cd7a-40fa-b23f-72e60cc247b8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:15:48.859740 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:48.859617    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 19:10:46 +0000 UTC" deadline="2028-02-13 15:10:51.29962925 +0000 UTC"
Sep 10 19:15:48.859740 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:48.859690    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12499h55m2.439943179s"
Sep 10 19:15:48.943406 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:48.943369    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvlcw"
Sep 10 19:15:48.943559 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:48.943502    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvlcw" podUID="94b7fd86-07b1-4d2c-869f-787a7fcc6dca"
Sep 10 19:15:48.943998 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:48.943977    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcs72"
Sep 10 19:15:48.944131 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:48.944079    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcs72" podUID="80a423f3-63d3-42fa-a233-166ab675777e"
Sep 10 19:15:49.942585 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:49.942550    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9stjj"
Sep 10 19:15:49.943167 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:49.942706    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9stjj" podUID="24f84abd-cd7a-40fa-b23f-72e60cc247b8"
Sep 10 19:15:50.463650 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:50.463599    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94b7fd86-07b1-4d2c-869f-787a7fcc6dca-metrics-certs\") pod \"network-metrics-daemon-tvlcw\" (UID: \"94b7fd86-07b1-4d2c-869f-787a7fcc6dca\") " pod="openshift-multus/network-metrics-daemon-tvlcw"
Sep 10 19:15:50.463842 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:50.463686    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/80a423f3-63d3-42fa-a233-166ab675777e-original-pull-secret\") pod \"global-pull-secret-syncer-vcs72\" (UID: \"80a423f3-63d3-42fa-a233-166ab675777e\") " pod="kube-system/global-pull-secret-syncer-vcs72"
Sep 10 19:15:50.463842 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:50.463748    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:15:50.463842 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:50.463781    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 19:15:50.463842 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:50.463817    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94b7fd86-07b1-4d2c-869f-787a7fcc6dca-metrics-certs podName:94b7fd86-07b1-4d2c-869f-787a7fcc6dca nodeName:}" failed. No retries permitted until 2026-09-10 19:15:54.463799923 +0000 UTC m=+10.064707160 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/94b7fd86-07b1-4d2c-869f-787a7fcc6dca-metrics-certs") pod "network-metrics-daemon-tvlcw" (UID: "94b7fd86-07b1-4d2c-869f-787a7fcc6dca") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:15:50.463842 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:50.463835    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80a423f3-63d3-42fa-a233-166ab675777e-original-pull-secret podName:80a423f3-63d3-42fa-a233-166ab675777e nodeName:}" failed. No retries permitted until 2026-09-10 19:15:54.463826678 +0000 UTC m=+10.064733899 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/80a423f3-63d3-42fa-a233-166ab675777e-original-pull-secret") pod "global-pull-secret-syncer-vcs72" (UID: "80a423f3-63d3-42fa-a233-166ab675777e") : object "kube-system"/"original-pull-secret" not registered
Sep 10 19:15:50.564457 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:50.564420    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqrjw\" (UniqueName: \"kubernetes.io/projected/24f84abd-cd7a-40fa-b23f-72e60cc247b8-kube-api-access-qqrjw\") pod \"network-check-target-9stjj\" (UID: \"24f84abd-cd7a-40fa-b23f-72e60cc247b8\") " pod="openshift-network-diagnostics/network-check-target-9stjj"
Sep 10 19:15:50.564645 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:50.564605    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 19:15:50.564725 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:50.564650    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 19:15:50.564725 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:50.564665    2564 projected.go:196] Error preparing data for projected volume kube-api-access-qqrjw for pod openshift-network-diagnostics/network-check-target-9stjj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:15:50.564725 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:50.564722    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/24f84abd-cd7a-40fa-b23f-72e60cc247b8-kube-api-access-qqrjw podName:24f84abd-cd7a-40fa-b23f-72e60cc247b8 nodeName:}" failed. No retries permitted until 2026-09-10 19:15:54.564704411 +0000 UTC m=+10.165611633 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-qqrjw" (UniqueName: "kubernetes.io/projected/24f84abd-cd7a-40fa-b23f-72e60cc247b8-kube-api-access-qqrjw") pod "network-check-target-9stjj" (UID: "24f84abd-cd7a-40fa-b23f-72e60cc247b8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:15:50.946431 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:50.946352    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvlcw"
Sep 10 19:15:50.946431 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:50.946376    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcs72"
Sep 10 19:15:50.946944 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:50.946466    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvlcw" podUID="94b7fd86-07b1-4d2c-869f-787a7fcc6dca"
Sep 10 19:15:50.946944 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:50.946519    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcs72" podUID="80a423f3-63d3-42fa-a233-166ab675777e"
Sep 10 19:15:51.943338 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:51.943302    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9stjj"
Sep 10 19:15:51.943514 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:51.943427    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9stjj" podUID="24f84abd-cd7a-40fa-b23f-72e60cc247b8"
Sep 10 19:15:52.946435 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:52.946405    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcs72"
Sep 10 19:15:52.946435 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:52.946432    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvlcw"
Sep 10 19:15:52.946892 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:52.946506    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcs72" podUID="80a423f3-63d3-42fa-a233-166ab675777e"
Sep 10 19:15:52.946892 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:52.946679    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvlcw" podUID="94b7fd86-07b1-4d2c-869f-787a7fcc6dca"
Sep 10 19:15:53.942920 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:53.942841    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9stjj"
Sep 10 19:15:53.943087 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:53.942958    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9stjj" podUID="24f84abd-cd7a-40fa-b23f-72e60cc247b8"
Sep 10 19:15:54.498242 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:54.498207    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94b7fd86-07b1-4d2c-869f-787a7fcc6dca-metrics-certs\") pod \"network-metrics-daemon-tvlcw\" (UID: \"94b7fd86-07b1-4d2c-869f-787a7fcc6dca\") " pod="openshift-multus/network-metrics-daemon-tvlcw"
Sep 10 19:15:54.498704 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:54.498276    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/80a423f3-63d3-42fa-a233-166ab675777e-original-pull-secret\") pod \"global-pull-secret-syncer-vcs72\" (UID: \"80a423f3-63d3-42fa-a233-166ab675777e\") " pod="kube-system/global-pull-secret-syncer-vcs72"
Sep 10 19:15:54.498704 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:54.498364    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:15:54.498704 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:54.498393    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 19:15:54.498704 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:54.498442    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94b7fd86-07b1-4d2c-869f-787a7fcc6dca-metrics-certs podName:94b7fd86-07b1-4d2c-869f-787a7fcc6dca nodeName:}" failed. No retries permitted until 2026-09-10 19:16:02.498420514 +0000 UTC m=+18.099327755 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/94b7fd86-07b1-4d2c-869f-787a7fcc6dca-metrics-certs") pod "network-metrics-daemon-tvlcw" (UID: "94b7fd86-07b1-4d2c-869f-787a7fcc6dca") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:15:54.498704 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:54.498462    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80a423f3-63d3-42fa-a233-166ab675777e-original-pull-secret podName:80a423f3-63d3-42fa-a233-166ab675777e nodeName:}" failed. No retries permitted until 2026-09-10 19:16:02.498451985 +0000 UTC m=+18.099359207 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/80a423f3-63d3-42fa-a233-166ab675777e-original-pull-secret") pod "global-pull-secret-syncer-vcs72" (UID: "80a423f3-63d3-42fa-a233-166ab675777e") : object "kube-system"/"original-pull-secret" not registered
Sep 10 19:15:54.599072 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:54.599040    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqrjw\" (UniqueName: \"kubernetes.io/projected/24f84abd-cd7a-40fa-b23f-72e60cc247b8-kube-api-access-qqrjw\") pod \"network-check-target-9stjj\" (UID: \"24f84abd-cd7a-40fa-b23f-72e60cc247b8\") " pod="openshift-network-diagnostics/network-check-target-9stjj"
Sep 10 19:15:54.599261 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:54.599203    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 19:15:54.599261 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:54.599223    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 19:15:54.599261 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:54.599234    2564 projected.go:196] Error preparing data for projected volume kube-api-access-qqrjw for pod openshift-network-diagnostics/network-check-target-9stjj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:15:54.599411 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:54.599325    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/24f84abd-cd7a-40fa-b23f-72e60cc247b8-kube-api-access-qqrjw podName:24f84abd-cd7a-40fa-b23f-72e60cc247b8 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:02.599306425 +0000 UTC m=+18.200213658 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-qqrjw" (UniqueName: "kubernetes.io/projected/24f84abd-cd7a-40fa-b23f-72e60cc247b8-kube-api-access-qqrjw") pod "network-check-target-9stjj" (UID: "24f84abd-cd7a-40fa-b23f-72e60cc247b8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:15:54.946482 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:54.946403    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcs72"
Sep 10 19:15:54.946659 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:54.946527    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcs72" podUID="80a423f3-63d3-42fa-a233-166ab675777e"
Sep 10 19:15:54.946948 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:54.946925    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvlcw"
Sep 10 19:15:54.947074 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:54.947034    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvlcw" podUID="94b7fd86-07b1-4d2c-869f-787a7fcc6dca"
Sep 10 19:15:55.942726 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:55.942691    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9stjj"
Sep 10 19:15:55.943181 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:55.942804    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9stjj" podUID="24f84abd-cd7a-40fa-b23f-72e60cc247b8"
Sep 10 19:15:56.942906 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:56.942876    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvlcw"
Sep 10 19:15:56.943357 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:56.943010    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvlcw" podUID="94b7fd86-07b1-4d2c-869f-787a7fcc6dca"
Sep 10 19:15:56.943357 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:56.942879    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcs72"
Sep 10 19:15:56.943357 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:56.943308    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcs72" podUID="80a423f3-63d3-42fa-a233-166ab675777e"
Sep 10 19:15:57.942733 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:57.942688    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9stjj"
Sep 10 19:15:57.942914 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:57.942797    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9stjj" podUID="24f84abd-cd7a-40fa-b23f-72e60cc247b8"
Sep 10 19:15:58.943451 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:58.943409    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvlcw"
Sep 10 19:15:58.944009 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:58.943540    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvlcw" podUID="94b7fd86-07b1-4d2c-869f-787a7fcc6dca"
Sep 10 19:15:58.944009 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:58.943581    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcs72"
Sep 10 19:15:58.944009 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:58.943671    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcs72" podUID="80a423f3-63d3-42fa-a233-166ab675777e"
Sep 10 19:15:59.943367 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:15:59.943327    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9stjj"
Sep 10 19:15:59.943538 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:15:59.943467    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9stjj" podUID="24f84abd-cd7a-40fa-b23f-72e60cc247b8"
Sep 10 19:16:00.946132 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:00.946103    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvlcw"
Sep 10 19:16:00.946132 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:00.946144    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcs72"
Sep 10 19:16:00.946699 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:00.946233    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvlcw" podUID="94b7fd86-07b1-4d2c-869f-787a7fcc6dca"
Sep 10 19:16:00.946699 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:00.946362    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcs72" podUID="80a423f3-63d3-42fa-a233-166ab675777e"
Sep 10 19:16:01.942917 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:01.942832    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9stjj"
Sep 10 19:16:01.943146 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:01.942959    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9stjj" podUID="24f84abd-cd7a-40fa-b23f-72e60cc247b8"
Sep 10 19:16:02.562804 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:02.562759    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/80a423f3-63d3-42fa-a233-166ab675777e-original-pull-secret\") pod \"global-pull-secret-syncer-vcs72\" (UID: \"80a423f3-63d3-42fa-a233-166ab675777e\") " pod="kube-system/global-pull-secret-syncer-vcs72"
Sep 10 19:16:02.563280 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:02.562862    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94b7fd86-07b1-4d2c-869f-787a7fcc6dca-metrics-certs\") pod \"network-metrics-daemon-tvlcw\" (UID: \"94b7fd86-07b1-4d2c-869f-787a7fcc6dca\") " pod="openshift-multus/network-metrics-daemon-tvlcw"
Sep 10 19:16:02.563280 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:02.562913    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 19:16:02.563280 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:02.562969    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:16:02.563280 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:02.563013    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80a423f3-63d3-42fa-a233-166ab675777e-original-pull-secret podName:80a423f3-63d3-42fa-a233-166ab675777e nodeName:}" failed. No retries permitted until 2026-09-10 19:16:18.562988895 +0000 UTC m=+34.163896148 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/80a423f3-63d3-42fa-a233-166ab675777e-original-pull-secret") pod "global-pull-secret-syncer-vcs72" (UID: "80a423f3-63d3-42fa-a233-166ab675777e") : object "kube-system"/"original-pull-secret" not registered
Sep 10 19:16:02.563280 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:02.563034    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94b7fd86-07b1-4d2c-869f-787a7fcc6dca-metrics-certs podName:94b7fd86-07b1-4d2c-869f-787a7fcc6dca nodeName:}" failed. No retries permitted until 2026-09-10 19:16:18.563023448 +0000 UTC m=+34.163930669 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/94b7fd86-07b1-4d2c-869f-787a7fcc6dca-metrics-certs") pod "network-metrics-daemon-tvlcw" (UID: "94b7fd86-07b1-4d2c-869f-787a7fcc6dca") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:16:02.663445 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:02.663412    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqrjw\" (UniqueName: \"kubernetes.io/projected/24f84abd-cd7a-40fa-b23f-72e60cc247b8-kube-api-access-qqrjw\") pod \"network-check-target-9stjj\" (UID: \"24f84abd-cd7a-40fa-b23f-72e60cc247b8\") " pod="openshift-network-diagnostics/network-check-target-9stjj"
Sep 10 19:16:02.663644 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:02.663560    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 19:16:02.663644 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:02.663578    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 19:16:02.663644 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:02.663587    2564 projected.go:196] Error preparing data for projected volume kube-api-access-qqrjw for pod openshift-network-diagnostics/network-check-target-9stjj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:16:02.663817 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:02.663648    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/24f84abd-cd7a-40fa-b23f-72e60cc247b8-kube-api-access-qqrjw podName:24f84abd-cd7a-40fa-b23f-72e60cc247b8 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:18.66363477 +0000 UTC m=+34.264542000 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-qqrjw" (UniqueName: "kubernetes.io/projected/24f84abd-cd7a-40fa-b23f-72e60cc247b8-kube-api-access-qqrjw") pod "network-check-target-9stjj" (UID: "24f84abd-cd7a-40fa-b23f-72e60cc247b8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:16:02.945463 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:02.945378    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvlcw"
Sep 10 19:16:02.945463 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:02.945403    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcs72"
Sep 10 19:16:02.945662 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:02.945511    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvlcw" podUID="94b7fd86-07b1-4d2c-869f-787a7fcc6dca"
Sep 10 19:16:02.945662 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:02.945634    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcs72" podUID="80a423f3-63d3-42fa-a233-166ab675777e"
Sep 10 19:16:03.942748 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:03.942714    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9stjj"
Sep 10 19:16:03.943362 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:03.942841    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9stjj" podUID="24f84abd-cd7a-40fa-b23f-72e60cc247b8"
Sep 10 19:16:04.943534 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:04.943432    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvlcw"
Sep 10 19:16:04.944008 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:04.943551    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvlcw" podUID="94b7fd86-07b1-4d2c-869f-787a7fcc6dca"
Sep 10 19:16:04.944008 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:04.943612    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcs72"
Sep 10 19:16:04.944008 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:04.943735    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcs72" podUID="80a423f3-63d3-42fa-a233-166ab675777e"
Sep 10 19:16:05.942603 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:05.942424    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9stjj"
Sep 10 19:16:05.942769 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:05.942697    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9stjj" podUID="24f84abd-cd7a-40fa-b23f-72e60cc247b8"
Sep 10 19:16:06.033989 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:06.033958    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-pvhzx" event={"ID":"c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2","Type":"ContainerStarted","Data":"2d78eac85a076b5cc025170f2b508289ce47df9dbdb71c4ac87378e7689a1ce2"}
Sep 10 19:16:06.035760 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:06.035728    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-wv24c" event={"ID":"09feb671-e7d9-467f-ac39-5ad97eb572fa","Type":"ContainerStarted","Data":"347f4adaec585a971c0ee4ffefe2ff70cc0defeaca118ff100034cfb5ba0c8e1"}
Sep 10 19:16:06.051725 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:06.051679    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-230.ec2.internal" podStartSLOduration=20.051643355 podStartE2EDuration="20.051643355s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:15:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:15:47.979744582 +0000 UTC m=+3.580651821" watchObservedRunningTime="2026-09-10 19:16:06.051643355 +0000 UTC m=+21.652550594"
Sep 10 19:16:06.052556 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:06.052473    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-pvhzx" podStartSLOduration=3.950788971 podStartE2EDuration="22.052455418s" totalImagesPullingTime="18.101666447s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:15:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:06.05160322 +0000 UTC m=+21.652510461" watchObservedRunningTime="2026-09-10 19:16:06.052455418 +0000 UTC m=+21.653362658"
Sep 10 19:16:06.068306 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:06.068239    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-wv24c" podStartSLOduration=2.996448878 podStartE2EDuration="21.068216814s" totalImagesPullingTime="18.071767936s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:15:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:06.067851476 +0000 UTC m=+21.668758718" watchObservedRunningTime="2026-09-10 19:16:06.068216814 +0000 UTC m=+21.669124055"
Sep 10 19:16:06.943660 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:06.943402    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvlcw"
Sep 10 19:16:06.943774 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:06.943457    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcs72"
Sep 10 19:16:06.943824 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:06.943791    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvlcw" podUID="94b7fd86-07b1-4d2c-869f-787a7fcc6dca"
Sep 10 19:16:06.943864 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:06.943843    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcs72" podUID="80a423f3-63d3-42fa-a233-166ab675777e"
Sep 10 19:16:07.039839 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:07.039808    2564 generic.go:356] "Generic (PLEG): container finished" podID="8daf278338a7c357076b7909cbd675ea" containerID="8fe855fe99e55002df216d7308335120c260f59b88b725354385d81415abb805" exitCode=0
Sep 10 19:16:07.040493 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:07.039901    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-230.ec2.internal" event={"ID":"8daf278338a7c357076b7909cbd675ea","Type":"ContainerDied","Data":"8fe855fe99e55002df216d7308335120c260f59b88b725354385d81415abb805"}
Sep 10 19:16:07.041834 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:07.041800    2564 generic.go:356] "Generic (PLEG): container finished" podID="ac0ef973-45eb-4f42-8c5d-d86695e32507" containerID="6fc26bb0d8a21f42eceb8a8f56591f17224d2252ec0e884d6ded94aa23a2580f" exitCode=0
Sep 10 19:16:07.041961 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:07.041889    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mzwh9" event={"ID":"ac0ef973-45eb-4f42-8c5d-d86695e32507","Type":"ContainerDied","Data":"6fc26bb0d8a21f42eceb8a8f56591f17224d2252ec0e884d6ded94aa23a2580f"}
Sep 10 19:16:07.043996 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:07.043965    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-klvjs" event={"ID":"10449b06-ca4d-4ee1-8d45-d33f3bd9aea1","Type":"ContainerStarted","Data":"3d18a08d856cc50155c1b28f770da6d0b10f8c70676199a3471602a71ff0d823"}
Sep 10 19:16:07.045681 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:07.045653    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-lhhr2" event={"ID":"6e38a77e-c0f7-43c1-a915-fd6a036cbb0e","Type":"ContainerStarted","Data":"bfa8374e3eb6b0a1bb2d0324c6f31219343261c6c841b3cf498da24b8f93dfa9"}
Sep 10 19:16:07.047198 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:07.047177    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-l8b8c" event={"ID":"e08684c6-1cc0-4f4c-a4cd-1fea1e46339a","Type":"ContainerStarted","Data":"35b4a624ea7be85fa1410fa7fc3731ca34de55e97477bdb4297aa512e63a553e"}
Sep 10 19:16:07.050384 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:07.050365    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-82crq" event={"ID":"299c1d38-48fb-48bd-a701-d89a74c1cecb","Type":"ContainerStarted","Data":"9aa290ab2e439a4fa6bbec882281b9958bd3b9e15c12c2e82953c242c917cd8a"}
Sep 10 19:16:07.050470 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:07.050430    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-82crq" event={"ID":"299c1d38-48fb-48bd-a701-d89a74c1cecb","Type":"ContainerStarted","Data":"abc892ea3fb1402a5ef052dd0a1c908ce811cf3f559df844a4370668fdd228be"}
Sep 10 19:16:07.050537 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:07.050470    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-82crq" event={"ID":"299c1d38-48fb-48bd-a701-d89a74c1cecb","Type":"ContainerStarted","Data":"fdf8c7ed249fe3955bb6ed6cc4e5e7be515dd14afd01f573c861dc9afbd5c006"}
Sep 10 19:16:07.050537 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:07.050497    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-82crq" event={"ID":"299c1d38-48fb-48bd-a701-d89a74c1cecb","Type":"ContainerStarted","Data":"89c66356374b8831ac1bc4f62be571ba83ca2287c4705107dcb784c610e27c4b"}
Sep 10 19:16:07.050537 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:07.050530    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-82crq" event={"ID":"299c1d38-48fb-48bd-a701-d89a74c1cecb","Type":"ContainerStarted","Data":"5b7d40b2f5f4bc5599d5095aeb9469f162f204734c26be7d51cff0b5a20ea766"}
Sep 10 19:16:07.050776 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:07.050557    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-82crq" event={"ID":"299c1d38-48fb-48bd-a701-d89a74c1cecb","Type":"ContainerStarted","Data":"ed2a0af795c7f2060f81e242d75a8a527b4d893858e4c775253572282c6f89bb"}
Sep 10 19:16:07.051852 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:07.051830    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7s7hj" event={"ID":"ae8c5b13-1925-4df6-b3a2-d4ace9a51801","Type":"ContainerStarted","Data":"c36ec903597b83b73241dc0dc939b94dcacef6da8e21ad4bf7c11361ae041175"}
Sep 10 19:16:07.097857 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:07.097809    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-lhhr2" podStartSLOduration=4.080555071 podStartE2EDuration="22.097792363s" totalImagesPullingTime="18.017237292s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:15:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:07.097364607 +0000 UTC m=+22.698271857" watchObservedRunningTime="2026-09-10 19:16:07.097792363 +0000 UTC m=+22.698699606"
Sep 10 19:16:07.112134 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:07.112087    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-l8b8c" podStartSLOduration=4.076017537 podStartE2EDuration="22.112066427s" totalImagesPullingTime="18.03604889s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:15:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:07.111774664 +0000 UTC m=+22.712681906" watchObservedRunningTime="2026-09-10 19:16:07.112066427 +0000 UTC m=+22.712973646"
Sep 10 19:16:07.126664 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:07.126610    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-7s7hj" podStartSLOduration=4.086009973 podStartE2EDuration="22.126597157s" totalImagesPullingTime="18.040587184s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:15:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:07.126419139 +0000 UTC m=+22.727326378" watchObservedRunningTime="2026-09-10 19:16:07.126597157 +0000 UTC m=+22.727504396"
Sep 10 19:16:07.942822 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:07.942789    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9stjj"
Sep 10 19:16:07.943042 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:07.942885    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9stjj" podUID="24f84abd-cd7a-40fa-b23f-72e60cc247b8"
Sep 10 19:16:08.056782 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:08.056506    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-5jjtz" event={"ID":"58cdd25d-25ab-403e-abcc-65f251ee8482","Type":"ContainerStarted","Data":"b58d0c15ecb4815a30ef5861f3f7910336b9c5159416e264c4efa7d641c58aec"}
Sep 10 19:16:08.058687 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:08.058652    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-230.ec2.internal" event={"ID":"8daf278338a7c357076b7909cbd675ea","Type":"ContainerStarted","Data":"34f3995542ee1c5c2f0245c8a392b1fcd86f821c2bf287903205ab11fa7ec29a"}
Sep 10 19:16:08.077712 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:08.077658    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-5jjtz" podStartSLOduration=5.008595502 podStartE2EDuration="23.077644521s" totalImagesPullingTime="18.069049019s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:15:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:08.077432156 +0000 UTC m=+23.678339399" watchObservedRunningTime="2026-09-10 19:16:08.077644521 +0000 UTC m=+23.678551761"
Sep 10 19:16:08.099657 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:08.099594    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-230.ec2.internal" podStartSLOduration=22.099578655 podStartE2EDuration="22.099578655s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:15:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:08.098276848 +0000 UTC m=+23.699184089" watchObservedRunningTime="2026-09-10 19:16:08.099578655 +0000 UTC m=+23.700485897"
Sep 10 19:16:08.943309 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:08.943275    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvlcw"
Sep 10 19:16:08.943756 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:08.943416    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvlcw" podUID="94b7fd86-07b1-4d2c-869f-787a7fcc6dca"
Sep 10 19:16:08.943756 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:08.943457    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcs72"
Sep 10 19:16:08.943756 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:08.943536    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcs72" podUID="80a423f3-63d3-42fa-a233-166ab675777e"
Sep 10 19:16:09.787153 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:09.787118    2564 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 10 19:16:09.875367 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:09.875281    2564 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-10T19:16:09.787138348Z","UUID":"13122160-6546-490f-821f-0305690e0d1e","Handler":null,"Name":"","Endpoint":""}
Sep 10 19:16:09.877057 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:09.877033    2564 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 10 19:16:09.877164 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:09.877064    2564 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 10 19:16:09.908948 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:09.908928    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-lhhr2"
Sep 10 19:16:09.942718 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:09.942591    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9stjj"
Sep 10 19:16:09.942887 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:09.942730    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9stjj" podUID="24f84abd-cd7a-40fa-b23f-72e60cc247b8"
Sep 10 19:16:10.068152 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:10.068118    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-82crq" event={"ID":"299c1d38-48fb-48bd-a701-d89a74c1cecb","Type":"ContainerStarted","Data":"39d256e6a7c25be8891257a88ec5126d5c1798a260876e7199bdcd113c353b35"}
Sep 10 19:16:10.073704 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:10.073659    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-klvjs" event={"ID":"10449b06-ca4d-4ee1-8d45-d33f3bd9aea1","Type":"ContainerStarted","Data":"43afeb482ca114c030135c70c55c7ae397f147ac1ce313d44d1a023c65a58847"}
Sep 10 19:16:10.942579 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:10.942507    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcs72"
Sep 10 19:16:10.942997 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:10.942508    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvlcw"
Sep 10 19:16:10.942997 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:10.942653    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcs72" podUID="80a423f3-63d3-42fa-a233-166ab675777e"
Sep 10 19:16:10.942997 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:10.942719    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvlcw" podUID="94b7fd86-07b1-4d2c-869f-787a7fcc6dca"
Sep 10 19:16:11.078846 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:11.078811    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-klvjs" event={"ID":"10449b06-ca4d-4ee1-8d45-d33f3bd9aea1","Type":"ContainerStarted","Data":"846c2f03085aa359373f7a99126bb237c9e2ecf9037a880e40beb15aae0aa484"}
Sep 10 19:16:11.106036 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:11.105980    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-klvjs" podStartSLOduration=11.126713353 podStartE2EDuration="26.105964535s" totalImagesPullingTime="14.979251182s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:15:45 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:11.105659913 +0000 UTC m=+26.706567176" watchObservedRunningTime="2026-09-10 19:16:11.105964535 +0000 UTC m=+26.706871775"
Sep 10 19:16:11.888131 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:11.888098    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-lhhr2"
Sep 10 19:16:11.888743 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:11.888722    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-lhhr2"
Sep 10 19:16:11.942693 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:11.942660    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9stjj"
Sep 10 19:16:11.943083 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:11.942775    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9stjj" podUID="24f84abd-cd7a-40fa-b23f-72e60cc247b8"
Sep 10 19:16:12.081797 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:12.081762    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-lhhr2"
Sep 10 19:16:12.946495 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:12.946285    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvlcw"
Sep 10 19:16:12.946815 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:12.946320    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcs72"
Sep 10 19:16:12.946815 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:12.946574    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvlcw" podUID="94b7fd86-07b1-4d2c-869f-787a7fcc6dca"
Sep 10 19:16:12.946815 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:12.946608    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcs72" podUID="80a423f3-63d3-42fa-a233-166ab675777e"
Sep 10 19:16:13.084687 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:13.084651    2564 generic.go:356] "Generic (PLEG): container finished" podID="ac0ef973-45eb-4f42-8c5d-d86695e32507" containerID="ad713ff9d32d0a33b5a2bade333b552823145da9dbd8bd6657489337340bf6ac" exitCode=0
Sep 10 19:16:13.084842 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:13.084689    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mzwh9" event={"ID":"ac0ef973-45eb-4f42-8c5d-d86695e32507","Type":"ContainerDied","Data":"ad713ff9d32d0a33b5a2bade333b552823145da9dbd8bd6657489337340bf6ac"}
Sep 10 19:16:13.088860 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:13.088838    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-82crq" event={"ID":"299c1d38-48fb-48bd-a701-d89a74c1cecb","Type":"ContainerStarted","Data":"037fd623cf6d154d816128f725ea114220f53d8e345899cc562f5657efd72b6f"}
Sep 10 19:16:13.132335 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:13.132290    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-82crq" podStartSLOduration=9.628986015 podStartE2EDuration="28.13227623s" totalImagesPullingTime="18.503290215s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:15:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:13.13202937 +0000 UTC m=+28.732936611" watchObservedRunningTime="2026-09-10 19:16:13.13227623 +0000 UTC m=+28.733183471"
Sep 10 19:16:13.942568 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:13.942503    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9stjj"
Sep 10 19:16:13.942718 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:13.942607    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9stjj" podUID="24f84abd-cd7a-40fa-b23f-72e60cc247b8"
Sep 10 19:16:14.092445 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:14.092417    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:16:14.093260 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:14.092510    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:16:14.093260 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:14.092565    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:16:14.109040 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:14.109016    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:16:14.109237 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:14.109219    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:16:14.332647 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:14.332220    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vcs72"]
Sep 10 19:16:14.332820 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:14.332690    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcs72"
Sep 10 19:16:14.332886 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:14.332813    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcs72" podUID="80a423f3-63d3-42fa-a233-166ab675777e"
Sep 10 19:16:14.333395 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:14.333356    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9stjj"]
Sep 10 19:16:14.333505 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:14.333454    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9stjj"
Sep 10 19:16:14.333571 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:14.333537    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9stjj" podUID="24f84abd-cd7a-40fa-b23f-72e60cc247b8"
Sep 10 19:16:14.334050 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:14.334023    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-tvlcw"]
Sep 10 19:16:14.334146 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:14.334115    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvlcw"
Sep 10 19:16:14.334211 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:14.334193    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvlcw" podUID="94b7fd86-07b1-4d2c-869f-787a7fcc6dca"
Sep 10 19:16:15.095600 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:15.095564    2564 generic.go:356] "Generic (PLEG): container finished" podID="ac0ef973-45eb-4f42-8c5d-d86695e32507" containerID="ac467bd113badbe009b33ef1557518058e1a954261ebb08359c86c6efd3ffd35" exitCode=0
Sep 10 19:16:15.096058 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:15.095656    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mzwh9" event={"ID":"ac0ef973-45eb-4f42-8c5d-d86695e32507","Type":"ContainerDied","Data":"ac467bd113badbe009b33ef1557518058e1a954261ebb08359c86c6efd3ffd35"}
Sep 10 19:16:15.942479 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:15.942452    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9stjj"
Sep 10 19:16:15.942664 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:15.942456    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcs72"
Sep 10 19:16:15.942664 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:15.942551    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9stjj" podUID="24f84abd-cd7a-40fa-b23f-72e60cc247b8"
Sep 10 19:16:15.942790 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:15.942683    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcs72" podUID="80a423f3-63d3-42fa-a233-166ab675777e"
Sep 10 19:16:15.942790 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:15.942455    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvlcw"
Sep 10 19:16:15.942790 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:15.942773    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvlcw" podUID="94b7fd86-07b1-4d2c-869f-787a7fcc6dca"
Sep 10 19:16:17.106068 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:17.106026    2564 generic.go:356] "Generic (PLEG): container finished" podID="ac0ef973-45eb-4f42-8c5d-d86695e32507" containerID="19ceeb8a51975b750554edbfaaeda6f0a909854c08d4b1a8cb6cc6db2c0b594f" exitCode=0
Sep 10 19:16:17.106068 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:17.106075    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mzwh9" event={"ID":"ac0ef973-45eb-4f42-8c5d-d86695e32507","Type":"ContainerDied","Data":"19ceeb8a51975b750554edbfaaeda6f0a909854c08d4b1a8cb6cc6db2c0b594f"}
Sep 10 19:16:17.942859 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:17.942829    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcs72"
Sep 10 19:16:17.943036 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:17.942816    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9stjj"
Sep 10 19:16:17.943036 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:17.942936    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcs72" podUID="80a423f3-63d3-42fa-a233-166ab675777e"
Sep 10 19:16:17.943137 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:17.943045    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9stjj" podUID="24f84abd-cd7a-40fa-b23f-72e60cc247b8"
Sep 10 19:16:17.943137 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:17.943121    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvlcw"
Sep 10 19:16:17.943236 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:17.943209    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvlcw" podUID="94b7fd86-07b1-4d2c-869f-787a7fcc6dca"
Sep 10 19:16:18.568176 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:18.568145    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94b7fd86-07b1-4d2c-869f-787a7fcc6dca-metrics-certs\") pod \"network-metrics-daemon-tvlcw\" (UID: \"94b7fd86-07b1-4d2c-869f-787a7fcc6dca\") " pod="openshift-multus/network-metrics-daemon-tvlcw"
Sep 10 19:16:18.568607 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:18.568194    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/80a423f3-63d3-42fa-a233-166ab675777e-original-pull-secret\") pod \"global-pull-secret-syncer-vcs72\" (UID: \"80a423f3-63d3-42fa-a233-166ab675777e\") " pod="kube-system/global-pull-secret-syncer-vcs72"
Sep 10 19:16:18.568607 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:18.568305    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 19:16:18.568607 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:18.568362    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80a423f3-63d3-42fa-a233-166ab675777e-original-pull-secret podName:80a423f3-63d3-42fa-a233-166ab675777e nodeName:}" failed. No retries permitted until 2026-09-10 19:16:50.568349266 +0000 UTC m=+66.169256484 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/80a423f3-63d3-42fa-a233-166ab675777e-original-pull-secret") pod "global-pull-secret-syncer-vcs72" (UID: "80a423f3-63d3-42fa-a233-166ab675777e") : object "kube-system"/"original-pull-secret" not registered
Sep 10 19:16:18.568607 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:18.568305    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:16:18.568607 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:18.568430    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94b7fd86-07b1-4d2c-869f-787a7fcc6dca-metrics-certs podName:94b7fd86-07b1-4d2c-869f-787a7fcc6dca nodeName:}" failed. No retries permitted until 2026-09-10 19:16:50.56841734 +0000 UTC m=+66.169324573 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/94b7fd86-07b1-4d2c-869f-787a7fcc6dca-metrics-certs") pod "network-metrics-daemon-tvlcw" (UID: "94b7fd86-07b1-4d2c-869f-787a7fcc6dca") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:16:18.668902 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:18.668860    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqrjw\" (UniqueName: \"kubernetes.io/projected/24f84abd-cd7a-40fa-b23f-72e60cc247b8-kube-api-access-qqrjw\") pod \"network-check-target-9stjj\" (UID: \"24f84abd-cd7a-40fa-b23f-72e60cc247b8\") " pod="openshift-network-diagnostics/network-check-target-9stjj"
Sep 10 19:16:18.669164 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:18.669053    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 19:16:18.669164 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:18.669079    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 19:16:18.669164 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:18.669092    2564 projected.go:196] Error preparing data for projected volume kube-api-access-qqrjw for pod openshift-network-diagnostics/network-check-target-9stjj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:16:18.669164 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:18.669156    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/24f84abd-cd7a-40fa-b23f-72e60cc247b8-kube-api-access-qqrjw podName:24f84abd-cd7a-40fa-b23f-72e60cc247b8 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:50.669136944 +0000 UTC m=+66.270044180 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-qqrjw" (UniqueName: "kubernetes.io/projected/24f84abd-cd7a-40fa-b23f-72e60cc247b8-kube-api-access-qqrjw") pod "network-check-target-9stjj" (UID: "24f84abd-cd7a-40fa-b23f-72e60cc247b8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:16:19.697701 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:19.697604    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-230.ec2.internal" event="NodeReady"
Sep 10 19:16:19.698188 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:19.697779    2564 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 10 19:16:19.751868 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:19.751820    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6cbff68f97-mt7bn"]
Sep 10 19:16:19.782531 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:19.782503    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-tc2c7"]
Sep 10 19:16:19.782683 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:19.782635    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cbff68f97-mt7bn"
Sep 10 19:16:19.784985 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:19.784960    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 10 19:16:19.784985 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:19.784960    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 10 19:16:19.785408 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:19.785392    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 10 19:16:19.785503 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:19.785414    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-qxq2w\""
Sep 10 19:16:19.791241 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:19.791221    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 10 19:16:19.796600 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:19.796581    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-67nsx"]
Sep 10 19:16:19.796714 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:19.796701    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tc2c7"
Sep 10 19:16:19.798980 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:19.798962    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 10 19:16:19.799091 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:19.799023    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-rjc7l\""
Sep 10 19:16:19.799091 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:19.799028    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 10 19:16:19.799091 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:19.799082    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 10 19:16:19.821207 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:19.821176    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tc2c7"]
Sep 10 19:16:19.821207 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:19.821212    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6cbff68f97-mt7bn"]
Sep 10 19:16:19.821409 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:19.821228    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-67nsx"]
Sep 10 19:16:19.821409 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:19.821249    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-67nsx"
Sep 10 19:16:19.823575 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:19.823558    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wkm4c\""
Sep 10 19:16:19.823694 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:19.823592    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 10 19:16:19.823694 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:19.823590    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 10 19:16:19.876865 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:19.876829    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0c459a94-07c5-44fb-b155-8a8f7aa46d25-image-registry-private-configuration\") pod \"image-registry-6cbff68f97-mt7bn\" (UID: \"0c459a94-07c5-44fb-b155-8a8f7aa46d25\") " pod="openshift-image-registry/image-registry-6cbff68f97-mt7bn"
Sep 10 19:16:19.877038 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:19.876870    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2vg4\" (UniqueName: \"kubernetes.io/projected/0c459a94-07c5-44fb-b155-8a8f7aa46d25-kube-api-access-q2vg4\") pod \"image-registry-6cbff68f97-mt7bn\" (UID: \"0c459a94-07c5-44fb-b155-8a8f7aa46d25\") " pod="openshift-image-registry/image-registry-6cbff68f97-mt7bn"
Sep 10 19:16:19.877038 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:19.876912    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6lk2\" (UniqueName: \"kubernetes.io/projected/90e5b1e2-b5f3-4306-8e86-363010cceb60-kube-api-access-z6lk2\") pod \"ingress-canary-tc2c7\" (UID: \"90e5b1e2-b5f3-4306-8e86-363010cceb60\") " pod="openshift-ingress-canary/ingress-canary-tc2c7"
Sep 10 19:16:19.877038 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:19.876938    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0c459a94-07c5-44fb-b155-8a8f7aa46d25-bound-sa-token\") pod \"image-registry-6cbff68f97-mt7bn\" (UID: \"0c459a94-07c5-44fb-b155-8a8f7aa46d25\") " pod="openshift-image-registry/image-registry-6cbff68f97-mt7bn"
Sep 10 19:16:19.877038 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:19.876960    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0c459a94-07c5-44fb-b155-8a8f7aa46d25-trusted-ca\") pod \"image-registry-6cbff68f97-mt7bn\" (UID: \"0c459a94-07c5-44fb-b155-8a8f7aa46d25\") " pod="openshift-image-registry/image-registry-6cbff68f97-mt7bn"
Sep 10 19:16:19.877253 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:19.877063    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0c459a94-07c5-44fb-b155-8a8f7aa46d25-registry-certificates\") pod \"image-registry-6cbff68f97-mt7bn\" (UID: \"0c459a94-07c5-44fb-b155-8a8f7aa46d25\") " pod="openshift-image-registry/image-registry-6cbff68f97-mt7bn"
Sep 10 19:16:19.877253 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:19.877136    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/90e5b1e2-b5f3-4306-8e86-363010cceb60-cert\") pod \"ingress-canary-tc2c7\" (UID: \"90e5b1e2-b5f3-4306-8e86-363010cceb60\") " pod="openshift-ingress-canary/ingress-canary-tc2c7"
Sep 10 19:16:19.877253 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:19.877162    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0c459a94-07c5-44fb-b155-8a8f7aa46d25-registry-tls\") pod \"image-registry-6cbff68f97-mt7bn\" (UID: \"0c459a94-07c5-44fb-b155-8a8f7aa46d25\") " pod="openshift-image-registry/image-registry-6cbff68f97-mt7bn"
Sep 10 19:16:19.877253 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:19.877194    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0c459a94-07c5-44fb-b155-8a8f7aa46d25-installation-pull-secrets\") pod \"image-registry-6cbff68f97-mt7bn\" (UID: \"0c459a94-07c5-44fb-b155-8a8f7aa46d25\") " pod="openshift-image-registry/image-registry-6cbff68f97-mt7bn"
Sep 10 19:16:19.877253 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:19.877224    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0c459a94-07c5-44fb-b155-8a8f7aa46d25-ca-trust-extracted\") pod \"image-registry-6cbff68f97-mt7bn\" (UID: \"0c459a94-07c5-44fb-b155-8a8f7aa46d25\") " pod="openshift-image-registry/image-registry-6cbff68f97-mt7bn"
Sep 10 19:16:19.943285 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:19.943250    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9stjj"
Sep 10 19:16:19.943285 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:19.943277    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvlcw"
Sep 10 19:16:19.943525 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:19.943257    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcs72"
Sep 10 19:16:19.946023 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:19.945995    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 19:16:19.946156 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:19.946022    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 10 19:16:19.946156 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:19.946115    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-8fmlh\""
Sep 10 19:16:19.946301 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:19.946282    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gqr5m\""
Sep 10 19:16:19.946301 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:19.946295    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 19:16:19.946415 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:19.946324    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 10 19:16:19.978113 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:19.978077    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0c459a94-07c5-44fb-b155-8a8f7aa46d25-image-registry-private-configuration\") pod \"image-registry-6cbff68f97-mt7bn\" (UID: \"0c459a94-07c5-44fb-b155-8a8f7aa46d25\") " pod="openshift-image-registry/image-registry-6cbff68f97-mt7bn"
Sep 10 19:16:19.978262 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:19.978114    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2vg4\" (UniqueName: \"kubernetes.io/projected/0c459a94-07c5-44fb-b155-8a8f7aa46d25-kube-api-access-q2vg4\") pod \"image-registry-6cbff68f97-mt7bn\" (UID: \"0c459a94-07c5-44fb-b155-8a8f7aa46d25\") " pod="openshift-image-registry/image-registry-6cbff68f97-mt7bn"
Sep 10 19:16:19.978262 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:19.978152    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6lk2\" (UniqueName: \"kubernetes.io/projected/90e5b1e2-b5f3-4306-8e86-363010cceb60-kube-api-access-z6lk2\") pod \"ingress-canary-tc2c7\" (UID: \"90e5b1e2-b5f3-4306-8e86-363010cceb60\") " pod="openshift-ingress-canary/ingress-canary-tc2c7"
Sep 10 19:16:19.978262 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:19.978177    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0c459a94-07c5-44fb-b155-8a8f7aa46d25-bound-sa-token\") pod \"image-registry-6cbff68f97-mt7bn\" (UID: \"0c459a94-07c5-44fb-b155-8a8f7aa46d25\") " pod="openshift-image-registry/image-registry-6cbff68f97-mt7bn"
Sep 10 19:16:19.978262 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:19.978203    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/28f7bb38-4380-48b2-8881-38f1b188b5c8-metrics-tls\") pod \"dns-default-67nsx\" (UID: \"28f7bb38-4380-48b2-8881-38f1b188b5c8\") " pod="openshift-dns/dns-default-67nsx"
Sep 10 19:16:19.978262 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:19.978224    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8bdq\" (UniqueName: \"kubernetes.io/projected/28f7bb38-4380-48b2-8881-38f1b188b5c8-kube-api-access-n8bdq\") pod \"dns-default-67nsx\" (UID: \"28f7bb38-4380-48b2-8881-38f1b188b5c8\") " pod="openshift-dns/dns-default-67nsx"
Sep 10 19:16:19.978262 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:19.978254    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0c459a94-07c5-44fb-b155-8a8f7aa46d25-trusted-ca\") pod \"image-registry-6cbff68f97-mt7bn\" (UID: \"0c459a94-07c5-44fb-b155-8a8f7aa46d25\") " pod="openshift-image-registry/image-registry-6cbff68f97-mt7bn"
Sep 10 19:16:19.978583 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:19.978316    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0c459a94-07c5-44fb-b155-8a8f7aa46d25-registry-certificates\") pod \"image-registry-6cbff68f97-mt7bn\" (UID: \"0c459a94-07c5-44fb-b155-8a8f7aa46d25\") " pod="openshift-image-registry/image-registry-6cbff68f97-mt7bn"
Sep 10 19:16:19.978583 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:19.978363    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/28f7bb38-4380-48b2-8881-38f1b188b5c8-config-volume\") pod \"dns-default-67nsx\" (UID: \"28f7bb38-4380-48b2-8881-38f1b188b5c8\") " pod="openshift-dns/dns-default-67nsx"
Sep 10 19:16:19.978583 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:19.978486    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28f7bb38-4380-48b2-8881-38f1b188b5c8-tmp-dir\") pod \"dns-default-67nsx\" (UID: \"28f7bb38-4380-48b2-8881-38f1b188b5c8\") " pod="openshift-dns/dns-default-67nsx"
Sep 10 19:16:19.978583 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:19.978530    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/90e5b1e2-b5f3-4306-8e86-363010cceb60-cert\") pod \"ingress-canary-tc2c7\" (UID: \"90e5b1e2-b5f3-4306-8e86-363010cceb60\") " pod="openshift-ingress-canary/ingress-canary-tc2c7"
Sep 10 19:16:19.978583 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:19.978554    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0c459a94-07c5-44fb-b155-8a8f7aa46d25-registry-tls\") pod \"image-registry-6cbff68f97-mt7bn\" (UID: \"0c459a94-07c5-44fb-b155-8a8f7aa46d25\") " pod="openshift-image-registry/image-registry-6cbff68f97-mt7bn"
Sep 10 19:16:19.978866 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:19.978589    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0c459a94-07c5-44fb-b155-8a8f7aa46d25-installation-pull-secrets\") pod \"image-registry-6cbff68f97-mt7bn\" (UID: \"0c459a94-07c5-44fb-b155-8a8f7aa46d25\") " pod="openshift-image-registry/image-registry-6cbff68f97-mt7bn"
Sep 10 19:16:19.978866 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:19.978648    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0c459a94-07c5-44fb-b155-8a8f7aa46d25-ca-trust-extracted\") pod \"image-registry-6cbff68f97-mt7bn\" (UID: \"0c459a94-07c5-44fb-b155-8a8f7aa46d25\") " pod="openshift-image-registry/image-registry-6cbff68f97-mt7bn"
Sep 10 19:16:19.978866 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:19.978676    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 19:16:19.978866 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:19.978692    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cbff68f97-mt7bn: secret "image-registry-tls" not found
Sep 10 19:16:19.978866 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:19.978750    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c459a94-07c5-44fb-b155-8a8f7aa46d25-registry-tls podName:0c459a94-07c5-44fb-b155-8a8f7aa46d25 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:20.478729937 +0000 UTC m=+36.079637174 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0c459a94-07c5-44fb-b155-8a8f7aa46d25-registry-tls") pod "image-registry-6cbff68f97-mt7bn" (UID: "0c459a94-07c5-44fb-b155-8a8f7aa46d25") : secret "image-registry-tls" not found
Sep 10 19:16:19.979142 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:19.978887    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0c459a94-07c5-44fb-b155-8a8f7aa46d25-registry-certificates\") pod \"image-registry-6cbff68f97-mt7bn\" (UID: \"0c459a94-07c5-44fb-b155-8a8f7aa46d25\") " pod="openshift-image-registry/image-registry-6cbff68f97-mt7bn"
Sep 10 19:16:19.979142 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:19.978972    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0c459a94-07c5-44fb-b155-8a8f7aa46d25-ca-trust-extracted\") pod \"image-registry-6cbff68f97-mt7bn\" (UID: \"0c459a94-07c5-44fb-b155-8a8f7aa46d25\") " pod="openshift-image-registry/image-registry-6cbff68f97-mt7bn"
Sep 10 19:16:19.979572 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:19.979550    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0c459a94-07c5-44fb-b155-8a8f7aa46d25-trusted-ca\") pod \"image-registry-6cbff68f97-mt7bn\" (UID: \"0c459a94-07c5-44fb-b155-8a8f7aa46d25\") " pod="openshift-image-registry/image-registry-6cbff68f97-mt7bn"
Sep 10 19:16:19.983073 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:19.983054    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0c459a94-07c5-44fb-b155-8a8f7aa46d25-image-registry-private-configuration\") pod \"image-registry-6cbff68f97-mt7bn\" (UID: \"0c459a94-07c5-44fb-b155-8a8f7aa46d25\") " pod="openshift-image-registry/image-registry-6cbff68f97-mt7bn"
Sep 10 19:16:19.983209 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:19.983072    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0c459a94-07c5-44fb-b155-8a8f7aa46d25-installation-pull-secrets\") pod \"image-registry-6cbff68f97-mt7bn\" (UID: \"0c459a94-07c5-44fb-b155-8a8f7aa46d25\") " pod="openshift-image-registry/image-registry-6cbff68f97-mt7bn"
Sep 10 19:16:19.983861 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:19.983843    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/90e5b1e2-b5f3-4306-8e86-363010cceb60-cert\") pod \"ingress-canary-tc2c7\" (UID: \"90e5b1e2-b5f3-4306-8e86-363010cceb60\") " pod="openshift-ingress-canary/ingress-canary-tc2c7"
Sep 10 19:16:19.990077 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:19.990054    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0c459a94-07c5-44fb-b155-8a8f7aa46d25-bound-sa-token\") pod \"image-registry-6cbff68f97-mt7bn\" (UID: \"0c459a94-07c5-44fb-b155-8a8f7aa46d25\") " pod="openshift-image-registry/image-registry-6cbff68f97-mt7bn"
Sep 10 19:16:19.993833 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:19.993808    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2vg4\" (UniqueName: \"kubernetes.io/projected/0c459a94-07c5-44fb-b155-8a8f7aa46d25-kube-api-access-q2vg4\") pod \"image-registry-6cbff68f97-mt7bn\" (UID: \"0c459a94-07c5-44fb-b155-8a8f7aa46d25\") " pod="openshift-image-registry/image-registry-6cbff68f97-mt7bn"
Sep 10 19:16:19.993938 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:19.993832    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6lk2\" (UniqueName: \"kubernetes.io/projected/90e5b1e2-b5f3-4306-8e86-363010cceb60-kube-api-access-z6lk2\") pod \"ingress-canary-tc2c7\" (UID: \"90e5b1e2-b5f3-4306-8e86-363010cceb60\") " pod="openshift-ingress-canary/ingress-canary-tc2c7"
Sep 10 19:16:20.035744 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:20.035705    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:20.079742 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:20.079696    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/28f7bb38-4380-48b2-8881-38f1b188b5c8-metrics-tls\") pod \"dns-default-67nsx\" (UID: \"28f7bb38-4380-48b2-8881-38f1b188b5c8\") " pod="openshift-dns/dns-default-67nsx"
Sep 10 19:16:20.079910 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:20.079750    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8bdq\" (UniqueName: \"kubernetes.io/projected/28f7bb38-4380-48b2-8881-38f1b188b5c8-kube-api-access-n8bdq\") pod \"dns-default-67nsx\" (UID: \"28f7bb38-4380-48b2-8881-38f1b188b5c8\") " pod="openshift-dns/dns-default-67nsx"
Sep 10 19:16:20.079910 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:20.079809    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 19:16:20.079910 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:20.079874    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28f7bb38-4380-48b2-8881-38f1b188b5c8-metrics-tls podName:28f7bb38-4380-48b2-8881-38f1b188b5c8 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:20.579854966 +0000 UTC m=+36.180762188 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/28f7bb38-4380-48b2-8881-38f1b188b5c8-metrics-tls") pod "dns-default-67nsx" (UID: "28f7bb38-4380-48b2-8881-38f1b188b5c8") : secret "dns-default-metrics-tls" not found
Sep 10 19:16:20.079910 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:20.079812    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/28f7bb38-4380-48b2-8881-38f1b188b5c8-config-volume\") pod \"dns-default-67nsx\" (UID: \"28f7bb38-4380-48b2-8881-38f1b188b5c8\") " pod="openshift-dns/dns-default-67nsx"
Sep 10 19:16:20.080121 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:20.079964    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28f7bb38-4380-48b2-8881-38f1b188b5c8-tmp-dir\") pod \"dns-default-67nsx\" (UID: \"28f7bb38-4380-48b2-8881-38f1b188b5c8\") " pod="openshift-dns/dns-default-67nsx"
Sep 10 19:16:20.080298 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:20.080279    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28f7bb38-4380-48b2-8881-38f1b188b5c8-tmp-dir\") pod \"dns-default-67nsx\" (UID: \"28f7bb38-4380-48b2-8881-38f1b188b5c8\") " pod="openshift-dns/dns-default-67nsx"
Sep 10 19:16:20.080382 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:20.080307    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/28f7bb38-4380-48b2-8881-38f1b188b5c8-config-volume\") pod \"dns-default-67nsx\" (UID: \"28f7bb38-4380-48b2-8881-38f1b188b5c8\") " pod="openshift-dns/dns-default-67nsx"
Sep 10 19:16:20.094246 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:20.094207    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8bdq\" (UniqueName: \"kubernetes.io/projected/28f7bb38-4380-48b2-8881-38f1b188b5c8-kube-api-access-n8bdq\") pod \"dns-default-67nsx\" (UID: \"28f7bb38-4380-48b2-8881-38f1b188b5c8\") " pod="openshift-dns/dns-default-67nsx"
Sep 10 19:16:20.105091 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:20.105035    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tc2c7"
Sep 10 19:16:20.135850 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:20.135812    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:20.198313 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:20.198276    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:20.483709 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:20.483673    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0c459a94-07c5-44fb-b155-8a8f7aa46d25-registry-tls\") pod \"image-registry-6cbff68f97-mt7bn\" (UID: \"0c459a94-07c5-44fb-b155-8a8f7aa46d25\") " pod="openshift-image-registry/image-registry-6cbff68f97-mt7bn"
Sep 10 19:16:20.483886 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:20.483846    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 19:16:20.483886 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:20.483870    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cbff68f97-mt7bn: secret "image-registry-tls" not found
Sep 10 19:16:20.483981 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:20.483939    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c459a94-07c5-44fb-b155-8a8f7aa46d25-registry-tls podName:0c459a94-07c5-44fb-b155-8a8f7aa46d25 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:21.483918258 +0000 UTC m=+37.084825478 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0c459a94-07c5-44fb-b155-8a8f7aa46d25-registry-tls") pod "image-registry-6cbff68f97-mt7bn" (UID: "0c459a94-07c5-44fb-b155-8a8f7aa46d25") : secret "image-registry-tls" not found
Sep 10 19:16:20.584575 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:20.584536    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/28f7bb38-4380-48b2-8881-38f1b188b5c8-metrics-tls\") pod \"dns-default-67nsx\" (UID: \"28f7bb38-4380-48b2-8881-38f1b188b5c8\") " pod="openshift-dns/dns-default-67nsx"
Sep 10 19:16:20.584759 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:20.584724    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 19:16:20.584828 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:20.584810    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28f7bb38-4380-48b2-8881-38f1b188b5c8-metrics-tls podName:28f7bb38-4380-48b2-8881-38f1b188b5c8 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:21.584788816 +0000 UTC m=+37.185696056 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/28f7bb38-4380-48b2-8881-38f1b188b5c8-metrics-tls") pod "dns-default-67nsx" (UID: "28f7bb38-4380-48b2-8881-38f1b188b5c8") : secret "dns-default-metrics-tls" not found
Sep 10 19:16:21.263315 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:21.263273    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:21.365060 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:21.365025    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:21.492587 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:21.492557    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0c459a94-07c5-44fb-b155-8a8f7aa46d25-registry-tls\") pod \"image-registry-6cbff68f97-mt7bn\" (UID: \"0c459a94-07c5-44fb-b155-8a8f7aa46d25\") " pod="openshift-image-registry/image-registry-6cbff68f97-mt7bn"
Sep 10 19:16:21.492779 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:21.492711    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 19:16:21.492779 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:21.492724    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cbff68f97-mt7bn: secret "image-registry-tls" not found
Sep 10 19:16:21.492891 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:21.492782    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c459a94-07c5-44fb-b155-8a8f7aa46d25-registry-tls podName:0c459a94-07c5-44fb-b155-8a8f7aa46d25 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:23.492767574 +0000 UTC m=+39.093674792 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0c459a94-07c5-44fb-b155-8a8f7aa46d25-registry-tls") pod "image-registry-6cbff68f97-mt7bn" (UID: "0c459a94-07c5-44fb-b155-8a8f7aa46d25") : secret "image-registry-tls" not found
Sep 10 19:16:21.592850 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:21.592828    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/28f7bb38-4380-48b2-8881-38f1b188b5c8-metrics-tls\") pod \"dns-default-67nsx\" (UID: \"28f7bb38-4380-48b2-8881-38f1b188b5c8\") " pod="openshift-dns/dns-default-67nsx"
Sep 10 19:16:21.593006 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:21.592987    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 19:16:21.593073 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:21.593062    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28f7bb38-4380-48b2-8881-38f1b188b5c8-metrics-tls podName:28f7bb38-4380-48b2-8881-38f1b188b5c8 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:23.593041174 +0000 UTC m=+39.193948417 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/28f7bb38-4380-48b2-8881-38f1b188b5c8-metrics-tls") pod "dns-default-67nsx" (UID: "28f7bb38-4380-48b2-8881-38f1b188b5c8") : secret "dns-default-metrics-tls" not found
Sep 10 19:16:21.632346 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:21.632312    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tc2c7"]
Sep 10 19:16:21.636337 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:16:21.636296    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90e5b1e2_b5f3_4306_8e86_363010cceb60.slice/crio-53de107e88bd63cbfbd7ea16f305272ee810e23f79e9af7c1e85eae15fdbc670 WatchSource:0}: Error finding container 53de107e88bd63cbfbd7ea16f305272ee810e23f79e9af7c1e85eae15fdbc670: Status 404 returned error can't find the container with id 53de107e88bd63cbfbd7ea16f305272ee810e23f79e9af7c1e85eae15fdbc670
Sep 10 19:16:22.121263 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:22.121233    2564 generic.go:356] "Generic (PLEG): container finished" podID="ac0ef973-45eb-4f42-8c5d-d86695e32507" containerID="78f044b08348e21c6654199abdcadeaed04c55c0b7dc695f0e39b0267eba070e" exitCode=0
Sep 10 19:16:22.121436 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:22.121307    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mzwh9" event={"ID":"ac0ef973-45eb-4f42-8c5d-d86695e32507","Type":"ContainerDied","Data":"78f044b08348e21c6654199abdcadeaed04c55c0b7dc695f0e39b0267eba070e"}
Sep 10 19:16:22.122460 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:22.122441    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tc2c7" event={"ID":"90e5b1e2-b5f3-4306-8e86-363010cceb60","Type":"ContainerStarted","Data":"53de107e88bd63cbfbd7ea16f305272ee810e23f79e9af7c1e85eae15fdbc670"}
Sep 10 19:16:23.128384 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:23.128345    2564 generic.go:356] "Generic (PLEG): container finished" podID="ac0ef973-45eb-4f42-8c5d-d86695e32507" containerID="828475ade7bea1f43757aee6694c8e8cd54b713bbc959c12e2595a50ba6cd0c5" exitCode=0
Sep 10 19:16:23.129272 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:23.128418    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mzwh9" event={"ID":"ac0ef973-45eb-4f42-8c5d-d86695e32507","Type":"ContainerDied","Data":"828475ade7bea1f43757aee6694c8e8cd54b713bbc959c12e2595a50ba6cd0c5"}
Sep 10 19:16:23.506583 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:23.506486    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0c459a94-07c5-44fb-b155-8a8f7aa46d25-registry-tls\") pod \"image-registry-6cbff68f97-mt7bn\" (UID: \"0c459a94-07c5-44fb-b155-8a8f7aa46d25\") " pod="openshift-image-registry/image-registry-6cbff68f97-mt7bn"
Sep 10 19:16:23.506736 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:23.506611    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 19:16:23.506736 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:23.506640    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cbff68f97-mt7bn: secret "image-registry-tls" not found
Sep 10 19:16:23.506736 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:23.506694    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c459a94-07c5-44fb-b155-8a8f7aa46d25-registry-tls podName:0c459a94-07c5-44fb-b155-8a8f7aa46d25 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:27.506677393 +0000 UTC m=+43.107584615 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0c459a94-07c5-44fb-b155-8a8f7aa46d25-registry-tls") pod "image-registry-6cbff68f97-mt7bn" (UID: "0c459a94-07c5-44fb-b155-8a8f7aa46d25") : secret "image-registry-tls" not found
Sep 10 19:16:23.607262 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:23.607230    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/28f7bb38-4380-48b2-8881-38f1b188b5c8-metrics-tls\") pod \"dns-default-67nsx\" (UID: \"28f7bb38-4380-48b2-8881-38f1b188b5c8\") " pod="openshift-dns/dns-default-67nsx"
Sep 10 19:16:23.607432 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:23.607362    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 19:16:23.607432 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:23.607423    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28f7bb38-4380-48b2-8881-38f1b188b5c8-metrics-tls podName:28f7bb38-4380-48b2-8881-38f1b188b5c8 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:27.607408148 +0000 UTC m=+43.208315366 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/28f7bb38-4380-48b2-8881-38f1b188b5c8-metrics-tls") pod "dns-default-67nsx" (UID: "28f7bb38-4380-48b2-8881-38f1b188b5c8") : secret "dns-default-metrics-tls" not found
Sep 10 19:16:24.056514 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:24.056253    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:24.135805 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:24.135726    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mzwh9" event={"ID":"ac0ef973-45eb-4f42-8c5d-d86695e32507","Type":"ContainerStarted","Data":"beb52bc3d60162121f98ca7a3ab260dc69fa8abd07db55e63e0e57b4306b69ec"}
Sep 10 19:16:24.160008 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:24.159970    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:24.160256 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:24.160209    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-mzwh9" podStartSLOduration=8.746732114 podStartE2EDuration="39.160190091s" totalImagesPullingTime="30.413457977s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:15:45 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:24.158892205 +0000 UTC m=+39.759799445" watchObservedRunningTime="2026-09-10 19:16:24.160190091 +0000 UTC m=+39.761097332"
Sep 10 19:16:24.356540 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:24.356500    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:24.410351 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:24.410269    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:25.139582 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:25.139492    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tc2c7" event={"ID":"90e5b1e2-b5f3-4306-8e86-363010cceb60","Type":"ContainerStarted","Data":"28290415ff12c6d16ec0173470c9709106741ea62aba1608e07ca254999a2fd8"}
Sep 10 19:16:25.157012 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:25.156960    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-tc2c7" podStartSLOduration=3.079998985 podStartE2EDuration="6.156944086s" totalImagesPullingTime="3.076945101s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:16:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:25.155785718 +0000 UTC m=+40.756692958" watchObservedRunningTime="2026-09-10 19:16:25.156944086 +0000 UTC m=+40.757851316"
Sep 10 19:16:25.201907 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:25.201880    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:25.262285 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:25.262249    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:27.044985 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:27.044946    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:27.107530 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:27.107483    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:27.207452 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:27.207402    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:27.521912 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:27.521808    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:27.535819 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:27.535789    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0c459a94-07c5-44fb-b155-8a8f7aa46d25-registry-tls\") pod \"image-registry-6cbff68f97-mt7bn\" (UID: \"0c459a94-07c5-44fb-b155-8a8f7aa46d25\") " pod="openshift-image-registry/image-registry-6cbff68f97-mt7bn"
Sep 10 19:16:27.535965 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:27.535949    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 19:16:27.536006 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:27.535969    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cbff68f97-mt7bn: secret "image-registry-tls" not found
Sep 10 19:16:27.536043 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:27.536023    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c459a94-07c5-44fb-b155-8a8f7aa46d25-registry-tls podName:0c459a94-07c5-44fb-b155-8a8f7aa46d25 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:35.536008855 +0000 UTC m=+51.136916072 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0c459a94-07c5-44fb-b155-8a8f7aa46d25-registry-tls") pod "image-registry-6cbff68f97-mt7bn" (UID: "0c459a94-07c5-44fb-b155-8a8f7aa46d25") : secret "image-registry-tls" not found
Sep 10 19:16:27.611404 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:27.611371    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:27.624958 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:27.624932    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-5v4m7"]
Sep 10 19:16:27.637102 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:27.637072    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/28f7bb38-4380-48b2-8881-38f1b188b5c8-metrics-tls\") pod \"dns-default-67nsx\" (UID: \"28f7bb38-4380-48b2-8881-38f1b188b5c8\") " pod="openshift-dns/dns-default-67nsx"
Sep 10 19:16:27.637235 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:27.637224    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 19:16:27.637288 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:27.637278    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28f7bb38-4380-48b2-8881-38f1b188b5c8-metrics-tls podName:28f7bb38-4380-48b2-8881-38f1b188b5c8 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:35.637263927 +0000 UTC m=+51.238171145 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/28f7bb38-4380-48b2-8881-38f1b188b5c8-metrics-tls") pod "dns-default-67nsx" (UID: "28f7bb38-4380-48b2-8881-38f1b188b5c8") : secret "dns-default-metrics-tls" not found
Sep 10 19:16:27.656558 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:27.656533    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5v4m7"]
Sep 10 19:16:27.656709 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:27.656656    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5v4m7"
Sep 10 19:16:27.658777 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:27.658756    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 10 19:16:27.658890 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:27.658793    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 10 19:16:27.658890 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:27.658837    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 10 19:16:27.659000 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:27.658903    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-mhsqp\""
Sep 10 19:16:27.659107 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:27.659092    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 10 19:16:27.678662 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:27.678632    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:27.803174 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:27.803092    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:27.838078 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:27.838045    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a66669df-07a2-4835-ac05-17a37bcfda7c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5v4m7\" (UID: \"a66669df-07a2-4835-ac05-17a37bcfda7c\") " pod="openshift-insights/insights-runtime-extractor-5v4m7"
Sep 10 19:16:27.838176 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:27.838085    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a66669df-07a2-4835-ac05-17a37bcfda7c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5v4m7\" (UID: \"a66669df-07a2-4835-ac05-17a37bcfda7c\") " pod="openshift-insights/insights-runtime-extractor-5v4m7"
Sep 10 19:16:27.838176 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:27.838124    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a66669df-07a2-4835-ac05-17a37bcfda7c-data-volume\") pod \"insights-runtime-extractor-5v4m7\" (UID: \"a66669df-07a2-4835-ac05-17a37bcfda7c\") " pod="openshift-insights/insights-runtime-extractor-5v4m7"
Sep 10 19:16:27.838270 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:27.838191    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a66669df-07a2-4835-ac05-17a37bcfda7c-crio-socket\") pod \"insights-runtime-extractor-5v4m7\" (UID: \"a66669df-07a2-4835-ac05-17a37bcfda7c\") " pod="openshift-insights/insights-runtime-extractor-5v4m7"
Sep 10 19:16:27.838270 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:27.838224    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzlws\" (UniqueName: \"kubernetes.io/projected/a66669df-07a2-4835-ac05-17a37bcfda7c-kube-api-access-jzlws\") pod \"insights-runtime-extractor-5v4m7\" (UID: \"a66669df-07a2-4835-ac05-17a37bcfda7c\") " pod="openshift-insights/insights-runtime-extractor-5v4m7"
Sep 10 19:16:27.938912 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:27.938878    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a66669df-07a2-4835-ac05-17a37bcfda7c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5v4m7\" (UID: \"a66669df-07a2-4835-ac05-17a37bcfda7c\") " pod="openshift-insights/insights-runtime-extractor-5v4m7"
Sep 10 19:16:27.938983 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:27.938919    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a66669df-07a2-4835-ac05-17a37bcfda7c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5v4m7\" (UID: \"a66669df-07a2-4835-ac05-17a37bcfda7c\") " pod="openshift-insights/insights-runtime-extractor-5v4m7"
Sep 10 19:16:27.939014 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:27.938978    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a66669df-07a2-4835-ac05-17a37bcfda7c-data-volume\") pod \"insights-runtime-extractor-5v4m7\" (UID: \"a66669df-07a2-4835-ac05-17a37bcfda7c\") " pod="openshift-insights/insights-runtime-extractor-5v4m7"
Sep 10 19:16:27.939014 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:27.939003    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 19:16:27.939014 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:27.939007    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a66669df-07a2-4835-ac05-17a37bcfda7c-crio-socket\") pod \"insights-runtime-extractor-5v4m7\" (UID: \"a66669df-07a2-4835-ac05-17a37bcfda7c\") " pod="openshift-insights/insights-runtime-extractor-5v4m7"
Sep 10 19:16:27.939113 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:27.939060    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzlws\" (UniqueName: \"kubernetes.io/projected/a66669df-07a2-4835-ac05-17a37bcfda7c-kube-api-access-jzlws\") pod \"insights-runtime-extractor-5v4m7\" (UID: \"a66669df-07a2-4835-ac05-17a37bcfda7c\") " pod="openshift-insights/insights-runtime-extractor-5v4m7"
Sep 10 19:16:27.939113 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:27.939078    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a66669df-07a2-4835-ac05-17a37bcfda7c-kube-rbac-proxy-cm podName:a66669df-07a2-4835-ac05-17a37bcfda7c nodeName:}" failed. No retries permitted until 2026-09-10 19:16:28.439058392 +0000 UTC m=+44.039965633 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a66669df-07a2-4835-ac05-17a37bcfda7c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5v4m7" (UID: "a66669df-07a2-4835-ac05-17a37bcfda7c") : configmap "kube-rbac-proxy" not found
Sep 10 19:16:27.939113 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:27.939089    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 19:16:27.939224 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:27.939152    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a66669df-07a2-4835-ac05-17a37bcfda7c-insights-runtime-extractor-tls podName:a66669df-07a2-4835-ac05-17a37bcfda7c nodeName:}" failed. No retries permitted until 2026-09-10 19:16:28.439134063 +0000 UTC m=+44.040041284 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a66669df-07a2-4835-ac05-17a37bcfda7c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5v4m7" (UID: "a66669df-07a2-4835-ac05-17a37bcfda7c") : secret "insights-runtime-extractor-tls" not found
Sep 10 19:16:27.939265 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:27.939239    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a66669df-07a2-4835-ac05-17a37bcfda7c-crio-socket\") pod \"insights-runtime-extractor-5v4m7\" (UID: \"a66669df-07a2-4835-ac05-17a37bcfda7c\") " pod="openshift-insights/insights-runtime-extractor-5v4m7"
Sep 10 19:16:27.939386 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:27.939366    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a66669df-07a2-4835-ac05-17a37bcfda7c-data-volume\") pod \"insights-runtime-extractor-5v4m7\" (UID: \"a66669df-07a2-4835-ac05-17a37bcfda7c\") " pod="openshift-insights/insights-runtime-extractor-5v4m7"
Sep 10 19:16:27.950198 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:27.950172    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzlws\" (UniqueName: \"kubernetes.io/projected/a66669df-07a2-4835-ac05-17a37bcfda7c-kube-api-access-jzlws\") pod \"insights-runtime-extractor-5v4m7\" (UID: \"a66669df-07a2-4835-ac05-17a37bcfda7c\") " pod="openshift-insights/insights-runtime-extractor-5v4m7"
Sep 10 19:16:27.994428 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:27.994394    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:28.093954 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:28.093872    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:28.443720 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:28.443608    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a66669df-07a2-4835-ac05-17a37bcfda7c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5v4m7\" (UID: \"a66669df-07a2-4835-ac05-17a37bcfda7c\") " pod="openshift-insights/insights-runtime-extractor-5v4m7"
Sep 10 19:16:28.443720 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:28.443666    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a66669df-07a2-4835-ac05-17a37bcfda7c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5v4m7\" (UID: \"a66669df-07a2-4835-ac05-17a37bcfda7c\") " pod="openshift-insights/insights-runtime-extractor-5v4m7"
Sep 10 19:16:28.443946 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:28.443739    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 19:16:28.443946 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:28.443815    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a66669df-07a2-4835-ac05-17a37bcfda7c-kube-rbac-proxy-cm podName:a66669df-07a2-4835-ac05-17a37bcfda7c nodeName:}" failed. No retries permitted until 2026-09-10 19:16:29.443794392 +0000 UTC m=+45.044701611 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a66669df-07a2-4835-ac05-17a37bcfda7c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5v4m7" (UID: "a66669df-07a2-4835-ac05-17a37bcfda7c") : configmap "kube-rbac-proxy" not found
Sep 10 19:16:28.443946 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:28.443823    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 19:16:28.443946 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:28.443880    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a66669df-07a2-4835-ac05-17a37bcfda7c-insights-runtime-extractor-tls podName:a66669df-07a2-4835-ac05-17a37bcfda7c nodeName:}" failed. No retries permitted until 2026-09-10 19:16:29.443868544 +0000 UTC m=+45.044775762 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a66669df-07a2-4835-ac05-17a37bcfda7c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5v4m7" (UID: "a66669df-07a2-4835-ac05-17a37bcfda7c") : secret "insights-runtime-extractor-tls" not found
Sep 10 19:16:28.561418 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:28.561382    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:28.616782 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:28.616745    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:28.692860 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:28.692822    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:28.793556 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:28.793481    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:29.451675 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:29.451612    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a66669df-07a2-4835-ac05-17a37bcfda7c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5v4m7\" (UID: \"a66669df-07a2-4835-ac05-17a37bcfda7c\") " pod="openshift-insights/insights-runtime-extractor-5v4m7"
Sep 10 19:16:29.452058 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:29.451741    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a66669df-07a2-4835-ac05-17a37bcfda7c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5v4m7\" (UID: \"a66669df-07a2-4835-ac05-17a37bcfda7c\") " pod="openshift-insights/insights-runtime-extractor-5v4m7"
Sep 10 19:16:29.452058 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:29.451765    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 19:16:29.452058 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:29.451807    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 19:16:29.452058 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:29.451831    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a66669df-07a2-4835-ac05-17a37bcfda7c-insights-runtime-extractor-tls podName:a66669df-07a2-4835-ac05-17a37bcfda7c nodeName:}" failed. No retries permitted until 2026-09-10 19:16:31.451815787 +0000 UTC m=+47.052723005 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a66669df-07a2-4835-ac05-17a37bcfda7c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5v4m7" (UID: "a66669df-07a2-4835-ac05-17a37bcfda7c") : secret "insights-runtime-extractor-tls" not found
Sep 10 19:16:29.452058 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:29.451848    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a66669df-07a2-4835-ac05-17a37bcfda7c-kube-rbac-proxy-cm podName:a66669df-07a2-4835-ac05-17a37bcfda7c nodeName:}" failed. No retries permitted until 2026-09-10 19:16:31.451841375 +0000 UTC m=+47.052748593 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a66669df-07a2-4835-ac05-17a37bcfda7c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5v4m7" (UID: "a66669df-07a2-4835-ac05-17a37bcfda7c") : configmap "kube-rbac-proxy" not found
Sep 10 19:16:29.779027 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:29.779004    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-l8b8c_e08684c6-1cc0-4f4c-a4cd-1fea1e46339a/dns-node-resolver/0.log"
Sep 10 19:16:30.336935 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:30.336895    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:30.388501 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:30.388469    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:30.586861 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:30.586836    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7s7hj_ae8c5b13-1925-4df6-b3a2-d4ace9a51801/node-ca/0.log"
Sep 10 19:16:31.244262 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:31.244230    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:31.379466 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:31.379437    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tc2c7_90e5b1e2-b5f3-4306-8e86-363010cceb60/serve-healthcheck-canary/0.log"
Sep 10 19:16:31.466713 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:31.466676    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a66669df-07a2-4835-ac05-17a37bcfda7c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5v4m7\" (UID: \"a66669df-07a2-4835-ac05-17a37bcfda7c\") " pod="openshift-insights/insights-runtime-extractor-5v4m7"
Sep 10 19:16:31.466866 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:31.466720    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a66669df-07a2-4835-ac05-17a37bcfda7c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5v4m7\" (UID: \"a66669df-07a2-4835-ac05-17a37bcfda7c\") " pod="openshift-insights/insights-runtime-extractor-5v4m7"
Sep 10 19:16:31.466866 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:31.466801    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 19:16:31.466866 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:31.466862    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a66669df-07a2-4835-ac05-17a37bcfda7c-kube-rbac-proxy-cm podName:a66669df-07a2-4835-ac05-17a37bcfda7c nodeName:}" failed. No retries permitted until 2026-09-10 19:16:35.466846895 +0000 UTC m=+51.067754113 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a66669df-07a2-4835-ac05-17a37bcfda7c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5v4m7" (UID: "a66669df-07a2-4835-ac05-17a37bcfda7c") : configmap "kube-rbac-proxy" not found
Sep 10 19:16:31.467011 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:31.466875    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 19:16:31.467011 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:31.466935    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a66669df-07a2-4835-ac05-17a37bcfda7c-insights-runtime-extractor-tls podName:a66669df-07a2-4835-ac05-17a37bcfda7c nodeName:}" failed. No retries permitted until 2026-09-10 19:16:35.466919407 +0000 UTC m=+51.067826638 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a66669df-07a2-4835-ac05-17a37bcfda7c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5v4m7" (UID: "a66669df-07a2-4835-ac05-17a37bcfda7c") : secret "insights-runtime-extractor-tls" not found
Sep 10 19:16:32.563719 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:32.563690    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:33.334034 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:33.333992    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:35.005160 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:35.005121    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:35.105870 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:35.105823    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:35.210983 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:35.210946    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:35.305983 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:35.305904    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:35.405935 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:35.405898    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:35.496142 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:35.496103    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a66669df-07a2-4835-ac05-17a37bcfda7c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5v4m7\" (UID: \"a66669df-07a2-4835-ac05-17a37bcfda7c\") " pod="openshift-insights/insights-runtime-extractor-5v4m7"
Sep 10 19:16:35.496320 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:35.496148    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a66669df-07a2-4835-ac05-17a37bcfda7c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5v4m7\" (UID: \"a66669df-07a2-4835-ac05-17a37bcfda7c\") " pod="openshift-insights/insights-runtime-extractor-5v4m7"
Sep 10 19:16:35.496320 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:35.496301    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 19:16:35.496427 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:35.496384    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a66669df-07a2-4835-ac05-17a37bcfda7c-insights-runtime-extractor-tls podName:a66669df-07a2-4835-ac05-17a37bcfda7c nodeName:}" failed. No retries permitted until 2026-09-10 19:16:43.496362592 +0000 UTC m=+59.097269824 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a66669df-07a2-4835-ac05-17a37bcfda7c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5v4m7" (UID: "a66669df-07a2-4835-ac05-17a37bcfda7c") : secret "insights-runtime-extractor-tls" not found
Sep 10 19:16:35.496742 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:35.496718    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a66669df-07a2-4835-ac05-17a37bcfda7c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5v4m7\" (UID: \"a66669df-07a2-4835-ac05-17a37bcfda7c\") " pod="openshift-insights/insights-runtime-extractor-5v4m7"
Sep 10 19:16:35.596969 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:35.596872    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0c459a94-07c5-44fb-b155-8a8f7aa46d25-registry-tls\") pod \"image-registry-6cbff68f97-mt7bn\" (UID: \"0c459a94-07c5-44fb-b155-8a8f7aa46d25\") " pod="openshift-image-registry/image-registry-6cbff68f97-mt7bn"
Sep 10 19:16:35.597112 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:35.597028    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 19:16:35.597112 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:35.597047    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cbff68f97-mt7bn: secret "image-registry-tls" not found
Sep 10 19:16:35.597112 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:35.597101    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c459a94-07c5-44fb-b155-8a8f7aa46d25-registry-tls podName:0c459a94-07c5-44fb-b155-8a8f7aa46d25 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:51.597085386 +0000 UTC m=+67.197992604 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0c459a94-07c5-44fb-b155-8a8f7aa46d25-registry-tls") pod "image-registry-6cbff68f97-mt7bn" (UID: "0c459a94-07c5-44fb-b155-8a8f7aa46d25") : secret "image-registry-tls" not found
Sep 10 19:16:35.697396 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:35.697360    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/28f7bb38-4380-48b2-8881-38f1b188b5c8-metrics-tls\") pod \"dns-default-67nsx\" (UID: \"28f7bb38-4380-48b2-8881-38f1b188b5c8\") " pod="openshift-dns/dns-default-67nsx"
Sep 10 19:16:35.697528 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:35.697474    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 19:16:35.697571 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:16:35.697537    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28f7bb38-4380-48b2-8881-38f1b188b5c8-metrics-tls podName:28f7bb38-4380-48b2-8881-38f1b188b5c8 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:51.697520639 +0000 UTC m=+67.298427859 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/28f7bb38-4380-48b2-8881-38f1b188b5c8-metrics-tls") pod "dns-default-67nsx" (UID: "28f7bb38-4380-48b2-8881-38f1b188b5c8") : secret "dns-default-metrics-tls" not found
Sep 10 19:16:36.397674 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:36.397614    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:36.790192 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:36.790157    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:37.002130 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:37.002095    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:37.202909 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:37.202823    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:37.385863 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:37.385830    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:37.987872 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:37.987835    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:38.230605 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:38.230568    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:38.586281 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:38.586248    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:38.790297 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:38.790268    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:38.987474 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:38.987389    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:39.089902 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:39.089866    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:39.388618 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:39.388578    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:39.603532 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:39.603501    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:39.666398 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:39.666309    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:40.196347 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:40.196313    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:40.587039 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:40.587007    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:40.689683 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:40.689643    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:40.748953 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:40.748916    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:40.805846 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:40.805813    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:41.452593 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:41.452550    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:41.552829 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:41.552797    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:41.787537 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:41.787498    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:42.124982 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:42.124894    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:42.589770 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:42.589723    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:42.788592 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:42.788562    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:43.394561 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:43.394527    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:43.553202 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:43.553148    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a66669df-07a2-4835-ac05-17a37bcfda7c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5v4m7\" (UID: \"a66669df-07a2-4835-ac05-17a37bcfda7c\") " pod="openshift-insights/insights-runtime-extractor-5v4m7"
Sep 10 19:16:43.555446 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:43.555412    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a66669df-07a2-4835-ac05-17a37bcfda7c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5v4m7\" (UID: \"a66669df-07a2-4835-ac05-17a37bcfda7c\") " pod="openshift-insights/insights-runtime-extractor-5v4m7"
Sep 10 19:16:43.566409 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:43.566387    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5v4m7"
Sep 10 19:16:43.837757 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:43.837726    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5v4m7"]
Sep 10 19:16:43.841319 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:16:43.841283    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda66669df_07a2_4835_ac05_17a37bcfda7c.slice/crio-2fe32cb354d02a85b6bdff632e69d31a60fcc0a3bff4dd3cd52b2e1e7e6745f9 WatchSource:0}: Error finding container 2fe32cb354d02a85b6bdff632e69d31a60fcc0a3bff4dd3cd52b2e1e7e6745f9: Status 404 returned error can't find the container with id 2fe32cb354d02a85b6bdff632e69d31a60fcc0a3bff4dd3cd52b2e1e7e6745f9
Sep 10 19:16:44.023808 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:44.023765    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:44.184026 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:44.183944    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5v4m7" event={"ID":"a66669df-07a2-4835-ac05-17a37bcfda7c","Type":"ContainerStarted","Data":"1baa50e225e0efe6c00f90e02621e55c02828d1831026927b6a22332f4974b13"}
Sep 10 19:16:44.184026 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:44.184003    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5v4m7" event={"ID":"a66669df-07a2-4835-ac05-17a37bcfda7c","Type":"ContainerStarted","Data":"2fe32cb354d02a85b6bdff632e69d31a60fcc0a3bff4dd3cd52b2e1e7e6745f9"}
Sep 10 19:16:44.187432 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:44.187406    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:44.389450 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:44.389410    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:44.787389 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:44.787355    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:45.188479 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:45.188441    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5v4m7" event={"ID":"a66669df-07a2-4835-ac05-17a37bcfda7c","Type":"ContainerStarted","Data":"464606e4500316a0bff9aab62b9896620a6c3f2213d10c00961c6a575c3113e9"}
Sep 10 19:16:45.292409 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:45.292378    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:45.590007 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:45.589972    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:45.800237 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:45.800190    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:46.118151 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:46.118119    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-82crq"
Sep 10 19:16:47.194976 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:47.194941    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5v4m7" event={"ID":"a66669df-07a2-4835-ac05-17a37bcfda7c","Type":"ContainerStarted","Data":"c560a82ed890053c7de6add11e6b6c533b415f0eb227258fd57f9e15e44d7f34"}
Sep 10 19:16:47.224303 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:47.224240    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-5v4m7" podStartSLOduration=17.586772194 podStartE2EDuration="20.224224242s" totalImagesPullingTime="2.637452048s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:16:27 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:47.223253139 +0000 UTC m=+62.824160379" watchObservedRunningTime="2026-09-10 19:16:47.224224242 +0000 UTC m=+62.825131483"
Sep 10 19:16:50.004663 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:50.004615    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:50.062450 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:50.062418    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:50.606214 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:50.606162    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/80a423f3-63d3-42fa-a233-166ab675777e-original-pull-secret\") pod \"global-pull-secret-syncer-vcs72\" (UID: \"80a423f3-63d3-42fa-a233-166ab675777e\") " pod="kube-system/global-pull-secret-syncer-vcs72"
Sep 10 19:16:50.606385 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:50.606247    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94b7fd86-07b1-4d2c-869f-787a7fcc6dca-metrics-certs\") pod \"network-metrics-daemon-tvlcw\" (UID: \"94b7fd86-07b1-4d2c-869f-787a7fcc6dca\") " pod="openshift-multus/network-metrics-daemon-tvlcw"
Sep 10 19:16:50.608980 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:50.608964    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 19:16:50.609023 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:50.608968    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 19:16:50.619314 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:50.619280    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/80a423f3-63d3-42fa-a233-166ab675777e-original-pull-secret\") pod \"global-pull-secret-syncer-vcs72\" (UID: \"80a423f3-63d3-42fa-a233-166ab675777e\") " pod="kube-system/global-pull-secret-syncer-vcs72"
Sep 10 19:16:50.619314 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:50.619307    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94b7fd86-07b1-4d2c-869f-787a7fcc6dca-metrics-certs\") pod \"network-metrics-daemon-tvlcw\" (UID: \"94b7fd86-07b1-4d2c-869f-787a7fcc6dca\") " pod="openshift-multus/network-metrics-daemon-tvlcw"
Sep 10 19:16:50.706804 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:50.706756    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqrjw\" (UniqueName: \"kubernetes.io/projected/24f84abd-cd7a-40fa-b23f-72e60cc247b8-kube-api-access-qqrjw\") pod \"network-check-target-9stjj\" (UID: \"24f84abd-cd7a-40fa-b23f-72e60cc247b8\") " pod="openshift-network-diagnostics/network-check-target-9stjj"
Sep 10 19:16:50.709479 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:50.709461    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 10 19:16:50.719745 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:50.719726    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 10 19:16:50.729827 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:50.729808    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqrjw\" (UniqueName: \"kubernetes.io/projected/24f84abd-cd7a-40fa-b23f-72e60cc247b8-kube-api-access-qqrjw\") pod \"network-check-target-9stjj\" (UID: \"24f84abd-cd7a-40fa-b23f-72e60cc247b8\") " pod="openshift-network-diagnostics/network-check-target-9stjj"
Sep 10 19:16:50.858529 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:50.858440    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-8fmlh\""
Sep 10 19:16:50.861822 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:50.861803    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcs72"
Sep 10 19:16:50.865860 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:50.865831    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9stjj"
Sep 10 19:16:50.870785 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:50.870765    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gqr5m\""
Sep 10 19:16:50.879031 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:50.879012    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvlcw"
Sep 10 19:16:51.141192 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:51.141114    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vcs72"]
Sep 10 19:16:51.144169 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:51.144140    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9stjj"]
Sep 10 19:16:51.145446 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:16:51.145423    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80a423f3_63d3_42fa_a233_166ab675777e.slice/crio-6ab08daca00833222cfbaa9cbf30467431f6ebe3be5f1cc369f444baf66dcf89 WatchSource:0}: Error finding container 6ab08daca00833222cfbaa9cbf30467431f6ebe3be5f1cc369f444baf66dcf89: Status 404 returned error can't find the container with id 6ab08daca00833222cfbaa9cbf30467431f6ebe3be5f1cc369f444baf66dcf89
Sep 10 19:16:51.147132 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:16:51.147112    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24f84abd_cd7a_40fa_b23f_72e60cc247b8.slice/crio-3073e2d64b370a55898f98d58c402fd0c34e7376f20e4a13d6696f29d98fb4dc WatchSource:0}: Error finding container 3073e2d64b370a55898f98d58c402fd0c34e7376f20e4a13d6696f29d98fb4dc: Status 404 returned error can't find the container with id 3073e2d64b370a55898f98d58c402fd0c34e7376f20e4a13d6696f29d98fb4dc
Sep 10 19:16:51.158669 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:51.158651    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-tvlcw"]
Sep 10 19:16:51.160682 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:16:51.160663    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94b7fd86_07b1_4d2c_869f_787a7fcc6dca.slice/crio-506b9527a675f0378daace277343d6ca3cd76b7bf76dcd8fcff4698b26ab0dac WatchSource:0}: Error finding container 506b9527a675f0378daace277343d6ca3cd76b7bf76dcd8fcff4698b26ab0dac: Status 404 returned error can't find the container with id 506b9527a675f0378daace277343d6ca3cd76b7bf76dcd8fcff4698b26ab0dac
Sep 10 19:16:51.205088 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:51.205064    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9stjj" event={"ID":"24f84abd-cd7a-40fa-b23f-72e60cc247b8","Type":"ContainerStarted","Data":"3073e2d64b370a55898f98d58c402fd0c34e7376f20e4a13d6696f29d98fb4dc"}
Sep 10 19:16:51.206265 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:51.206244    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-tvlcw" event={"ID":"94b7fd86-07b1-4d2c-869f-787a7fcc6dca","Type":"ContainerStarted","Data":"506b9527a675f0378daace277343d6ca3cd76b7bf76dcd8fcff4698b26ab0dac"}
Sep 10 19:16:51.207358 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:51.207337    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vcs72" event={"ID":"80a423f3-63d3-42fa-a233-166ab675777e","Type":"ContainerStarted","Data":"6ab08daca00833222cfbaa9cbf30467431f6ebe3be5f1cc369f444baf66dcf89"}
Sep 10 19:16:51.273512 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:51.273481    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:51.373002 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:51.372971    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:51.429348 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:51.429271    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:51.570676 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:51.570641    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:51.614500 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:51.614460    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0c459a94-07c5-44fb-b155-8a8f7aa46d25-registry-tls\") pod \"image-registry-6cbff68f97-mt7bn\" (UID: \"0c459a94-07c5-44fb-b155-8a8f7aa46d25\") " pod="openshift-image-registry/image-registry-6cbff68f97-mt7bn"
Sep 10 19:16:51.617254 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:51.617224    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0c459a94-07c5-44fb-b155-8a8f7aa46d25-registry-tls\") pod \"image-registry-6cbff68f97-mt7bn\" (UID: \"0c459a94-07c5-44fb-b155-8a8f7aa46d25\") " pod="openshift-image-registry/image-registry-6cbff68f97-mt7bn"
Sep 10 19:16:51.715822 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:51.715735    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/28f7bb38-4380-48b2-8881-38f1b188b5c8-metrics-tls\") pod \"dns-default-67nsx\" (UID: \"28f7bb38-4380-48b2-8881-38f1b188b5c8\") " pod="openshift-dns/dns-default-67nsx"
Sep 10 19:16:51.718404 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:51.718377    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/28f7bb38-4380-48b2-8881-38f1b188b5c8-metrics-tls\") pod \"dns-default-67nsx\" (UID: \"28f7bb38-4380-48b2-8881-38f1b188b5c8\") " pod="openshift-dns/dns-default-67nsx"
Sep 10 19:16:51.754833 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:51.754800    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:51.855505 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:51.855465    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:51.895021 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:51.894989    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-qxq2w\""
Sep 10 19:16:51.903010 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:51.902983    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cbff68f97-mt7bn"
Sep 10 19:16:51.931364 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:51.931336    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wkm4c\""
Sep 10 19:16:51.939518 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:51.939493    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-67nsx"
Sep 10 19:16:51.955382 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:51.955218    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:52.031361 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:52.031282    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:52.199743 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:52.199670    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6cbff68f97-mt7bn"]
Sep 10 19:16:52.216516 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:16:52.216483    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c459a94_07c5_44fb_b155_8a8f7aa46d25.slice/crio-7b5dbac6890443ab1aa2a1ee02d6c7080c1159a40fc56cfb0398389a32dcd727 WatchSource:0}: Error finding container 7b5dbac6890443ab1aa2a1ee02d6c7080c1159a40fc56cfb0398389a32dcd727: Status 404 returned error can't find the container with id 7b5dbac6890443ab1aa2a1ee02d6c7080c1159a40fc56cfb0398389a32dcd727
Sep 10 19:16:52.252918 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:52.252889    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-67nsx"]
Sep 10 19:16:52.256711 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:16:52.256640    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28f7bb38_4380_48b2_8881_38f1b188b5c8.slice/crio-c5ae109fdac4cee8f129ba38c5eb4da8958fa6c42bae7df7a771c7054970bfba WatchSource:0}: Error finding container c5ae109fdac4cee8f129ba38c5eb4da8958fa6c42bae7df7a771c7054970bfba: Status 404 returned error can't find the container with id c5ae109fdac4cee8f129ba38c5eb4da8958fa6c42bae7df7a771c7054970bfba
Sep 10 19:16:52.341840 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:52.341671    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:52.404089 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:52.403934    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:52.975872 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:52.975803    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:53.028447 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.027522    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7667ff4d6d-lpk2n"]
Sep 10 19:16:53.050234 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.050208    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7667ff4d6d-lpk2n"]
Sep 10 19:16:53.050370 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.050312    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7667ff4d6d-lpk2n"
Sep 10 19:16:53.056193 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.055410    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 10 19:16:53.056193 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.055661    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-x5nlb\""
Sep 10 19:16:53.056193 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.055842    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 10 19:16:53.056193 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.055875    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Sep 10 19:16:53.056193 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.056045    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 10 19:16:53.090392 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.090305    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6cbff68f97-mt7bn"]
Sep 10 19:16:53.116552 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.116483    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:53.126659 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.126613    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-85d98c998f-ssx24"]
Sep 10 19:16:53.127684 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.127443    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/eaca23fa-c738-4b43-bbcf-49c77a53ddfa-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7667ff4d6d-lpk2n\" (UID: \"eaca23fa-c738-4b43-bbcf-49c77a53ddfa\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7667ff4d6d-lpk2n"
Sep 10 19:16:53.127684 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.127510    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mlbjk\" (UniqueName: \"kubernetes.io/projected/eaca23fa-c738-4b43-bbcf-49c77a53ddfa-kube-api-access-mlbjk\") pod \"managed-serviceaccount-addon-agent-7667ff4d6d-lpk2n\" (UID: \"eaca23fa-c738-4b43-bbcf-49c77a53ddfa\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7667ff4d6d-lpk2n"
Sep 10 19:16:53.158443 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.158413    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-54895dc98-2h4rn"]
Sep 10 19:16:53.158567 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.158544    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-ssx24"
Sep 10 19:16:53.161001 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.160975    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 10 19:16:53.161650 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.161227    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-2n99h\""
Sep 10 19:16:53.161650 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.161476    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 10 19:16:53.172389 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.172368    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-ssx24"]
Sep 10 19:16:53.172500 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.172408    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-54895dc98-2h4rn"]
Sep 10 19:16:53.172562 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.172502    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-54895dc98-2h4rn"
Sep 10 19:16:53.192867 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.192742    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:53.220148 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.219671    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-tvlcw" event={"ID":"94b7fd86-07b1-4d2c-869f-787a7fcc6dca","Type":"ContainerStarted","Data":"841a1a7c34bc16f3b0982bad3822904e8e1eb7e09843e91d56afc7a539825292"}
Sep 10 19:16:53.221256 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.221226    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-67nsx" event={"ID":"28f7bb38-4380-48b2-8881-38f1b188b5c8","Type":"ContainerStarted","Data":"c5ae109fdac4cee8f129ba38c5eb4da8958fa6c42bae7df7a771c7054970bfba"}
Sep 10 19:16:53.223552 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.223527    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6cbff68f97-mt7bn" event={"ID":"0c459a94-07c5-44fb-b155-8a8f7aa46d25","Type":"ContainerStarted","Data":"a7d5ef2f6dc507b03bfe91d8d98d928a758cec069f034ac1fb5c5e1a40048944"}
Sep 10 19:16:53.223679 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.223570    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6cbff68f97-mt7bn" event={"ID":"0c459a94-07c5-44fb-b155-8a8f7aa46d25","Type":"ContainerStarted","Data":"7b5dbac6890443ab1aa2a1ee02d6c7080c1159a40fc56cfb0398389a32dcd727"}
Sep 10 19:16:53.224193 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.224173    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6cbff68f97-mt7bn"
Sep 10 19:16:53.228121 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.228098    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/815cb529-d322-4a64-a0c3-35de02c583ee-installation-pull-secrets\") pod \"image-registry-54895dc98-2h4rn\" (UID: \"815cb529-d322-4a64-a0c3-35de02c583ee\") " pod="openshift-image-registry/image-registry-54895dc98-2h4rn"
Sep 10 19:16:53.228207 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.228144    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mlbjk\" (UniqueName: \"kubernetes.io/projected/eaca23fa-c738-4b43-bbcf-49c77a53ddfa-kube-api-access-mlbjk\") pod \"managed-serviceaccount-addon-agent-7667ff4d6d-lpk2n\" (UID: \"eaca23fa-c738-4b43-bbcf-49c77a53ddfa\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7667ff4d6d-lpk2n"
Sep 10 19:16:53.228207 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.228198    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/eaca23fa-c738-4b43-bbcf-49c77a53ddfa-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7667ff4d6d-lpk2n\" (UID: \"eaca23fa-c738-4b43-bbcf-49c77a53ddfa\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7667ff4d6d-lpk2n"
Sep 10 19:16:53.228319 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.228232    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/815cb529-d322-4a64-a0c3-35de02c583ee-registry-tls\") pod \"image-registry-54895dc98-2h4rn\" (UID: \"815cb529-d322-4a64-a0c3-35de02c583ee\") " pod="openshift-image-registry/image-registry-54895dc98-2h4rn"
Sep 10 19:16:53.228319 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.228257    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/815cb529-d322-4a64-a0c3-35de02c583ee-trusted-ca\") pod \"image-registry-54895dc98-2h4rn\" (UID: \"815cb529-d322-4a64-a0c3-35de02c583ee\") " pod="openshift-image-registry/image-registry-54895dc98-2h4rn"
Sep 10 19:16:53.228319 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.228282    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/815cb529-d322-4a64-a0c3-35de02c583ee-image-registry-private-configuration\") pod \"image-registry-54895dc98-2h4rn\" (UID: \"815cb529-d322-4a64-a0c3-35de02c583ee\") " pod="openshift-image-registry/image-registry-54895dc98-2h4rn"
Sep 10 19:16:53.228319 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.228308    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6tvd\" (UniqueName: \"kubernetes.io/projected/815cb529-d322-4a64-a0c3-35de02c583ee-kube-api-access-g6tvd\") pod \"image-registry-54895dc98-2h4rn\" (UID: \"815cb529-d322-4a64-a0c3-35de02c583ee\") " pod="openshift-image-registry/image-registry-54895dc98-2h4rn"
Sep 10 19:16:53.228513 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.228333    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/815cb529-d322-4a64-a0c3-35de02c583ee-ca-trust-extracted\") pod \"image-registry-54895dc98-2h4rn\" (UID: \"815cb529-d322-4a64-a0c3-35de02c583ee\") " pod="openshift-image-registry/image-registry-54895dc98-2h4rn"
Sep 10 19:16:53.228513 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.228362    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/815cb529-d322-4a64-a0c3-35de02c583ee-bound-sa-token\") pod \"image-registry-54895dc98-2h4rn\" (UID: \"815cb529-d322-4a64-a0c3-35de02c583ee\") " pod="openshift-image-registry/image-registry-54895dc98-2h4rn"
Sep 10 19:16:53.228513 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.228388    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/815cb529-d322-4a64-a0c3-35de02c583ee-registry-certificates\") pod \"image-registry-54895dc98-2h4rn\" (UID: \"815cb529-d322-4a64-a0c3-35de02c583ee\") " pod="openshift-image-registry/image-registry-54895dc98-2h4rn"
Sep 10 19:16:53.228513 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.228417    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvmxz\" (UniqueName: \"kubernetes.io/projected/ec666f15-107e-4574-9d6c-cc73b6840c2e-kube-api-access-lvmxz\") pod \"downloads-85d98c998f-ssx24\" (UID: \"ec666f15-107e-4574-9d6c-cc73b6840c2e\") " pod="openshift-console/downloads-85d98c998f-ssx24"
Sep 10 19:16:53.232758 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.232719    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/eaca23fa-c738-4b43-bbcf-49c77a53ddfa-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7667ff4d6d-lpk2n\" (UID: \"eaca23fa-c738-4b43-bbcf-49c77a53ddfa\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7667ff4d6d-lpk2n"
Sep 10 19:16:53.244572 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.244548    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlbjk\" (UniqueName: \"kubernetes.io/projected/eaca23fa-c738-4b43-bbcf-49c77a53ddfa-kube-api-access-mlbjk\") pod \"managed-serviceaccount-addon-agent-7667ff4d6d-lpk2n\" (UID: \"eaca23fa-c738-4b43-bbcf-49c77a53ddfa\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7667ff4d6d-lpk2n"
Sep 10 19:16:53.268724 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.268666    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:53.330497 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.329455    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/815cb529-d322-4a64-a0c3-35de02c583ee-installation-pull-secrets\") pod \"image-registry-54895dc98-2h4rn\" (UID: \"815cb529-d322-4a64-a0c3-35de02c583ee\") " pod="openshift-image-registry/image-registry-54895dc98-2h4rn"
Sep 10 19:16:53.330497 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.329593    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/815cb529-d322-4a64-a0c3-35de02c583ee-registry-tls\") pod \"image-registry-54895dc98-2h4rn\" (UID: \"815cb529-d322-4a64-a0c3-35de02c583ee\") " pod="openshift-image-registry/image-registry-54895dc98-2h4rn"
Sep 10 19:16:53.330497 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.329633    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/815cb529-d322-4a64-a0c3-35de02c583ee-trusted-ca\") pod \"image-registry-54895dc98-2h4rn\" (UID: \"815cb529-d322-4a64-a0c3-35de02c583ee\") " pod="openshift-image-registry/image-registry-54895dc98-2h4rn"
Sep 10 19:16:53.330497 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.329658    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/815cb529-d322-4a64-a0c3-35de02c583ee-image-registry-private-configuration\") pod \"image-registry-54895dc98-2h4rn\" (UID: \"815cb529-d322-4a64-a0c3-35de02c583ee\") " pod="openshift-image-registry/image-registry-54895dc98-2h4rn"
Sep 10 19:16:53.330497 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.329691    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6tvd\" (UniqueName: \"kubernetes.io/projected/815cb529-d322-4a64-a0c3-35de02c583ee-kube-api-access-g6tvd\") pod \"image-registry-54895dc98-2h4rn\" (UID: \"815cb529-d322-4a64-a0c3-35de02c583ee\") " pod="openshift-image-registry/image-registry-54895dc98-2h4rn"
Sep 10 19:16:53.330497 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.329718    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/815cb529-d322-4a64-a0c3-35de02c583ee-ca-trust-extracted\") pod \"image-registry-54895dc98-2h4rn\" (UID: \"815cb529-d322-4a64-a0c3-35de02c583ee\") " pod="openshift-image-registry/image-registry-54895dc98-2h4rn"
Sep 10 19:16:53.330497 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.329760    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/815cb529-d322-4a64-a0c3-35de02c583ee-bound-sa-token\") pod \"image-registry-54895dc98-2h4rn\" (UID: \"815cb529-d322-4a64-a0c3-35de02c583ee\") " pod="openshift-image-registry/image-registry-54895dc98-2h4rn"
Sep 10 19:16:53.330497 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.329802    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/815cb529-d322-4a64-a0c3-35de02c583ee-registry-certificates\") pod \"image-registry-54895dc98-2h4rn\" (UID: \"815cb529-d322-4a64-a0c3-35de02c583ee\") " pod="openshift-image-registry/image-registry-54895dc98-2h4rn"
Sep 10 19:16:53.330497 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.329844    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvmxz\" (UniqueName: \"kubernetes.io/projected/ec666f15-107e-4574-9d6c-cc73b6840c2e-kube-api-access-lvmxz\") pod \"downloads-85d98c998f-ssx24\" (UID: \"ec666f15-107e-4574-9d6c-cc73b6840c2e\") " pod="openshift-console/downloads-85d98c998f-ssx24"
Sep 10 19:16:53.333445 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.332249    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/815cb529-d322-4a64-a0c3-35de02c583ee-trusted-ca\") pod \"image-registry-54895dc98-2h4rn\" (UID: \"815cb529-d322-4a64-a0c3-35de02c583ee\") " pod="openshift-image-registry/image-registry-54895dc98-2h4rn"
Sep 10 19:16:53.333445 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.333294    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/815cb529-d322-4a64-a0c3-35de02c583ee-registry-certificates\") pod \"image-registry-54895dc98-2h4rn\" (UID: \"815cb529-d322-4a64-a0c3-35de02c583ee\") " pod="openshift-image-registry/image-registry-54895dc98-2h4rn"
Sep 10 19:16:53.333445 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.333409    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/815cb529-d322-4a64-a0c3-35de02c583ee-ca-trust-extracted\") pod \"image-registry-54895dc98-2h4rn\" (UID: \"815cb529-d322-4a64-a0c3-35de02c583ee\") " pod="openshift-image-registry/image-registry-54895dc98-2h4rn"
Sep 10 19:16:53.336199 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.336157    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/815cb529-d322-4a64-a0c3-35de02c583ee-image-registry-private-configuration\") pod \"image-registry-54895dc98-2h4rn\" (UID: \"815cb529-d322-4a64-a0c3-35de02c583ee\") " pod="openshift-image-registry/image-registry-54895dc98-2h4rn"
Sep 10 19:16:53.339944 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.338854    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/815cb529-d322-4a64-a0c3-35de02c583ee-installation-pull-secrets\") pod \"image-registry-54895dc98-2h4rn\" (UID: \"815cb529-d322-4a64-a0c3-35de02c583ee\") " pod="openshift-image-registry/image-registry-54895dc98-2h4rn"
Sep 10 19:16:53.341363 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.340898    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvmxz\" (UniqueName: \"kubernetes.io/projected/ec666f15-107e-4574-9d6c-cc73b6840c2e-kube-api-access-lvmxz\") pod \"downloads-85d98c998f-ssx24\" (UID: \"ec666f15-107e-4574-9d6c-cc73b6840c2e\") " pod="openshift-console/downloads-85d98c998f-ssx24"
Sep 10 19:16:53.342167 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.342124    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/815cb529-d322-4a64-a0c3-35de02c583ee-registry-tls\") pod \"image-registry-54895dc98-2h4rn\" (UID: \"815cb529-d322-4a64-a0c3-35de02c583ee\") " pod="openshift-image-registry/image-registry-54895dc98-2h4rn"
Sep 10 19:16:53.344083 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.344041    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/815cb529-d322-4a64-a0c3-35de02c583ee-bound-sa-token\") pod \"image-registry-54895dc98-2h4rn\" (UID: \"815cb529-d322-4a64-a0c3-35de02c583ee\") " pod="openshift-image-registry/image-registry-54895dc98-2h4rn"
Sep 10 19:16:53.344408 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.344389    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6tvd\" (UniqueName: \"kubernetes.io/projected/815cb529-d322-4a64-a0c3-35de02c583ee-kube-api-access-g6tvd\") pod \"image-registry-54895dc98-2h4rn\" (UID: \"815cb529-d322-4a64-a0c3-35de02c583ee\") " pod="openshift-image-registry/image-registry-54895dc98-2h4rn"
Sep 10 19:16:53.366840 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.366810    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:53.374699 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.374675    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7667ff4d6d-lpk2n"
Sep 10 19:16:53.467706 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.467638    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:53.480584 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.479877    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-ssx24"
Sep 10 19:16:53.485742 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.485654    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-54895dc98-2h4rn"
Sep 10 19:16:53.698403 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.698147    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6cbff68f97-mt7bn" podStartSLOduration=63.698124269 podStartE2EDuration="1m3.698124269s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:15:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:53.251194716 +0000 UTC m=+68.852101959" watchObservedRunningTime="2026-09-10 19:16:53.698124269 +0000 UTC m=+69.299031510"
Sep 10 19:16:53.699371 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.699309    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7667ff4d6d-lpk2n"]
Sep 10 19:16:53.703267 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:16:53.703236    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeaca23fa_c738_4b43_bbcf_49c77a53ddfa.slice/crio-6477bb684e9bafab1bb2e41d452bd5afccb1a720fcf58acacb34428926b40575 WatchSource:0}: Error finding container 6477bb684e9bafab1bb2e41d452bd5afccb1a720fcf58acacb34428926b40575: Status 404 returned error can't find the container with id 6477bb684e9bafab1bb2e41d452bd5afccb1a720fcf58acacb34428926b40575
Sep 10 19:16:53.785309 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.785199    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-ssx24"]
Sep 10 19:16:53.790591 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:16:53.790558    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec666f15_107e_4574_9d6c_cc73b6840c2e.slice/crio-1f283221a239f90e02dcf56ea23ad9c959d396cd50a7dbcb4bced4d669e41a43 WatchSource:0}: Error finding container 1f283221a239f90e02dcf56ea23ad9c959d396cd50a7dbcb4bced4d669e41a43: Status 404 returned error can't find the container with id 1f283221a239f90e02dcf56ea23ad9c959d396cd50a7dbcb4bced4d669e41a43
Sep 10 19:16:53.813432 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:53.813396    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-54895dc98-2h4rn"]
Sep 10 19:16:53.817400 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:16:53.817370    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod815cb529_d322_4a64_a0c3_35de02c583ee.slice/crio-15031841bcb16237eab319ee8618ccfcaa0b00bf00d7fe9be92527395a7088e8 WatchSource:0}: Error finding container 15031841bcb16237eab319ee8618ccfcaa0b00bf00d7fe9be92527395a7088e8: Status 404 returned error can't find the container with id 15031841bcb16237eab319ee8618ccfcaa0b00bf00d7fe9be92527395a7088e8
Sep 10 19:16:54.006819 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:54.006762    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:54.095271 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:54.095182    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:54.197901 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:54.197862    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:54.230057 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:54.229966    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-54895dc98-2h4rn" event={"ID":"815cb529-d322-4a64-a0c3-35de02c583ee","Type":"ContainerStarted","Data":"317c4bbfe4329922d1bc5aa56c26c1fdaf0230388a4ca624426f9f6dd24b0b52"}
Sep 10 19:16:54.230057 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:54.230020    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-54895dc98-2h4rn" event={"ID":"815cb529-d322-4a64-a0c3-35de02c583ee","Type":"ContainerStarted","Data":"15031841bcb16237eab319ee8618ccfcaa0b00bf00d7fe9be92527395a7088e8"}
Sep 10 19:16:54.230561 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:54.230163    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-54895dc98-2h4rn"
Sep 10 19:16:54.231505 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:54.231475    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7667ff4d6d-lpk2n" event={"ID":"eaca23fa-c738-4b43-bbcf-49c77a53ddfa","Type":"ContainerStarted","Data":"6477bb684e9bafab1bb2e41d452bd5afccb1a720fcf58acacb34428926b40575"}
Sep 10 19:16:54.232882 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:54.232855    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-ssx24" event={"ID":"ec666f15-107e-4574-9d6c-cc73b6840c2e","Type":"ContainerStarted","Data":"1f283221a239f90e02dcf56ea23ad9c959d396cd50a7dbcb4bced4d669e41a43"}
Sep 10 19:16:54.235294 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:54.234978    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-tvlcw" event={"ID":"94b7fd86-07b1-4d2c-869f-787a7fcc6dca","Type":"ContainerStarted","Data":"fa64bcc926692f88613f69afd0e0c4a62a0933f687d45fa9aa9324567b7193a9"}
Sep 10 19:16:54.267018 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:54.266964    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-54895dc98-2h4rn" podStartSLOduration=1.266942552 podStartE2EDuration="1.266942552s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:16:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:54.266930936 +0000 UTC m=+69.867838186" watchObservedRunningTime="2026-09-10 19:16:54.266942552 +0000 UTC m=+69.867849824"
Sep 10 19:16:54.268729 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:54.268698    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:54.315138 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:54.315071    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-tvlcw" podStartSLOduration=68.440483677 podStartE2EDuration="1m10.315051212s" totalImagesPullingTime="1.874567535s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:15:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:54.313651364 +0000 UTC m=+69.914558608" watchObservedRunningTime="2026-09-10 19:16:54.315051212 +0000 UTC m=+69.915958451"
Sep 10 19:16:54.326832 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:54.326799    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:54.425294 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:54.425200    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:54.525503 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:54.525466    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:54.854258 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:54.854219    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:54.954080 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:54.954039    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:55.861035 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:55.860943    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:56.251411 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:56.251354    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:56.310528 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:56.310484    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:57.242337 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:57.242291    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:57.250720 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:57.250687    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vcs72" event={"ID":"80a423f3-63d3-42fa-a233-166ab675777e","Type":"ContainerStarted","Data":"40192f9d2d092d77b45e28027bde2749c2e92ea261a29746c661e1c387b14373"}
Sep 10 19:16:57.255991 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:57.255966    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-67nsx" event={"ID":"28f7bb38-4380-48b2-8881-38f1b188b5c8","Type":"ContainerStarted","Data":"b85fffb9e45601361dbbbd5522ac2b591b05822b017c0133322f7aaee9e362bd"}
Sep 10 19:16:57.259495 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:57.259464    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9stjj" event={"ID":"24f84abd-cd7a-40fa-b23f-72e60cc247b8","Type":"ContainerStarted","Data":"53d76ec96fc6eb4b71b2f01b7fc3ecc6ee56d113ef0d1a611a6aec2cf71a24d4"}
Sep 10 19:16:57.260020 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:57.260000    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-9stjj"
Sep 10 19:16:57.283949 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:57.283893    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-vcs72" podStartSLOduration=66.399501182 podStartE2EDuration="1m12.283869939s" totalImagesPullingTime="5.884368757s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:15:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:57.267958531 +0000 UTC m=+72.868865772" watchObservedRunningTime="2026-09-10 19:16:57.283869939 +0000 UTC m=+72.884777176"
Sep 10 19:16:57.284849 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:57.284822    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-5ctrq"]
Sep 10 19:16:57.288739 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:57.287519    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-9stjj" podStartSLOduration=66.395569544 podStartE2EDuration="1m12.287504856s" totalImagesPullingTime="5.891935312s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:15:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:57.286916809 +0000 UTC m=+72.887824040" watchObservedRunningTime="2026-09-10 19:16:57.287504856 +0000 UTC m=+72.888412096"
Sep 10 19:16:57.293711 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:57.292829    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-5ctrq"
Sep 10 19:16:57.297172 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:57.295883    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 10 19:16:57.297172 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:57.296119    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 10 19:16:57.297172 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:57.296306    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-q7p9c\""
Sep 10 19:16:57.297172 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:57.296547    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 10 19:16:57.297172 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:57.296814    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 10 19:16:57.297172 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:57.297021    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 10 19:16:57.309254 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:57.306734    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-5ctrq"]
Sep 10 19:16:57.309254 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:57.308778    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:57.365024 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:57.364989    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/86305da4-90a4-46c4-89c2-57af64a862fd-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-5ctrq\" (UID: \"86305da4-90a4-46c4-89c2-57af64a862fd\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-5ctrq"
Sep 10 19:16:57.365206 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:57.365060    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/86305da4-90a4-46c4-89c2-57af64a862fd-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-5ctrq\" (UID: \"86305da4-90a4-46c4-89c2-57af64a862fd\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-5ctrq"
Sep 10 19:16:57.365206 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:57.365104    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/86305da4-90a4-46c4-89c2-57af64a862fd-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-5ctrq\" (UID: \"86305da4-90a4-46c4-89c2-57af64a862fd\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-5ctrq"
Sep 10 19:16:57.365206 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:57.365140    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvbcx\" (UniqueName: \"kubernetes.io/projected/86305da4-90a4-46c4-89c2-57af64a862fd-kube-api-access-fvbcx\") pod \"prometheus-operator-6f4946db56-5ctrq\" (UID: \"86305da4-90a4-46c4-89c2-57af64a862fd\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-5ctrq"
Sep 10 19:16:57.397378 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:57.397341    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:57.467531 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:57.466014    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvbcx\" (UniqueName: \"kubernetes.io/projected/86305da4-90a4-46c4-89c2-57af64a862fd-kube-api-access-fvbcx\") pod \"prometheus-operator-6f4946db56-5ctrq\" (UID: \"86305da4-90a4-46c4-89c2-57af64a862fd\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-5ctrq"
Sep 10 19:16:57.467531 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:57.466075    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/86305da4-90a4-46c4-89c2-57af64a862fd-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-5ctrq\" (UID: \"86305da4-90a4-46c4-89c2-57af64a862fd\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-5ctrq"
Sep 10 19:16:57.467531 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:57.466129    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/86305da4-90a4-46c4-89c2-57af64a862fd-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-5ctrq\" (UID: \"86305da4-90a4-46c4-89c2-57af64a862fd\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-5ctrq"
Sep 10 19:16:57.467531 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:57.466186    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/86305da4-90a4-46c4-89c2-57af64a862fd-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-5ctrq\" (UID: \"86305da4-90a4-46c4-89c2-57af64a862fd\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-5ctrq"
Sep 10 19:16:57.467877 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:57.467713    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/86305da4-90a4-46c4-89c2-57af64a862fd-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-5ctrq\" (UID: \"86305da4-90a4-46c4-89c2-57af64a862fd\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-5ctrq"
Sep 10 19:16:57.469893 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:57.469843    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/86305da4-90a4-46c4-89c2-57af64a862fd-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-5ctrq\" (UID: \"86305da4-90a4-46c4-89c2-57af64a862fd\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-5ctrq"
Sep 10 19:16:57.470961 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:57.470922    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/86305da4-90a4-46c4-89c2-57af64a862fd-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-5ctrq\" (UID: \"86305da4-90a4-46c4-89c2-57af64a862fd\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-5ctrq"
Sep 10 19:16:57.480550 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:57.480443    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvbcx\" (UniqueName: \"kubernetes.io/projected/86305da4-90a4-46c4-89c2-57af64a862fd-kube-api-access-fvbcx\") pod \"prometheus-operator-6f4946db56-5ctrq\" (UID: \"86305da4-90a4-46c4-89c2-57af64a862fd\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-5ctrq"
Sep 10 19:16:57.620379 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:57.620286    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-5ctrq"
Sep 10 19:16:57.650330 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:57.650263    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:57.750019 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:57.749978    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:57.918437 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:57.918358    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-5ctrq"]
Sep 10 19:16:57.921814 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:16:57.921782    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86305da4_90a4_46c4_89c2_57af64a862fd.slice/crio-613a35020dedb38e5f14dcb65204e9f2f4c45cef4c79f4a3f661fadd882c2327 WatchSource:0}: Error finding container 613a35020dedb38e5f14dcb65204e9f2f4c45cef4c79f4a3f661fadd882c2327: Status 404 returned error can't find the container with id 613a35020dedb38e5f14dcb65204e9f2f4c45cef4c79f4a3f661fadd882c2327
Sep 10 19:16:58.264284 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:58.264244    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-5ctrq" event={"ID":"86305da4-90a4-46c4-89c2-57af64a862fd","Type":"ContainerStarted","Data":"613a35020dedb38e5f14dcb65204e9f2f4c45cef4c79f4a3f661fadd882c2327"}
Sep 10 19:16:58.266362 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:58.266330    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-67nsx" event={"ID":"28f7bb38-4380-48b2-8881-38f1b188b5c8","Type":"ContainerStarted","Data":"7b5497a172503abd80a3a9dbe7ba51f79220ebd7a4da0a2cce4f61948673a5c2"}
Sep 10 19:16:58.266502 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:58.266479    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-67nsx"
Sep 10 19:16:58.285437 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:58.285377    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-67nsx" podStartSLOduration=34.50768408 podStartE2EDuration="39.285357964s" totalImagesPullingTime="4.777673884s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:16:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:58.284875277 +0000 UTC m=+73.885782519" watchObservedRunningTime="2026-09-10 19:16:58.285357964 +0000 UTC m=+73.886265205"
Sep 10 19:16:59.272914 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:59.272860    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7667ff4d6d-lpk2n" event={"ID":"eaca23fa-c738-4b43-bbcf-49c77a53ddfa","Type":"ContainerStarted","Data":"55cafb5af50e00604b1d09799c16b7b0f604280ecdbad86f728c5d6537cc879d"}
Sep 10 19:16:59.288880 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:59.288834    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7667ff4d6d-lpk2n" podStartSLOduration=0.84303785 podStartE2EDuration="6.288819519s" totalImagesPullingTime="5.445781669s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:16:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:59.287871075 +0000 UTC m=+74.888778314" watchObservedRunningTime="2026-09-10 19:16:59.288819519 +0000 UTC m=+74.889726759"
Sep 10 19:16:59.980290 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:16:59.980253    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:00.035027 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:00.034973    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:00.279834 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:00.279801    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-5ctrq" event={"ID":"86305da4-90a4-46c4-89c2-57af64a862fd","Type":"ContainerStarted","Data":"e661a7880c1d52dd709bb4bfeeebedcf8021e426607f5430d7941d574b6d4742"}
Sep 10 19:17:00.280267 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:00.279854    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-5ctrq" event={"ID":"86305da4-90a4-46c4-89c2-57af64a862fd","Type":"ContainerStarted","Data":"86acd4ef2733b608dd06d22b1c7f9193bb993658c6253aeec179f82fbbf7f46c"}
Sep 10 19:17:00.297944 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:00.297891    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-6f4946db56-5ctrq" podStartSLOduration=1.148516616 podStartE2EDuration="3.297875097s" totalImagesPullingTime="2.149358481s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:16:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:17:00.29652459 +0000 UTC m=+75.897431831" watchObservedRunningTime="2026-09-10 19:17:00.297875097 +0000 UTC m=+75.898782335"
Sep 10 19:17:00.376463 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:00.376423    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:01.184779 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:01.184743    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7f94744dd8-k8jg5"]
Sep 10 19:17:01.187891 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:01.187868    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f94744dd8-k8jg5"
Sep 10 19:17:01.190069 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:01.190043    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 10 19:17:01.190197 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:01.190050    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 10 19:17:01.191014 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:01.190993    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 10 19:17:01.191145 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:01.191067    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-2hqsq\""
Sep 10 19:17:01.191145 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:01.191083    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 10 19:17:01.191145 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:01.191138    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 10 19:17:01.198269 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:01.198248    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f94744dd8-k8jg5"]
Sep 10 19:17:01.298782 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:01.298743    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bn2x7\" (UniqueName: \"kubernetes.io/projected/13aa0d02-73a3-4f27-993b-46ab70dbed06-kube-api-access-bn2x7\") pod \"console-7f94744dd8-k8jg5\" (UID: \"13aa0d02-73a3-4f27-993b-46ab70dbed06\") " pod="openshift-console/console-7f94744dd8-k8jg5"
Sep 10 19:17:01.298782 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:01.298785    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/13aa0d02-73a3-4f27-993b-46ab70dbed06-console-oauth-config\") pod \"console-7f94744dd8-k8jg5\" (UID: \"13aa0d02-73a3-4f27-993b-46ab70dbed06\") " pod="openshift-console/console-7f94744dd8-k8jg5"
Sep 10 19:17:01.299289 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:01.298806    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/13aa0d02-73a3-4f27-993b-46ab70dbed06-service-ca\") pod \"console-7f94744dd8-k8jg5\" (UID: \"13aa0d02-73a3-4f27-993b-46ab70dbed06\") " pod="openshift-console/console-7f94744dd8-k8jg5"
Sep 10 19:17:01.299289 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:01.298881    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/13aa0d02-73a3-4f27-993b-46ab70dbed06-oauth-serving-cert\") pod \"console-7f94744dd8-k8jg5\" (UID: \"13aa0d02-73a3-4f27-993b-46ab70dbed06\") " pod="openshift-console/console-7f94744dd8-k8jg5"
Sep 10 19:17:01.299289 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:01.298910    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/13aa0d02-73a3-4f27-993b-46ab70dbed06-console-config\") pod \"console-7f94744dd8-k8jg5\" (UID: \"13aa0d02-73a3-4f27-993b-46ab70dbed06\") " pod="openshift-console/console-7f94744dd8-k8jg5"
Sep 10 19:17:01.299289 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:01.298942    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/13aa0d02-73a3-4f27-993b-46ab70dbed06-console-serving-cert\") pod \"console-7f94744dd8-k8jg5\" (UID: \"13aa0d02-73a3-4f27-993b-46ab70dbed06\") " pod="openshift-console/console-7f94744dd8-k8jg5"
Sep 10 19:17:01.400107 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:01.400072    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/13aa0d02-73a3-4f27-993b-46ab70dbed06-console-serving-cert\") pod \"console-7f94744dd8-k8jg5\" (UID: \"13aa0d02-73a3-4f27-993b-46ab70dbed06\") " pod="openshift-console/console-7f94744dd8-k8jg5"
Sep 10 19:17:01.400341 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:01.400316    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bn2x7\" (UniqueName: \"kubernetes.io/projected/13aa0d02-73a3-4f27-993b-46ab70dbed06-kube-api-access-bn2x7\") pod \"console-7f94744dd8-k8jg5\" (UID: \"13aa0d02-73a3-4f27-993b-46ab70dbed06\") " pod="openshift-console/console-7f94744dd8-k8jg5"
Sep 10 19:17:01.400457 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:01.400386    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/13aa0d02-73a3-4f27-993b-46ab70dbed06-console-oauth-config\") pod \"console-7f94744dd8-k8jg5\" (UID: \"13aa0d02-73a3-4f27-993b-46ab70dbed06\") " pod="openshift-console/console-7f94744dd8-k8jg5"
Sep 10 19:17:01.400457 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:01.400421    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/13aa0d02-73a3-4f27-993b-46ab70dbed06-service-ca\") pod \"console-7f94744dd8-k8jg5\" (UID: \"13aa0d02-73a3-4f27-993b-46ab70dbed06\") " pod="openshift-console/console-7f94744dd8-k8jg5"
Sep 10 19:17:01.400563 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:01.400509    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/13aa0d02-73a3-4f27-993b-46ab70dbed06-oauth-serving-cert\") pod \"console-7f94744dd8-k8jg5\" (UID: \"13aa0d02-73a3-4f27-993b-46ab70dbed06\") " pod="openshift-console/console-7f94744dd8-k8jg5"
Sep 10 19:17:01.400613 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:01.400574    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/13aa0d02-73a3-4f27-993b-46ab70dbed06-console-config\") pod \"console-7f94744dd8-k8jg5\" (UID: \"13aa0d02-73a3-4f27-993b-46ab70dbed06\") " pod="openshift-console/console-7f94744dd8-k8jg5"
Sep 10 19:17:01.401231 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:01.401205    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/13aa0d02-73a3-4f27-993b-46ab70dbed06-oauth-serving-cert\") pod \"console-7f94744dd8-k8jg5\" (UID: \"13aa0d02-73a3-4f27-993b-46ab70dbed06\") " pod="openshift-console/console-7f94744dd8-k8jg5"
Sep 10 19:17:01.401356 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:01.401326    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/13aa0d02-73a3-4f27-993b-46ab70dbed06-console-config\") pod \"console-7f94744dd8-k8jg5\" (UID: \"13aa0d02-73a3-4f27-993b-46ab70dbed06\") " pod="openshift-console/console-7f94744dd8-k8jg5"
Sep 10 19:17:01.401356 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:01.401339    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/13aa0d02-73a3-4f27-993b-46ab70dbed06-service-ca\") pod \"console-7f94744dd8-k8jg5\" (UID: \"13aa0d02-73a3-4f27-993b-46ab70dbed06\") " pod="openshift-console/console-7f94744dd8-k8jg5"
Sep 10 19:17:01.403116 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:01.403091    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/13aa0d02-73a3-4f27-993b-46ab70dbed06-console-serving-cert\") pod \"console-7f94744dd8-k8jg5\" (UID: \"13aa0d02-73a3-4f27-993b-46ab70dbed06\") " pod="openshift-console/console-7f94744dd8-k8jg5"
Sep 10 19:17:01.403246 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:01.403224    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/13aa0d02-73a3-4f27-993b-46ab70dbed06-console-oauth-config\") pod \"console-7f94744dd8-k8jg5\" (UID: \"13aa0d02-73a3-4f27-993b-46ab70dbed06\") " pod="openshift-console/console-7f94744dd8-k8jg5"
Sep 10 19:17:01.409999 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:01.409975    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bn2x7\" (UniqueName: \"kubernetes.io/projected/13aa0d02-73a3-4f27-993b-46ab70dbed06-kube-api-access-bn2x7\") pod \"console-7f94744dd8-k8jg5\" (UID: \"13aa0d02-73a3-4f27-993b-46ab70dbed06\") " pod="openshift-console/console-7f94744dd8-k8jg5"
Sep 10 19:17:01.499897 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:01.499864    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f94744dd8-k8jg5"
Sep 10 19:17:01.603603 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:01.603571    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:01.659198 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:01.659159    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:01.777709 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:01.777613    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f94744dd8-k8jg5"]
Sep 10 19:17:01.781595 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:17:01.781562    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13aa0d02_73a3_4f27_993b_46ab70dbed06.slice/crio-ed4622632dbd3ca54e3717c6495f5813c69146d521a54370ca09155bed1bf869 WatchSource:0}: Error finding container ed4622632dbd3ca54e3717c6495f5813c69146d521a54370ca09155bed1bf869: Status 404 returned error can't find the container with id ed4622632dbd3ca54e3717c6495f5813c69146d521a54370ca09155bed1bf869
Sep 10 19:17:02.289524 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:02.289488    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f94744dd8-k8jg5" event={"ID":"13aa0d02-73a3-4f27-993b-46ab70dbed06","Type":"ContainerStarted","Data":"ed4622632dbd3ca54e3717c6495f5813c69146d521a54370ca09155bed1bf869"}
Sep 10 19:17:02.414125 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:02.414067    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:02.641683 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:02.641575    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:02.665327 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:02.665295    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-k72xf"]
Sep 10 19:17:02.668795 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:02.668770    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-k72xf"
Sep 10 19:17:02.671408 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:02.671383    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 10 19:17:02.671547 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:02.671383    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 10 19:17:02.671714 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:02.671697    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-zg7ls\""
Sep 10 19:17:02.672287 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:02.672261    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 10 19:17:02.685566 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:02.685539    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-qgvtq"]
Sep 10 19:17:02.688975 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:02.688956    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-qgvtq"
Sep 10 19:17:02.691332 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:02.691308    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 10 19:17:02.691426 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:02.691336    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-gfmwj\""
Sep 10 19:17:02.691426 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:02.691327    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 10 19:17:02.691705 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:02.691689    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 10 19:17:02.700720 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:02.700614    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-qgvtq"]
Sep 10 19:17:02.711000 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:02.710966    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ee547a46-3813-4faa-a7bd-7e05be04e865-sys\") pod \"node-exporter-k72xf\" (UID: \"ee547a46-3813-4faa-a7bd-7e05be04e865\") " pod="openshift-monitoring/node-exporter-k72xf"
Sep 10 19:17:02.711164 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:02.711026    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ee547a46-3813-4faa-a7bd-7e05be04e865-node-exporter-accelerators-collector-config\") pod \"node-exporter-k72xf\" (UID: \"ee547a46-3813-4faa-a7bd-7e05be04e865\") " pod="openshift-monitoring/node-exporter-k72xf"
Sep 10 19:17:02.711164 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:02.711060    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ee547a46-3813-4faa-a7bd-7e05be04e865-node-exporter-tls\") pod \"node-exporter-k72xf\" (UID: \"ee547a46-3813-4faa-a7bd-7e05be04e865\") " pod="openshift-monitoring/node-exporter-k72xf"
Sep 10 19:17:02.711164 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:02.711093    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ee547a46-3813-4faa-a7bd-7e05be04e865-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-k72xf\" (UID: \"ee547a46-3813-4faa-a7bd-7e05be04e865\") " pod="openshift-monitoring/node-exporter-k72xf"
Sep 10 19:17:02.711164 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:02.711121    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ee547a46-3813-4faa-a7bd-7e05be04e865-node-exporter-wtmp\") pod \"node-exporter-k72xf\" (UID: \"ee547a46-3813-4faa-a7bd-7e05be04e865\") " pod="openshift-monitoring/node-exporter-k72xf"
Sep 10 19:17:02.711164 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:02.711148    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvtz8\" (UniqueName: \"kubernetes.io/projected/ee547a46-3813-4faa-a7bd-7e05be04e865-kube-api-access-xvtz8\") pod \"node-exporter-k72xf\" (UID: \"ee547a46-3813-4faa-a7bd-7e05be04e865\") " pod="openshift-monitoring/node-exporter-k72xf"
Sep 10 19:17:02.711347 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:02.711259    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ee547a46-3813-4faa-a7bd-7e05be04e865-root\") pod \"node-exporter-k72xf\" (UID: \"ee547a46-3813-4faa-a7bd-7e05be04e865\") " pod="openshift-monitoring/node-exporter-k72xf"
Sep 10 19:17:02.711347 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:02.711317    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee547a46-3813-4faa-a7bd-7e05be04e865-metrics-client-ca\") pod \"node-exporter-k72xf\" (UID: \"ee547a46-3813-4faa-a7bd-7e05be04e865\") " pod="openshift-monitoring/node-exporter-k72xf"
Sep 10 19:17:02.711438 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:02.711354    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ee547a46-3813-4faa-a7bd-7e05be04e865-node-exporter-textfile\") pod \"node-exporter-k72xf\" (UID: \"ee547a46-3813-4faa-a7bd-7e05be04e865\") " pod="openshift-monitoring/node-exporter-k72xf"
Sep 10 19:17:02.719333 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:02.719303    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:02.812920 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:02.812884    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ee547a46-3813-4faa-a7bd-7e05be04e865-node-exporter-accelerators-collector-config\") pod \"node-exporter-k72xf\" (UID: \"ee547a46-3813-4faa-a7bd-7e05be04e865\") " pod="openshift-monitoring/node-exporter-k72xf"
Sep 10 19:17:02.813114 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:02.812933    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ee547a46-3813-4faa-a7bd-7e05be04e865-node-exporter-tls\") pod \"node-exporter-k72xf\" (UID: \"ee547a46-3813-4faa-a7bd-7e05be04e865\") " pod="openshift-monitoring/node-exporter-k72xf"
Sep 10 19:17:02.813114 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:02.812975    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4bwk\" (UniqueName: \"kubernetes.io/projected/eeec0ca3-75b9-4590-a29e-609fa17b7e7e-kube-api-access-x4bwk\") pod \"kube-state-metrics-74cb57df7-qgvtq\" (UID: \"eeec0ca3-75b9-4590-a29e-609fa17b7e7e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-qgvtq"
Sep 10 19:17:02.813114 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:02.813004    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ee547a46-3813-4faa-a7bd-7e05be04e865-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-k72xf\" (UID: \"ee547a46-3813-4faa-a7bd-7e05be04e865\") " pod="openshift-monitoring/node-exporter-k72xf"
Sep 10 19:17:02.813114 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:02.813034    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ee547a46-3813-4faa-a7bd-7e05be04e865-node-exporter-wtmp\") pod \"node-exporter-k72xf\" (UID: \"ee547a46-3813-4faa-a7bd-7e05be04e865\") " pod="openshift-monitoring/node-exporter-k72xf"
Sep 10 19:17:02.813114 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:02.813064    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvtz8\" (UniqueName: \"kubernetes.io/projected/ee547a46-3813-4faa-a7bd-7e05be04e865-kube-api-access-xvtz8\") pod \"node-exporter-k72xf\" (UID: \"ee547a46-3813-4faa-a7bd-7e05be04e865\") " pod="openshift-monitoring/node-exporter-k72xf"
Sep 10 19:17:02.813114 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:02.813094    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ee547a46-3813-4faa-a7bd-7e05be04e865-root\") pod \"node-exporter-k72xf\" (UID: \"ee547a46-3813-4faa-a7bd-7e05be04e865\") " pod="openshift-monitoring/node-exporter-k72xf"
Sep 10 19:17:02.813374 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:02.813130    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eeec0ca3-75b9-4590-a29e-609fa17b7e7e-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-qgvtq\" (UID: \"eeec0ca3-75b9-4590-a29e-609fa17b7e7e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-qgvtq"
Sep 10 19:17:02.813374 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:02.813168    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee547a46-3813-4faa-a7bd-7e05be04e865-metrics-client-ca\") pod \"node-exporter-k72xf\" (UID: \"ee547a46-3813-4faa-a7bd-7e05be04e865\") " pod="openshift-monitoring/node-exporter-k72xf"
Sep 10 19:17:02.813374 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:02.813197    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ee547a46-3813-4faa-a7bd-7e05be04e865-node-exporter-textfile\") pod \"node-exporter-k72xf\" (UID: \"ee547a46-3813-4faa-a7bd-7e05be04e865\") " pod="openshift-monitoring/node-exporter-k72xf"
Sep 10 19:17:02.813374 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:02.813242    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eeec0ca3-75b9-4590-a29e-609fa17b7e7e-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-qgvtq\" (UID: \"eeec0ca3-75b9-4590-a29e-609fa17b7e7e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-qgvtq"
Sep 10 19:17:02.813374 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:02.813277    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/eeec0ca3-75b9-4590-a29e-609fa17b7e7e-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-qgvtq\" (UID: \"eeec0ca3-75b9-4590-a29e-609fa17b7e7e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-qgvtq"
Sep 10 19:17:02.813374 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:02.813308    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ee547a46-3813-4faa-a7bd-7e05be04e865-sys\") pod \"node-exporter-k72xf\" (UID: \"ee547a46-3813-4faa-a7bd-7e05be04e865\") " pod="openshift-monitoring/node-exporter-k72xf"
Sep 10 19:17:02.813374 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:02.813343    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/eeec0ca3-75b9-4590-a29e-609fa17b7e7e-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-qgvtq\" (UID: \"eeec0ca3-75b9-4590-a29e-609fa17b7e7e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-qgvtq"
Sep 10 19:17:02.813374 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:02.813368    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/eeec0ca3-75b9-4590-a29e-609fa17b7e7e-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-qgvtq\" (UID: \"eeec0ca3-75b9-4590-a29e-609fa17b7e7e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-qgvtq"
Sep 10 19:17:02.814054 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:02.814025    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ee547a46-3813-4faa-a7bd-7e05be04e865-node-exporter-accelerators-collector-config\") pod \"node-exporter-k72xf\" (UID: \"ee547a46-3813-4faa-a7bd-7e05be04e865\") " pod="openshift-monitoring/node-exporter-k72xf"
Sep 10 19:17:02.814171 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:02.814111    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ee547a46-3813-4faa-a7bd-7e05be04e865-sys\") pod \"node-exporter-k72xf\" (UID: \"ee547a46-3813-4faa-a7bd-7e05be04e865\") " pod="openshift-monitoring/node-exporter-k72xf"
Sep 10 19:17:02.814351 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:02.814327    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee547a46-3813-4faa-a7bd-7e05be04e865-metrics-client-ca\") pod \"node-exporter-k72xf\" (UID: \"ee547a46-3813-4faa-a7bd-7e05be04e865\") " pod="openshift-monitoring/node-exporter-k72xf"
Sep 10 19:17:02.814447 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:02.814356    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ee547a46-3813-4faa-a7bd-7e05be04e865-node-exporter-textfile\") pod \"node-exporter-k72xf\" (UID: \"ee547a46-3813-4faa-a7bd-7e05be04e865\") " pod="openshift-monitoring/node-exporter-k72xf"
Sep 10 19:17:02.814447 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:02.814397    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ee547a46-3813-4faa-a7bd-7e05be04e865-node-exporter-wtmp\") pod \"node-exporter-k72xf\" (UID: \"ee547a46-3813-4faa-a7bd-7e05be04e865\") " pod="openshift-monitoring/node-exporter-k72xf"
Sep 10 19:17:02.814447 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:02.814418    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ee547a46-3813-4faa-a7bd-7e05be04e865-root\") pod \"node-exporter-k72xf\" (UID: \"ee547a46-3813-4faa-a7bd-7e05be04e865\") " pod="openshift-monitoring/node-exporter-k72xf"
Sep 10 19:17:02.820299 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:02.817280    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ee547a46-3813-4faa-a7bd-7e05be04e865-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-k72xf\" (UID: \"ee547a46-3813-4faa-a7bd-7e05be04e865\") " pod="openshift-monitoring/node-exporter-k72xf"
Sep 10 19:17:02.820299 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:02.817397    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ee547a46-3813-4faa-a7bd-7e05be04e865-node-exporter-tls\") pod \"node-exporter-k72xf\" (UID: \"ee547a46-3813-4faa-a7bd-7e05be04e865\") " pod="openshift-monitoring/node-exporter-k72xf"
Sep 10 19:17:02.823554 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:02.823530    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvtz8\" (UniqueName: \"kubernetes.io/projected/ee547a46-3813-4faa-a7bd-7e05be04e865-kube-api-access-xvtz8\") pod \"node-exporter-k72xf\" (UID: \"ee547a46-3813-4faa-a7bd-7e05be04e865\") " pod="openshift-monitoring/node-exporter-k72xf"
Sep 10 19:17:02.854096 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:02.854064    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:02.915219 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:02.914581    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eeec0ca3-75b9-4590-a29e-609fa17b7e7e-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-qgvtq\" (UID: \"eeec0ca3-75b9-4590-a29e-609fa17b7e7e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-qgvtq"
Sep 10 19:17:02.915219 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:02.914647    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/eeec0ca3-75b9-4590-a29e-609fa17b7e7e-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-qgvtq\" (UID: \"eeec0ca3-75b9-4590-a29e-609fa17b7e7e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-qgvtq"
Sep 10 19:17:02.915219 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:02.914691    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/eeec0ca3-75b9-4590-a29e-609fa17b7e7e-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-qgvtq\" (UID: \"eeec0ca3-75b9-4590-a29e-609fa17b7e7e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-qgvtq"
Sep 10 19:17:02.915219 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:02.914714    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/eeec0ca3-75b9-4590-a29e-609fa17b7e7e-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-qgvtq\" (UID: \"eeec0ca3-75b9-4590-a29e-609fa17b7e7e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-qgvtq"
Sep 10 19:17:02.915219 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:02.914757    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4bwk\" (UniqueName: \"kubernetes.io/projected/eeec0ca3-75b9-4590-a29e-609fa17b7e7e-kube-api-access-x4bwk\") pod \"kube-state-metrics-74cb57df7-qgvtq\" (UID: \"eeec0ca3-75b9-4590-a29e-609fa17b7e7e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-qgvtq"
Sep 10 19:17:02.915219 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:02.914811    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eeec0ca3-75b9-4590-a29e-609fa17b7e7e-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-qgvtq\" (UID: \"eeec0ca3-75b9-4590-a29e-609fa17b7e7e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-qgvtq"
Sep 10 19:17:02.915219 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:17:02.915121    2564 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Sep 10 19:17:02.915219 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:02.915154    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/eeec0ca3-75b9-4590-a29e-609fa17b7e7e-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-qgvtq\" (UID: \"eeec0ca3-75b9-4590-a29e-609fa17b7e7e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-qgvtq"
Sep 10 19:17:02.915219 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:17:02.915201    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eeec0ca3-75b9-4590-a29e-609fa17b7e7e-kube-state-metrics-tls podName:eeec0ca3-75b9-4590-a29e-609fa17b7e7e nodeName:}" failed. No retries permitted until 2026-09-10 19:17:03.415177979 +0000 UTC m=+79.016085205 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/eeec0ca3-75b9-4590-a29e-609fa17b7e7e-kube-state-metrics-tls") pod "kube-state-metrics-74cb57df7-qgvtq" (UID: "eeec0ca3-75b9-4590-a29e-609fa17b7e7e") : secret "kube-state-metrics-tls" not found
Sep 10 19:17:02.915821 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:02.915527    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/eeec0ca3-75b9-4590-a29e-609fa17b7e7e-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-qgvtq\" (UID: \"eeec0ca3-75b9-4590-a29e-609fa17b7e7e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-qgvtq"
Sep 10 19:17:02.915821 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:02.915536    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eeec0ca3-75b9-4590-a29e-609fa17b7e7e-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-qgvtq\" (UID: \"eeec0ca3-75b9-4590-a29e-609fa17b7e7e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-qgvtq"
Sep 10 19:17:02.917728 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:02.917704    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eeec0ca3-75b9-4590-a29e-609fa17b7e7e-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-qgvtq\" (UID: \"eeec0ca3-75b9-4590-a29e-609fa17b7e7e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-qgvtq"
Sep 10 19:17:02.925785 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:02.925759    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4bwk\" (UniqueName: \"kubernetes.io/projected/eeec0ca3-75b9-4590-a29e-609fa17b7e7e-kube-api-access-x4bwk\") pod \"kube-state-metrics-74cb57df7-qgvtq\" (UID: \"eeec0ca3-75b9-4590-a29e-609fa17b7e7e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-qgvtq"
Sep 10 19:17:02.956972 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:02.956935    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:02.981841 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:02.981806    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-k72xf"
Sep 10 19:17:02.996638 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:17:02.996587    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee547a46_3813_4faa_a7bd_7e05be04e865.slice/crio-a5324865238f0b3d9f6a916c4bafd7651d28eb647964093fa4553839cd7acd01 WatchSource:0}: Error finding container a5324865238f0b3d9f6a916c4bafd7651d28eb647964093fa4553839cd7acd01: Status 404 returned error can't find the container with id a5324865238f0b3d9f6a916c4bafd7651d28eb647964093fa4553839cd7acd01
Sep 10 19:17:03.295463 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:03.295420    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-k72xf" event={"ID":"ee547a46-3813-4faa-a7bd-7e05be04e865","Type":"ContainerStarted","Data":"a5324865238f0b3d9f6a916c4bafd7651d28eb647964093fa4553839cd7acd01"}
Sep 10 19:17:03.421371 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:03.421326    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/eeec0ca3-75b9-4590-a29e-609fa17b7e7e-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-qgvtq\" (UID: \"eeec0ca3-75b9-4590-a29e-609fa17b7e7e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-qgvtq"
Sep 10 19:17:03.421864 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:17:03.421480    2564 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Sep 10 19:17:03.421864 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:17:03.421569    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eeec0ca3-75b9-4590-a29e-609fa17b7e7e-kube-state-metrics-tls podName:eeec0ca3-75b9-4590-a29e-609fa17b7e7e nodeName:}" failed. No retries permitted until 2026-09-10 19:17:04.421548104 +0000 UTC m=+80.022455336 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/eeec0ca3-75b9-4590-a29e-609fa17b7e7e-kube-state-metrics-tls") pod "kube-state-metrics-74cb57df7-qgvtq" (UID: "eeec0ca3-75b9-4590-a29e-609fa17b7e7e") : secret "kube-state-metrics-tls" not found
Sep 10 19:17:03.527424 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:03.527360    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:03.632126 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:03.632047    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:03.732865 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:03.732830    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:03.827922 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:03.827875    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:04.114494 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:04.114461    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:04.175258 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:04.175178    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:04.434675 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:04.434569    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/eeec0ca3-75b9-4590-a29e-609fa17b7e7e-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-qgvtq\" (UID: \"eeec0ca3-75b9-4590-a29e-609fa17b7e7e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-qgvtq"
Sep 10 19:17:04.437607 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:04.437579    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/eeec0ca3-75b9-4590-a29e-609fa17b7e7e-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-qgvtq\" (UID: \"eeec0ca3-75b9-4590-a29e-609fa17b7e7e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-qgvtq"
Sep 10 19:17:04.498789 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:04.498747    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-qgvtq"
Sep 10 19:17:05.010367 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:05.010331    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:05.089107 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:05.089065    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:05.259719 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:05.259617    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:05.359801 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:05.359770    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:05.545758 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:05.545674    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-qgvtq"]
Sep 10 19:17:05.549718 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:17:05.549674    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeeec0ca3_75b9_4590_a29e_609fa17b7e7e.slice/crio-0aa83fb5eda3ec3ae596d93e785e86581bb0418ce138d96f0722c5861fee25c4 WatchSource:0}: Error finding container 0aa83fb5eda3ec3ae596d93e785e86581bb0418ce138d96f0722c5861fee25c4: Status 404 returned error can't find the container with id 0aa83fb5eda3ec3ae596d93e785e86581bb0418ce138d96f0722c5861fee25c4
Sep 10 19:17:05.897934 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:05.897845    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:06.309529 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:06.309498    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f94744dd8-k8jg5" event={"ID":"13aa0d02-73a3-4f27-993b-46ab70dbed06","Type":"ContainerStarted","Data":"f90abf4c46e09038abcaf4083b3eaedda276893fad997ebe301cb2c6f9d5c7b4"}
Sep 10 19:17:06.311196 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:06.311169    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-qgvtq" event={"ID":"eeec0ca3-75b9-4590-a29e-609fa17b7e7e","Type":"ContainerStarted","Data":"0aa83fb5eda3ec3ae596d93e785e86581bb0418ce138d96f0722c5861fee25c4"}
Sep 10 19:17:06.312931 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:06.312908    2564 generic.go:356] "Generic (PLEG): container finished" podID="ee547a46-3813-4faa-a7bd-7e05be04e865" containerID="c1935e93210d37acf97d91ac91e7feda3b57348efce98728aacf7a0c2e165b1b" exitCode=0
Sep 10 19:17:06.313035 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:06.312970    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-k72xf" event={"ID":"ee547a46-3813-4faa-a7bd-7e05be04e865","Type":"ContainerDied","Data":"c1935e93210d37acf97d91ac91e7feda3b57348efce98728aacf7a0c2e165b1b"}
Sep 10 19:17:06.328779 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:06.328725    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7f94744dd8-k8jg5" podStartSLOduration=1.7916094949999999 podStartE2EDuration="5.328705651s" totalImagesPullingTime="3.537096156s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:17:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:17:06.326539251 +0000 UTC m=+81.927446490" watchObservedRunningTime="2026-09-10 19:17:06.328705651 +0000 UTC m=+81.929612892"
Sep 10 19:17:06.375099 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:06.375070    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:06.888848 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:06.888806    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:07.076094 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:07.076062    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-7578c5ccf6-wpp2l"]
Sep 10 19:17:07.079537 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:07.079509    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7578c5ccf6-wpp2l"
Sep 10 19:17:07.082708 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:07.082577    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 10 19:17:07.082708 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:07.082653    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 10 19:17:07.083693 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:07.083673    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 10 19:17:07.083792 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:07.083694    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-mmnrb\""
Sep 10 19:17:07.083792 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:07.083700    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 10 19:17:07.084014 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:07.083995    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-egb8s1r863581\""
Sep 10 19:17:07.089395 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:07.089338    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7578c5ccf6-wpp2l"]
Sep 10 19:17:07.156423 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:07.156340    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/69415242-9c1a-4b21-bcf2-a4afb82cdbd2-secret-metrics-server-tls\") pod \"metrics-server-7578c5ccf6-wpp2l\" (UID: \"69415242-9c1a-4b21-bcf2-a4afb82cdbd2\") " pod="openshift-monitoring/metrics-server-7578c5ccf6-wpp2l"
Sep 10 19:17:07.156423 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:07.156393    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgbsh\" (UniqueName: \"kubernetes.io/projected/69415242-9c1a-4b21-bcf2-a4afb82cdbd2-kube-api-access-bgbsh\") pod \"metrics-server-7578c5ccf6-wpp2l\" (UID: \"69415242-9c1a-4b21-bcf2-a4afb82cdbd2\") " pod="openshift-monitoring/metrics-server-7578c5ccf6-wpp2l"
Sep 10 19:17:07.156645 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:07.156480    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/69415242-9c1a-4b21-bcf2-a4afb82cdbd2-metrics-server-audit-profiles\") pod \"metrics-server-7578c5ccf6-wpp2l\" (UID: \"69415242-9c1a-4b21-bcf2-a4afb82cdbd2\") " pod="openshift-monitoring/metrics-server-7578c5ccf6-wpp2l"
Sep 10 19:17:07.156645 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:07.156536    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/69415242-9c1a-4b21-bcf2-a4afb82cdbd2-client-ca-bundle\") pod \"metrics-server-7578c5ccf6-wpp2l\" (UID: \"69415242-9c1a-4b21-bcf2-a4afb82cdbd2\") " pod="openshift-monitoring/metrics-server-7578c5ccf6-wpp2l"
Sep 10 19:17:07.156645 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:07.156577    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69415242-9c1a-4b21-bcf2-a4afb82cdbd2-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7578c5ccf6-wpp2l\" (UID: \"69415242-9c1a-4b21-bcf2-a4afb82cdbd2\") " pod="openshift-monitoring/metrics-server-7578c5ccf6-wpp2l"
Sep 10 19:17:07.156645 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:07.156606    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/69415242-9c1a-4b21-bcf2-a4afb82cdbd2-secret-metrics-server-client-certs\") pod \"metrics-server-7578c5ccf6-wpp2l\" (UID: \"69415242-9c1a-4b21-bcf2-a4afb82cdbd2\") " pod="openshift-monitoring/metrics-server-7578c5ccf6-wpp2l"
Sep 10 19:17:07.156855 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:07.156660    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/69415242-9c1a-4b21-bcf2-a4afb82cdbd2-audit-log\") pod \"metrics-server-7578c5ccf6-wpp2l\" (UID: \"69415242-9c1a-4b21-bcf2-a4afb82cdbd2\") " pod="openshift-monitoring/metrics-server-7578c5ccf6-wpp2l"
Sep 10 19:17:07.257654 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:07.257604    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/69415242-9c1a-4b21-bcf2-a4afb82cdbd2-audit-log\") pod \"metrics-server-7578c5ccf6-wpp2l\" (UID: \"69415242-9c1a-4b21-bcf2-a4afb82cdbd2\") " pod="openshift-monitoring/metrics-server-7578c5ccf6-wpp2l"
Sep 10 19:17:07.257768 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:07.257691    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/69415242-9c1a-4b21-bcf2-a4afb82cdbd2-secret-metrics-server-tls\") pod \"metrics-server-7578c5ccf6-wpp2l\" (UID: \"69415242-9c1a-4b21-bcf2-a4afb82cdbd2\") " pod="openshift-monitoring/metrics-server-7578c5ccf6-wpp2l"
Sep 10 19:17:07.257768 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:07.257730    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgbsh\" (UniqueName: \"kubernetes.io/projected/69415242-9c1a-4b21-bcf2-a4afb82cdbd2-kube-api-access-bgbsh\") pod \"metrics-server-7578c5ccf6-wpp2l\" (UID: \"69415242-9c1a-4b21-bcf2-a4afb82cdbd2\") " pod="openshift-monitoring/metrics-server-7578c5ccf6-wpp2l"
Sep 10 19:17:07.257867 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:07.257768    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/69415242-9c1a-4b21-bcf2-a4afb82cdbd2-metrics-server-audit-profiles\") pod \"metrics-server-7578c5ccf6-wpp2l\" (UID: \"69415242-9c1a-4b21-bcf2-a4afb82cdbd2\") " pod="openshift-monitoring/metrics-server-7578c5ccf6-wpp2l"
Sep 10 19:17:07.257867 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:07.257824    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/69415242-9c1a-4b21-bcf2-a4afb82cdbd2-client-ca-bundle\") pod \"metrics-server-7578c5ccf6-wpp2l\" (UID: \"69415242-9c1a-4b21-bcf2-a4afb82cdbd2\") " pod="openshift-monitoring/metrics-server-7578c5ccf6-wpp2l"
Sep 10 19:17:07.257867 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:07.257857    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69415242-9c1a-4b21-bcf2-a4afb82cdbd2-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7578c5ccf6-wpp2l\" (UID: \"69415242-9c1a-4b21-bcf2-a4afb82cdbd2\") " pod="openshift-monitoring/metrics-server-7578c5ccf6-wpp2l"
Sep 10 19:17:07.258015 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:07.257884    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/69415242-9c1a-4b21-bcf2-a4afb82cdbd2-secret-metrics-server-client-certs\") pod \"metrics-server-7578c5ccf6-wpp2l\" (UID: \"69415242-9c1a-4b21-bcf2-a4afb82cdbd2\") " pod="openshift-monitoring/metrics-server-7578c5ccf6-wpp2l"
Sep 10 19:17:07.258015 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:07.257969    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/69415242-9c1a-4b21-bcf2-a4afb82cdbd2-audit-log\") pod \"metrics-server-7578c5ccf6-wpp2l\" (UID: \"69415242-9c1a-4b21-bcf2-a4afb82cdbd2\") " pod="openshift-monitoring/metrics-server-7578c5ccf6-wpp2l"
Sep 10 19:17:07.258838 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:07.258796    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69415242-9c1a-4b21-bcf2-a4afb82cdbd2-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7578c5ccf6-wpp2l\" (UID: \"69415242-9c1a-4b21-bcf2-a4afb82cdbd2\") " pod="openshift-monitoring/metrics-server-7578c5ccf6-wpp2l"
Sep 10 19:17:07.259173 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:07.259153    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/69415242-9c1a-4b21-bcf2-a4afb82cdbd2-metrics-server-audit-profiles\") pod \"metrics-server-7578c5ccf6-wpp2l\" (UID: \"69415242-9c1a-4b21-bcf2-a4afb82cdbd2\") " pod="openshift-monitoring/metrics-server-7578c5ccf6-wpp2l"
Sep 10 19:17:07.260565 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:07.260544    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/69415242-9c1a-4b21-bcf2-a4afb82cdbd2-secret-metrics-server-tls\") pod \"metrics-server-7578c5ccf6-wpp2l\" (UID: \"69415242-9c1a-4b21-bcf2-a4afb82cdbd2\") " pod="openshift-monitoring/metrics-server-7578c5ccf6-wpp2l"
Sep 10 19:17:07.260796 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:07.260777    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/69415242-9c1a-4b21-bcf2-a4afb82cdbd2-client-ca-bundle\") pod \"metrics-server-7578c5ccf6-wpp2l\" (UID: \"69415242-9c1a-4b21-bcf2-a4afb82cdbd2\") " pod="openshift-monitoring/metrics-server-7578c5ccf6-wpp2l"
Sep 10 19:17:07.260843 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:07.260777    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/69415242-9c1a-4b21-bcf2-a4afb82cdbd2-secret-metrics-server-client-certs\") pod \"metrics-server-7578c5ccf6-wpp2l\" (UID: \"69415242-9c1a-4b21-bcf2-a4afb82cdbd2\") " pod="openshift-monitoring/metrics-server-7578c5ccf6-wpp2l"
Sep 10 19:17:07.266711 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:07.266686    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgbsh\" (UniqueName: \"kubernetes.io/projected/69415242-9c1a-4b21-bcf2-a4afb82cdbd2-kube-api-access-bgbsh\") pod \"metrics-server-7578c5ccf6-wpp2l\" (UID: \"69415242-9c1a-4b21-bcf2-a4afb82cdbd2\") " pod="openshift-monitoring/metrics-server-7578c5ccf6-wpp2l"
Sep 10 19:17:07.320205 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:07.320127    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-qgvtq" event={"ID":"eeec0ca3-75b9-4590-a29e-609fa17b7e7e","Type":"ContainerStarted","Data":"69e063228a2aefa06163e3c2fef1c63ede8092542fc228279c880bb460af0a79"}
Sep 10 19:17:07.323860 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:07.323823    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-k72xf" event={"ID":"ee547a46-3813-4faa-a7bd-7e05be04e865","Type":"ContainerStarted","Data":"3e028bdda5e22fb671a30733c369a21abda9ec4fd0465d5802e16de62f149e73"}
Sep 10 19:17:07.323954 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:07.323933    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-k72xf" event={"ID":"ee547a46-3813-4faa-a7bd-7e05be04e865","Type":"ContainerStarted","Data":"1ec106da9abdcdb338949cccbcd2b703e88ea97bc49a9ab5a33c07460045774c"}
Sep 10 19:17:07.344042 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:07.343985    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-k72xf" podStartSLOduration=2.3495262869999998 podStartE2EDuration="5.343965058s" totalImagesPullingTime="2.918062735s" totalInitContainerRuntime="76.376036ms" isStatefulPod=true podCreationTimestamp="2026-09-10 19:17:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:17:07.343762116 +0000 UTC m=+82.944669358" watchObservedRunningTime="2026-09-10 19:17:07.343965058 +0000 UTC m=+82.944872301"
Sep 10 19:17:07.392583 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:07.392360    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7578c5ccf6-wpp2l"
Sep 10 19:17:07.431053 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:07.430958    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:07.527464 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:07.527397    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:07.642598 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:07.642556    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:07.703916 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:07.703708    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7578c5ccf6-wpp2l"]
Sep 10 19:17:07.706865 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:17:07.706833    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69415242_9c1a_4b21_bcf2_a4afb82cdbd2.slice/crio-b782ffe24096ab3dbd4e905e9253b99c4375b9f73867f576a0d3bcad136d0dd6 WatchSource:0}: Error finding container b782ffe24096ab3dbd4e905e9253b99c4375b9f73867f576a0d3bcad136d0dd6: Status 404 returned error can't find the container with id b782ffe24096ab3dbd4e905e9253b99c4375b9f73867f576a0d3bcad136d0dd6
Sep 10 19:17:08.276098 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:08.276070    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-67nsx"
Sep 10 19:17:08.330123 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:08.330084    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7578c5ccf6-wpp2l" event={"ID":"69415242-9c1a-4b21-bcf2-a4afb82cdbd2","Type":"ContainerStarted","Data":"b782ffe24096ab3dbd4e905e9253b99c4375b9f73867f576a0d3bcad136d0dd6"}
Sep 10 19:17:08.333277 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:08.333186    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-qgvtq" event={"ID":"eeec0ca3-75b9-4590-a29e-609fa17b7e7e","Type":"ContainerStarted","Data":"bd0c82420db0f5d2e904521b97090ac86e37b7976a8287f57820a8054d6ffa34"}
Sep 10 19:17:08.333277 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:08.333239    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-qgvtq" event={"ID":"eeec0ca3-75b9-4590-a29e-609fa17b7e7e","Type":"ContainerStarted","Data":"fa8291f726d4af42391750b11e527e3a5caa5bc02c357ddcb1e4b44c428ff7f3"}
Sep 10 19:17:08.352950 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:08.352888    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-74cb57df7-qgvtq" podStartSLOduration=4.657455968 podStartE2EDuration="6.352866379s" totalImagesPullingTime="1.695410411s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:17:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:17:08.352186827 +0000 UTC m=+83.953094069" watchObservedRunningTime="2026-09-10 19:17:08.352866379 +0000 UTC m=+83.953773623"
Sep 10 19:17:08.931004 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:08.930967    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:08.936256 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:08.936214    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 19:17:08.944097 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:08.943919    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:08.946497 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:08.946472    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 10 19:17:08.946497 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:08.946484    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 10 19:17:08.946816 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:08.946794    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 10 19:17:08.946948 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:08.946920    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 10 19:17:08.947094 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:08.946795    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-xd7q5\""
Sep 10 19:17:08.947847 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:08.947829    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-2alcjquntm8be\""
Sep 10 19:17:08.948098 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:08.948080    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 10 19:17:08.948649 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:08.948296    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 10 19:17:08.948649 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:08.948351    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 10 19:17:08.948649 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:08.948495    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 10 19:17:08.948649 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:08.948507    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 10 19:17:08.948873 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:08.948732    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 10 19:17:08.949400 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:08.949382    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 10 19:17:08.950121 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:08.950057    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 10 19:17:08.951818 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:08.951797    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 10 19:17:08.954318 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:08.954145    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 10 19:17:08.962083 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:08.962058    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 19:17:09.078446 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:09.078169    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:09.078446 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:09.078221    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:09.078446 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:09.078252    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-config\") pod \"prometheus-k8s-0\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:09.078446 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:09.078274    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:09.078446 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:09.078304    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-config-out\") pod \"prometheus-k8s-0\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:09.078446 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:09.078326    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:09.078446 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:09.078346    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:09.078446 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:09.078381    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-web-config\") pod \"prometheus-k8s-0\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:09.078446 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:09.078427    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:09.078446 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:09.078457    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:09.079198 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:09.078526    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnb4r\" (UniqueName: \"kubernetes.io/projected/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-kube-api-access-lnb4r\") pod \"prometheus-k8s-0\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:09.079198 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:09.078617    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:09.079198 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:09.078681    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:09.079198 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:09.078708    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:09.079198 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:09.078738    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:09.079198 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:09.078769    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:09.079198 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:09.078796    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:09.079198 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:09.078851    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:09.079198 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:09.078923    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:09.079198 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:09.078959    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:09.180172 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:09.180141    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:09.180369 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:09.180184    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:09.180730 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:09.180692    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:09.180850 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:09.180749    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:09.180850 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:09.180783    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:09.180850 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:09.180818    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-config\") pod \"prometheus-k8s-0\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:09.180850 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:09.180847    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:09.181083 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:09.180883    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-config-out\") pod \"prometheus-k8s-0\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:09.181083 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:09.180908    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:09.181083 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:09.180929    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:09.181083 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:09.180978    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:09.181083 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:09.180936    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:09.181083 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:09.181035    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-web-config\") pod \"prometheus-k8s-0\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:09.181083 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:09.181066    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:09.181413 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:09.181099    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:09.181413 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:09.181125    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnb4r\" (UniqueName: \"kubernetes.io/projected/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-kube-api-access-lnb4r\") pod \"prometheus-k8s-0\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:09.181413 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:09.181201    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:09.181413 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:09.181238    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:09.181413 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:09.181255    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:09.181413 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:09.181265    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:09.181413 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:09.181313    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:09.181413 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:09.181352    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:09.181413 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:09.181389    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:09.181874 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:09.181507    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:09.183870 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:09.183838    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:09.184006 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:09.183988    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:09.184932 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:09.184636    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:09.184932 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:09.184719    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:09.184932 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:09.184904    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-web-config\") pod \"prometheus-k8s-0\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:09.185184 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:09.185142    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-config-out\") pod \"prometheus-k8s-0\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:09.185363 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:09.185343    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:09.186442 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:09.186408    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-config\") pod \"prometheus-k8s-0\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:09.186615 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:09.186588    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:09.189243 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:09.187519    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:09.189243 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:09.188918    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:09.189524 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:09.189502    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:09.190486 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:09.190458    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:09.191168 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:09.191125    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:09.191299 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:09.191277    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:09.191840 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:09.191820    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnb4r\" (UniqueName: \"kubernetes.io/projected/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-kube-api-access-lnb4r\") pod \"prometheus-k8s-0\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:09.259166 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:09.259127    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:10.241637 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:10.241580    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:10.294859 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:10.294824    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:11.500612 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:11.500576    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7f94744dd8-k8jg5"
Sep 10 19:17:11.501047 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:11.500658    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7f94744dd8-k8jg5"
Sep 10 19:17:11.506374 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:11.506347    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7f94744dd8-k8jg5"
Sep 10 19:17:12.352726 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:12.352694    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7f94744dd8-k8jg5"
Sep 10 19:17:12.789491 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:12.789457    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:12.858095 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:12.858049    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:13.227357 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:13.227258    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:13.291693 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:13.291655    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:14.185328 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:14.185290    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:14.241976 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:14.241945    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6cbff68f97-mt7bn"
Sep 10 19:17:14.244242 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:14.244194    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:15.243877 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:15.243847    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-54895dc98-2h4rn"
Sep 10 19:17:15.386797 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:15.386758    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:15.441698 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:15.441653    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:16.195614 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:16.195588    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 19:17:16.248589 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:17:16.248553    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3dbc922_35d8_4cb0_a435_9b61a2ca7f3a.slice/crio-bd6c9d6c49ee0f2220d097eacec16a5fc8075eef57a021b00aaee4caecaca9a9 WatchSource:0}: Error finding container bd6c9d6c49ee0f2220d097eacec16a5fc8075eef57a021b00aaee4caecaca9a9: Status 404 returned error can't find the container with id bd6c9d6c49ee0f2220d097eacec16a5fc8075eef57a021b00aaee4caecaca9a9
Sep 10 19:17:16.363555 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:16.363530    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a","Type":"ContainerStarted","Data":"bd6c9d6c49ee0f2220d097eacec16a5fc8075eef57a021b00aaee4caecaca9a9"}
Sep 10 19:17:17.206245 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:17.206129    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:17.313845 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:17.313783    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:17.370926 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:17.370888    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-ssx24" event={"ID":"ec666f15-107e-4574-9d6c-cc73b6840c2e","Type":"ContainerStarted","Data":"9860ead05dc7eec71bf972cc2faedfa0ba364751cd262b51e4b9dd0498c6f4d7"}
Sep 10 19:17:17.371470 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:17.371414    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-85d98c998f-ssx24"
Sep 10 19:17:17.381351 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:17.381299    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-85d98c998f-ssx24"
Sep 10 19:17:17.393012 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:17.391870    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-85d98c998f-ssx24" podStartSLOduration=1.868473749 podStartE2EDuration="24.391853317s" totalImagesPullingTime="22.523379568s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:16:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:17:17.390464095 +0000 UTC m=+92.991371335" watchObservedRunningTime="2026-09-10 19:17:17.391853317 +0000 UTC m=+92.992760558"
Sep 10 19:17:17.787799 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:17.787763    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:18.898583 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:18.898546    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:18.953505 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:18.953462    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:19.257786 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:19.257741    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6cbff68f97-mt7bn" podUID="0c459a94-07c5-44fb-b155-8a8f7aa46d25" containerName="registry" containerID="cri-o://a7d5ef2f6dc507b03bfe91d8d98d928a758cec069f034ac1fb5c5e1a40048944" gracePeriod=30
Sep 10 19:17:19.381453 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:19.381425    2564 generic.go:356] "Generic (PLEG): container finished" podID="0c459a94-07c5-44fb-b155-8a8f7aa46d25" containerID="a7d5ef2f6dc507b03bfe91d8d98d928a758cec069f034ac1fb5c5e1a40048944" exitCode=0
Sep 10 19:17:19.381615 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:19.381512    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6cbff68f97-mt7bn" event={"ID":"0c459a94-07c5-44fb-b155-8a8f7aa46d25","Type":"ContainerDied","Data":"a7d5ef2f6dc507b03bfe91d8d98d928a758cec069f034ac1fb5c5e1a40048944"}
Sep 10 19:17:19.383780 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:19.383528    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7578c5ccf6-wpp2l" event={"ID":"69415242-9c1a-4b21-bcf2-a4afb82cdbd2","Type":"ContainerStarted","Data":"4b530ee9980ffd7bd2d2932e2fef87842bb23ce359b6389ab0d582c9f2642818"}
Sep 10 19:17:19.385705 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:19.385673    2564 generic.go:356] "Generic (PLEG): container finished" podID="f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a" containerID="587658c07f740026e6cac9858f8b0fc15d119726bc1fa33486d85243916373c1" exitCode=0
Sep 10 19:17:19.385927 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:19.385823    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a","Type":"ContainerDied","Data":"587658c07f740026e6cac9858f8b0fc15d119726bc1fa33486d85243916373c1"}
Sep 10 19:17:19.403142 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:19.402836    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-7578c5ccf6-wpp2l" podStartSLOduration=1.702749548 podStartE2EDuration="12.402815167s" totalImagesPullingTime="10.700065619s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:17:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:17:19.402527821 +0000 UTC m=+95.003435063" watchObservedRunningTime="2026-09-10 19:17:19.402815167 +0000 UTC m=+95.003722409"
Sep 10 19:17:19.566801 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:19.566769    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cbff68f97-mt7bn"
Sep 10 19:17:19.684145 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:19.684113    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0c459a94-07c5-44fb-b155-8a8f7aa46d25-image-registry-private-configuration\") pod \"0c459a94-07c5-44fb-b155-8a8f7aa46d25\" (UID: \"0c459a94-07c5-44fb-b155-8a8f7aa46d25\") "
Sep 10 19:17:19.684421 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:19.684400    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0c459a94-07c5-44fb-b155-8a8f7aa46d25-ca-trust-extracted\") pod \"0c459a94-07c5-44fb-b155-8a8f7aa46d25\" (UID: \"0c459a94-07c5-44fb-b155-8a8f7aa46d25\") "
Sep 10 19:17:19.684559 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:19.684547    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0c459a94-07c5-44fb-b155-8a8f7aa46d25-registry-certificates\") pod \"0c459a94-07c5-44fb-b155-8a8f7aa46d25\" (UID: \"0c459a94-07c5-44fb-b155-8a8f7aa46d25\") "
Sep 10 19:17:19.684697 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:19.684681    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0c459a94-07c5-44fb-b155-8a8f7aa46d25-bound-sa-token\") pod \"0c459a94-07c5-44fb-b155-8a8f7aa46d25\" (UID: \"0c459a94-07c5-44fb-b155-8a8f7aa46d25\") "
Sep 10 19:17:19.685858 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:19.685726    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c459a94-07c5-44fb-b155-8a8f7aa46d25-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "0c459a94-07c5-44fb-b155-8a8f7aa46d25" (UID: "0c459a94-07c5-44fb-b155-8a8f7aa46d25"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:17:19.685978 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:19.685873    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0c459a94-07c5-44fb-b155-8a8f7aa46d25-registry-tls\") pod \"0c459a94-07c5-44fb-b155-8a8f7aa46d25\" (UID: \"0c459a94-07c5-44fb-b155-8a8f7aa46d25\") "
Sep 10 19:17:19.685978 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:19.685936    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q2vg4\" (UniqueName: \"kubernetes.io/projected/0c459a94-07c5-44fb-b155-8a8f7aa46d25-kube-api-access-q2vg4\") pod \"0c459a94-07c5-44fb-b155-8a8f7aa46d25\" (UID: \"0c459a94-07c5-44fb-b155-8a8f7aa46d25\") "
Sep 10 19:17:19.685978 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:19.685964    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0c459a94-07c5-44fb-b155-8a8f7aa46d25-trusted-ca\") pod \"0c459a94-07c5-44fb-b155-8a8f7aa46d25\" (UID: \"0c459a94-07c5-44fb-b155-8a8f7aa46d25\") "
Sep 10 19:17:19.686134 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:19.686028    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0c459a94-07c5-44fb-b155-8a8f7aa46d25-installation-pull-secrets\") pod \"0c459a94-07c5-44fb-b155-8a8f7aa46d25\" (UID: \"0c459a94-07c5-44fb-b155-8a8f7aa46d25\") "
Sep 10 19:17:19.686299 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:19.686275    2564 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0c459a94-07c5-44fb-b155-8a8f7aa46d25-registry-certificates\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:17:19.687938 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:19.687904    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0c459a94-07c5-44fb-b155-8a8f7aa46d25-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "0c459a94-07c5-44fb-b155-8a8f7aa46d25" (UID: "0c459a94-07c5-44fb-b155-8a8f7aa46d25"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:17:19.688881 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:19.688850    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c459a94-07c5-44fb-b155-8a8f7aa46d25-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "0c459a94-07c5-44fb-b155-8a8f7aa46d25" (UID: "0c459a94-07c5-44fb-b155-8a8f7aa46d25"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:17:19.689102 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:19.689079    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c459a94-07c5-44fb-b155-8a8f7aa46d25-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "0c459a94-07c5-44fb-b155-8a8f7aa46d25" (UID: "0c459a94-07c5-44fb-b155-8a8f7aa46d25"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:17:19.689692 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:19.689596    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0c459a94-07c5-44fb-b155-8a8f7aa46d25-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "0c459a94-07c5-44fb-b155-8a8f7aa46d25" (UID: "0c459a94-07c5-44fb-b155-8a8f7aa46d25"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:17:19.689692 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:19.689607    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c459a94-07c5-44fb-b155-8a8f7aa46d25-kube-api-access-q2vg4" (OuterVolumeSpecName: "kube-api-access-q2vg4") pod "0c459a94-07c5-44fb-b155-8a8f7aa46d25" (UID: "0c459a94-07c5-44fb-b155-8a8f7aa46d25"). InnerVolumeSpecName "kube-api-access-q2vg4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:17:19.689692 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:19.689684    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c459a94-07c5-44fb-b155-8a8f7aa46d25-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "0c459a94-07c5-44fb-b155-8a8f7aa46d25" (UID: "0c459a94-07c5-44fb-b155-8a8f7aa46d25"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:17:19.696071 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:19.696015    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c459a94-07c5-44fb-b155-8a8f7aa46d25-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "0c459a94-07c5-44fb-b155-8a8f7aa46d25" (UID: "0c459a94-07c5-44fb-b155-8a8f7aa46d25"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:17:19.787742 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:19.787667    2564 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0c459a94-07c5-44fb-b155-8a8f7aa46d25-bound-sa-token\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:17:19.787742 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:19.787703    2564 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0c459a94-07c5-44fb-b155-8a8f7aa46d25-registry-tls\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:17:19.787742 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:19.787719    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q2vg4\" (UniqueName: \"kubernetes.io/projected/0c459a94-07c5-44fb-b155-8a8f7aa46d25-kube-api-access-q2vg4\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:17:19.787742 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:19.787733    2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0c459a94-07c5-44fb-b155-8a8f7aa46d25-trusted-ca\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:17:19.787742 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:19.787747    2564 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0c459a94-07c5-44fb-b155-8a8f7aa46d25-installation-pull-secrets\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:17:19.788035 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:19.787764    2564 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0c459a94-07c5-44fb-b155-8a8f7aa46d25-image-registry-private-configuration\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:17:19.788035 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:19.787777    2564 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0c459a94-07c5-44fb-b155-8a8f7aa46d25-ca-trust-extracted\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:17:20.001422 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:20.001383    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:20.081278 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:20.081197    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:20.145410 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:20.145369    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:20.246454 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:20.246420    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:20.394365 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:20.394268    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cbff68f97-mt7bn"
Sep 10 19:17:20.394636 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:20.394265    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6cbff68f97-mt7bn" event={"ID":"0c459a94-07c5-44fb-b155-8a8f7aa46d25","Type":"ContainerDied","Data":"7b5dbac6890443ab1aa2a1ee02d6c7080c1159a40fc56cfb0398389a32dcd727"}
Sep 10 19:17:20.395288 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:20.395258    2564 scope.go:117] "RemoveContainer" containerID="a7d5ef2f6dc507b03bfe91d8d98d928a758cec069f034ac1fb5c5e1a40048944"
Sep 10 19:17:20.429732 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:20.429674    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6cbff68f97-mt7bn"]
Sep 10 19:17:20.433419 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:20.433333    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6cbff68f97-mt7bn"]
Sep 10 19:17:20.949636 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:20.949590    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c459a94-07c5-44fb-b155-8a8f7aa46d25" path="/var/lib/kubelet/pods/0c459a94-07c5-44fb-b155-8a8f7aa46d25/volumes"
Sep 10 19:17:21.305063 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:21.304916    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7f94744dd8-k8jg5"]
Sep 10 19:17:22.018715 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:22.018677    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:22.264639 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:22.264593    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:24.416678 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:24.416595    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a","Type":"ContainerStarted","Data":"ab5a5945bd22dd2c7affd8044765602cbcba8ac88e49e6674edad189da741df6"}
Sep 10 19:17:24.417038 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:24.416737    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a","Type":"ContainerStarted","Data":"423ab8ab659fb962c232769b6954c08bb47d5a49477c475f9100b7d7e0f0d4fc"}
Sep 10 19:17:26.119702 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:26.119663    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:26.175801 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:26.175734    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:27.393156 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:27.393072    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-7578c5ccf6-wpp2l"
Sep 10 19:17:27.393533 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:27.393161    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-7578c5ccf6-wpp2l"
Sep 10 19:17:27.432097 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:27.432064    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a","Type":"ContainerStarted","Data":"fd61e04df64a2ab106f9c39884abb3fd65b0aa621f809e09dba4ee49d6e7266f"}
Sep 10 19:17:27.432271 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:27.432123    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a","Type":"ContainerStarted","Data":"1c2dd76d55247af36a5f7feab7a6bb31e482ea77ce6b0a61ef597647f032b381"}
Sep 10 19:17:27.432271 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:27.432154    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a","Type":"ContainerStarted","Data":"6fe837d0189b954b19f649a4c5a5ae0553778f0dd1132df3d27fa9f944d14813"}
Sep 10 19:17:27.432271 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:27.432178    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a","Type":"ContainerStarted","Data":"afaf414825f92c83d6de30df05d873b6d29b4255da22b69c4a6916bb63a5f18d"}
Sep 10 19:17:27.462883 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:27.462829    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=10.190636161 podStartE2EDuration="19.462806974s" totalImagesPullingTime="9.247720361s" totalInitContainerRuntime="24.450452ms" isStatefulPod=false podCreationTimestamp="2026-09-10 19:17:08 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:17:27.461979974 +0000 UTC m=+103.062887206" watchObservedRunningTime="2026-09-10 19:17:27.462806974 +0000 UTC m=+103.063714218"
Sep 10 19:17:29.260274 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:29.260238    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:29.275735 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:29.275706    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-9stjj"
Sep 10 19:17:31.246753 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:31.246716    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:31.348253 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:31.348212    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:35.005360 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:35.005306    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:35.058788 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:35.058739    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:46.336581 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:46.336482    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7f94744dd8-k8jg5" podUID="13aa0d02-73a3-4f27-993b-46ab70dbed06" containerName="console" containerID="cri-o://f90abf4c46e09038abcaf4083b3eaedda276893fad997ebe301cb2c6f9d5c7b4" gracePeriod=15
Sep 10 19:17:46.512746 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:46.512715    2564 generic.go:356] "Generic (PLEG): container finished" podID="13aa0d02-73a3-4f27-993b-46ab70dbed06" containerID="f90abf4c46e09038abcaf4083b3eaedda276893fad997ebe301cb2c6f9d5c7b4" exitCode=0
Sep 10 19:17:46.512898 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:46.512780    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f94744dd8-k8jg5" event={"ID":"13aa0d02-73a3-4f27-993b-46ab70dbed06","Type":"ContainerDied","Data":"f90abf4c46e09038abcaf4083b3eaedda276893fad997ebe301cb2c6f9d5c7b4"}
Sep 10 19:17:46.649286 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:46.649241    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f94744dd8-k8jg5"
Sep 10 19:17:46.723752 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:46.723718    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/13aa0d02-73a3-4f27-993b-46ab70dbed06-console-serving-cert\") pod \"13aa0d02-73a3-4f27-993b-46ab70dbed06\" (UID: \"13aa0d02-73a3-4f27-993b-46ab70dbed06\") "
Sep 10 19:17:46.723920 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:46.723783    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/13aa0d02-73a3-4f27-993b-46ab70dbed06-oauth-serving-cert\") pod \"13aa0d02-73a3-4f27-993b-46ab70dbed06\" (UID: \"13aa0d02-73a3-4f27-993b-46ab70dbed06\") "
Sep 10 19:17:46.723920 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:46.723802    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/13aa0d02-73a3-4f27-993b-46ab70dbed06-console-oauth-config\") pod \"13aa0d02-73a3-4f27-993b-46ab70dbed06\" (UID: \"13aa0d02-73a3-4f27-993b-46ab70dbed06\") "
Sep 10 19:17:46.723920 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:46.723851    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/13aa0d02-73a3-4f27-993b-46ab70dbed06-service-ca\") pod \"13aa0d02-73a3-4f27-993b-46ab70dbed06\" (UID: \"13aa0d02-73a3-4f27-993b-46ab70dbed06\") "
Sep 10 19:17:46.724096 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:46.723993    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bn2x7\" (UniqueName: \"kubernetes.io/projected/13aa0d02-73a3-4f27-993b-46ab70dbed06-kube-api-access-bn2x7\") pod \"13aa0d02-73a3-4f27-993b-46ab70dbed06\" (UID: \"13aa0d02-73a3-4f27-993b-46ab70dbed06\") "
Sep 10 19:17:46.724096 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:46.724066    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/13aa0d02-73a3-4f27-993b-46ab70dbed06-console-config\") pod \"13aa0d02-73a3-4f27-993b-46ab70dbed06\" (UID: \"13aa0d02-73a3-4f27-993b-46ab70dbed06\") "
Sep 10 19:17:46.724235 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:46.724176    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13aa0d02-73a3-4f27-993b-46ab70dbed06-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "13aa0d02-73a3-4f27-993b-46ab70dbed06" (UID: "13aa0d02-73a3-4f27-993b-46ab70dbed06"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:17:46.724235 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:46.724191    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13aa0d02-73a3-4f27-993b-46ab70dbed06-service-ca" (OuterVolumeSpecName: "service-ca") pod "13aa0d02-73a3-4f27-993b-46ab70dbed06" (UID: "13aa0d02-73a3-4f27-993b-46ab70dbed06"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:17:46.724403 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:46.724377    2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/13aa0d02-73a3-4f27-993b-46ab70dbed06-oauth-serving-cert\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:17:46.724459 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:46.724409    2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/13aa0d02-73a3-4f27-993b-46ab70dbed06-service-ca\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:17:46.724548 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:46.724507    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13aa0d02-73a3-4f27-993b-46ab70dbed06-console-config" (OuterVolumeSpecName: "console-config") pod "13aa0d02-73a3-4f27-993b-46ab70dbed06" (UID: "13aa0d02-73a3-4f27-993b-46ab70dbed06"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:17:46.726222 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:46.726187    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/13aa0d02-73a3-4f27-993b-46ab70dbed06-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "13aa0d02-73a3-4f27-993b-46ab70dbed06" (UID: "13aa0d02-73a3-4f27-993b-46ab70dbed06"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:17:46.726498 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:46.726468    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13aa0d02-73a3-4f27-993b-46ab70dbed06-kube-api-access-bn2x7" (OuterVolumeSpecName: "kube-api-access-bn2x7") pod "13aa0d02-73a3-4f27-993b-46ab70dbed06" (UID: "13aa0d02-73a3-4f27-993b-46ab70dbed06"). InnerVolumeSpecName "kube-api-access-bn2x7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:17:46.726551 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:46.726510    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/13aa0d02-73a3-4f27-993b-46ab70dbed06-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "13aa0d02-73a3-4f27-993b-46ab70dbed06" (UID: "13aa0d02-73a3-4f27-993b-46ab70dbed06"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:17:46.824937 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:46.824902    2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/13aa0d02-73a3-4f27-993b-46ab70dbed06-console-serving-cert\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:17:46.824937 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:46.824930    2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/13aa0d02-73a3-4f27-993b-46ab70dbed06-console-oauth-config\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:17:46.824937 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:46.824940    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bn2x7\" (UniqueName: \"kubernetes.io/projected/13aa0d02-73a3-4f27-993b-46ab70dbed06-kube-api-access-bn2x7\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:17:46.824937 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:46.824949    2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/13aa0d02-73a3-4f27-993b-46ab70dbed06-console-config\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:17:47.400347 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:47.400317    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-7578c5ccf6-wpp2l"
Sep 10 19:17:47.404576 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:47.404547    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-7578c5ccf6-wpp2l"
Sep 10 19:17:47.518510 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:47.518484    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f94744dd8-k8jg5"
Sep 10 19:17:47.518711 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:47.518523    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f94744dd8-k8jg5" event={"ID":"13aa0d02-73a3-4f27-993b-46ab70dbed06","Type":"ContainerDied","Data":"ed4622632dbd3ca54e3717c6495f5813c69146d521a54370ca09155bed1bf869"}
Sep 10 19:17:47.518711 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:47.518579    2564 scope.go:117] "RemoveContainer" containerID="f90abf4c46e09038abcaf4083b3eaedda276893fad997ebe301cb2c6f9d5c7b4"
Sep 10 19:17:47.537834 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:47.537807    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7f94744dd8-k8jg5"]
Sep 10 19:17:47.542681 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:47.542660    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7f94744dd8-k8jg5"]
Sep 10 19:17:48.952091 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:48.952056    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13aa0d02-73a3-4f27-993b-46ab70dbed06" path="/var/lib/kubelet/pods/13aa0d02-73a3-4f27-993b-46ab70dbed06/volumes"
Sep 10 19:17:50.018854 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:50.018823    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:50.074327 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:50.074291    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:57.560966 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:17:57.560923    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:18:09.259526 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:09.259483    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:09.278011 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:09.277984    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:09.619841 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:09.619766    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:21.950611 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:21.950573    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:18:22.049197 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:22.049164    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:18:27.349172 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.349128    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 19:18:27.349563 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.349529    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a" containerName="prometheus" containerID="cri-o://423ab8ab659fb962c232769b6954c08bb47d5a49477c475f9100b7d7e0f0d4fc" gracePeriod=600
Sep 10 19:18:27.349667 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.349549    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a" containerName="kube-rbac-proxy-thanos" containerID="cri-o://fd61e04df64a2ab106f9c39884abb3fd65b0aa621f809e09dba4ee49d6e7266f" gracePeriod=600
Sep 10 19:18:27.349667 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.349568    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a" containerName="kube-rbac-proxy-web" containerID="cri-o://6fe837d0189b954b19f649a4c5a5ae0553778f0dd1132df3d27fa9f944d14813" gracePeriod=600
Sep 10 19:18:27.349667 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.349603    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a" containerName="thanos-sidecar" containerID="cri-o://afaf414825f92c83d6de30df05d873b6d29b4255da22b69c4a6916bb63a5f18d" gracePeriod=600
Sep 10 19:18:27.349841 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.349698    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a" containerName="config-reloader" containerID="cri-o://ab5a5945bd22dd2c7affd8044765602cbcba8ac88e49e6674edad189da741df6" gracePeriod=600
Sep 10 19:18:27.349841 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.349689    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a" containerName="kube-rbac-proxy" containerID="cri-o://1c2dd76d55247af36a5f7feab7a6bb31e482ea77ce6b0a61ef597647f032b381" gracePeriod=600
Sep 10 19:18:27.601753 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.601690    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:27.650731 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.650693    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-config\") pod \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") "
Sep 10 19:18:27.650926 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.650740    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-prometheus-trusted-ca-bundle\") pod \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") "
Sep 10 19:18:27.650926 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.650779    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-prometheus-k8s-rulefiles-0\") pod \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") "
Sep 10 19:18:27.650926 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.650827    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-prometheus-k8s-rulefiles-2\") pod \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") "
Sep 10 19:18:27.650926 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.650857    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-configmap-serving-certs-ca-bundle\") pod \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") "
Sep 10 19:18:27.651155 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.651005    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") "
Sep 10 19:18:27.651155 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.651049    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-prometheus-k8s-rulefiles-1\") pod \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") "
Sep 10 19:18:27.651155 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.651078    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-tls-assets\") pod \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") "
Sep 10 19:18:27.651155 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.651129    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") "
Sep 10 19:18:27.651155 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.651154    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-web-config\") pod \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") "
Sep 10 19:18:27.651378 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.651180    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-secret-prometheus-k8s-tls\") pod \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") "
Sep 10 19:18:27.651378 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.651203    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-thanos-prometheus-http-client-file\") pod \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") "
Sep 10 19:18:27.651378 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.651234    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-secret-kube-rbac-proxy\") pod \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") "
Sep 10 19:18:27.651378 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.651260    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-secret-metrics-client-certs\") pod \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") "
Sep 10 19:18:27.651378 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.651283    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lnb4r\" (UniqueName: \"kubernetes.io/projected/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-kube-api-access-lnb4r\") pod \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") "
Sep 10 19:18:27.651378 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.651317    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-configmap-kubelet-serving-ca-bundle\") pod \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") "
Sep 10 19:18:27.651378 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.651325    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a" (UID: "f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:18:27.651378 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.651356    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-secret-grpc-tls\") pod \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") "
Sep 10 19:18:27.651853 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.651387    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-config-out\") pod \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") "
Sep 10 19:18:27.651853 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.651418    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-configmap-metrics-client-ca\") pod \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") "
Sep 10 19:18:27.651853 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.651442    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-prometheus-k8s-db\") pod \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\" (UID: \"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a\") "
Sep 10 19:18:27.651853 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.651726    2564 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:18:27.652063 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.651899    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a" (UID: "f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:18:27.652063 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.652020    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a" (UID: "f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:18:27.652506 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.652272    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a" (UID: "f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:18:27.652917 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.652889    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a" (UID: "f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:18:27.653020 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.652987    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a" (UID: "f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:18:27.654230 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.653998    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a" (UID: "f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:18:27.654515 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.654479    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a" (UID: "f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:18:27.655255 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.655220    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a" (UID: "f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:18:27.655786 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.655760    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-config" (OuterVolumeSpecName: "config") pod "f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a" (UID: "f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:18:27.656145 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.656122    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a" (UID: "f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:18:27.657224 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.657145    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a" (UID: "f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:18:27.657224 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.657177    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-kube-api-access-lnb4r" (OuterVolumeSpecName: "kube-api-access-lnb4r") pod "f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a" (UID: "f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a"). InnerVolumeSpecName "kube-api-access-lnb4r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:18:27.657224 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.657203    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a" (UID: "f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:18:27.657416 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.657236    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a" (UID: "f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:18:27.657877 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.657843    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a" (UID: "f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:18:27.657969 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.657951    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a" (UID: "f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:18:27.658946 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.658919    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-config-out" (OuterVolumeSpecName: "config-out") pod "f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a" (UID: "f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:18:27.659035 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.658928    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a" (UID: "f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:18:27.673839 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.673164    2564 generic.go:356] "Generic (PLEG): container finished" podID="f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a" containerID="fd61e04df64a2ab106f9c39884abb3fd65b0aa621f809e09dba4ee49d6e7266f" exitCode=0
Sep 10 19:18:27.673839 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.673193    2564 generic.go:356] "Generic (PLEG): container finished" podID="f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a" containerID="1c2dd76d55247af36a5f7feab7a6bb31e482ea77ce6b0a61ef597647f032b381" exitCode=0
Sep 10 19:18:27.673839 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.673204    2564 generic.go:356] "Generic (PLEG): container finished" podID="f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a" containerID="6fe837d0189b954b19f649a4c5a5ae0553778f0dd1132df3d27fa9f944d14813" exitCode=0
Sep 10 19:18:27.673839 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.673213    2564 generic.go:356] "Generic (PLEG): container finished" podID="f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a" containerID="afaf414825f92c83d6de30df05d873b6d29b4255da22b69c4a6916bb63a5f18d" exitCode=0
Sep 10 19:18:27.673839 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.673220    2564 generic.go:356] "Generic (PLEG): container finished" podID="f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a" containerID="ab5a5945bd22dd2c7affd8044765602cbcba8ac88e49e6674edad189da741df6" exitCode=0
Sep 10 19:18:27.673839 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.673228    2564 generic.go:356] "Generic (PLEG): container finished" podID="f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a" containerID="423ab8ab659fb962c232769b6954c08bb47d5a49477c475f9100b7d7e0f0d4fc" exitCode=0
Sep 10 19:18:27.673839 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.673274    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:27.673839 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.673293    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a","Type":"ContainerDied","Data":"fd61e04df64a2ab106f9c39884abb3fd65b0aa621f809e09dba4ee49d6e7266f"}
Sep 10 19:18:27.673839 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.673365    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a","Type":"ContainerDied","Data":"1c2dd76d55247af36a5f7feab7a6bb31e482ea77ce6b0a61ef597647f032b381"}
Sep 10 19:18:27.673839 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.673409    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a","Type":"ContainerDied","Data":"6fe837d0189b954b19f649a4c5a5ae0553778f0dd1132df3d27fa9f944d14813"}
Sep 10 19:18:27.673839 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.673451    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a","Type":"ContainerDied","Data":"afaf414825f92c83d6de30df05d873b6d29b4255da22b69c4a6916bb63a5f18d"}
Sep 10 19:18:27.673839 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.673492    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a","Type":"ContainerDied","Data":"ab5a5945bd22dd2c7affd8044765602cbcba8ac88e49e6674edad189da741df6"}
Sep 10 19:18:27.673839 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.673534    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a","Type":"ContainerDied","Data":"423ab8ab659fb962c232769b6954c08bb47d5a49477c475f9100b7d7e0f0d4fc"}
Sep 10 19:18:27.673839 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.673574    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a","Type":"ContainerDied","Data":"bd6c9d6c49ee0f2220d097eacec16a5fc8075eef57a021b00aaee4caecaca9a9"}
Sep 10 19:18:27.673839 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.673643    2564 scope.go:117] "RemoveContainer" containerID="fd61e04df64a2ab106f9c39884abb3fd65b0aa621f809e09dba4ee49d6e7266f"
Sep 10 19:18:27.675344 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.675102    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-web-config" (OuterVolumeSpecName: "web-config") pod "f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a" (UID: "f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:18:27.681494 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.681475    2564 scope.go:117] "RemoveContainer" containerID="1c2dd76d55247af36a5f7feab7a6bb31e482ea77ce6b0a61ef597647f032b381"
Sep 10 19:18:27.688600 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.688568    2564 scope.go:117] "RemoveContainer" containerID="6fe837d0189b954b19f649a4c5a5ae0553778f0dd1132df3d27fa9f944d14813"
Sep 10 19:18:27.696140 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.696114    2564 scope.go:117] "RemoveContainer" containerID="afaf414825f92c83d6de30df05d873b6d29b4255da22b69c4a6916bb63a5f18d"
Sep 10 19:18:27.704603 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.704581    2564 scope.go:117] "RemoveContainer" containerID="ab5a5945bd22dd2c7affd8044765602cbcba8ac88e49e6674edad189da741df6"
Sep 10 19:18:27.711829 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.711810    2564 scope.go:117] "RemoveContainer" containerID="423ab8ab659fb962c232769b6954c08bb47d5a49477c475f9100b7d7e0f0d4fc"
Sep 10 19:18:27.719074 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.719051    2564 scope.go:117] "RemoveContainer" containerID="587658c07f740026e6cac9858f8b0fc15d119726bc1fa33486d85243916373c1"
Sep 10 19:18:27.725980 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.725960    2564 scope.go:117] "RemoveContainer" containerID="fd61e04df64a2ab106f9c39884abb3fd65b0aa621f809e09dba4ee49d6e7266f"
Sep 10 19:18:27.726280 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:18:27.726255    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd61e04df64a2ab106f9c39884abb3fd65b0aa621f809e09dba4ee49d6e7266f\": container with ID starting with fd61e04df64a2ab106f9c39884abb3fd65b0aa621f809e09dba4ee49d6e7266f not found: ID does not exist" containerID="fd61e04df64a2ab106f9c39884abb3fd65b0aa621f809e09dba4ee49d6e7266f"
Sep 10 19:18:27.726365 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.726289    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd61e04df64a2ab106f9c39884abb3fd65b0aa621f809e09dba4ee49d6e7266f"} err="failed to get container status \"fd61e04df64a2ab106f9c39884abb3fd65b0aa621f809e09dba4ee49d6e7266f\": rpc error: code = NotFound desc = could not find container \"fd61e04df64a2ab106f9c39884abb3fd65b0aa621f809e09dba4ee49d6e7266f\": container with ID starting with fd61e04df64a2ab106f9c39884abb3fd65b0aa621f809e09dba4ee49d6e7266f not found: ID does not exist"
Sep 10 19:18:27.726365 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.726322    2564 scope.go:117] "RemoveContainer" containerID="1c2dd76d55247af36a5f7feab7a6bb31e482ea77ce6b0a61ef597647f032b381"
Sep 10 19:18:27.726596 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:18:27.726577    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c2dd76d55247af36a5f7feab7a6bb31e482ea77ce6b0a61ef597647f032b381\": container with ID starting with 1c2dd76d55247af36a5f7feab7a6bb31e482ea77ce6b0a61ef597647f032b381 not found: ID does not exist" containerID="1c2dd76d55247af36a5f7feab7a6bb31e482ea77ce6b0a61ef597647f032b381"
Sep 10 19:18:27.726679 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.726608    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c2dd76d55247af36a5f7feab7a6bb31e482ea77ce6b0a61ef597647f032b381"} err="failed to get container status \"1c2dd76d55247af36a5f7feab7a6bb31e482ea77ce6b0a61ef597647f032b381\": rpc error: code = NotFound desc = could not find container \"1c2dd76d55247af36a5f7feab7a6bb31e482ea77ce6b0a61ef597647f032b381\": container with ID starting with 1c2dd76d55247af36a5f7feab7a6bb31e482ea77ce6b0a61ef597647f032b381 not found: ID does not exist"
Sep 10 19:18:27.726679 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.726657    2564 scope.go:117] "RemoveContainer" containerID="6fe837d0189b954b19f649a4c5a5ae0553778f0dd1132df3d27fa9f944d14813"
Sep 10 19:18:27.726896 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:18:27.726877    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6fe837d0189b954b19f649a4c5a5ae0553778f0dd1132df3d27fa9f944d14813\": container with ID starting with 6fe837d0189b954b19f649a4c5a5ae0553778f0dd1132df3d27fa9f944d14813 not found: ID does not exist" containerID="6fe837d0189b954b19f649a4c5a5ae0553778f0dd1132df3d27fa9f944d14813"
Sep 10 19:18:27.726958 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.726902    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fe837d0189b954b19f649a4c5a5ae0553778f0dd1132df3d27fa9f944d14813"} err="failed to get container status \"6fe837d0189b954b19f649a4c5a5ae0553778f0dd1132df3d27fa9f944d14813\": rpc error: code = NotFound desc = could not find container \"6fe837d0189b954b19f649a4c5a5ae0553778f0dd1132df3d27fa9f944d14813\": container with ID starting with 6fe837d0189b954b19f649a4c5a5ae0553778f0dd1132df3d27fa9f944d14813 not found: ID does not exist"
Sep 10 19:18:27.726958 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.726917    2564 scope.go:117] "RemoveContainer" containerID="afaf414825f92c83d6de30df05d873b6d29b4255da22b69c4a6916bb63a5f18d"
Sep 10 19:18:27.727164 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:18:27.727148    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"afaf414825f92c83d6de30df05d873b6d29b4255da22b69c4a6916bb63a5f18d\": container with ID starting with afaf414825f92c83d6de30df05d873b6d29b4255da22b69c4a6916bb63a5f18d not found: ID does not exist" containerID="afaf414825f92c83d6de30df05d873b6d29b4255da22b69c4a6916bb63a5f18d"
Sep 10 19:18:27.727204 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.727166    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afaf414825f92c83d6de30df05d873b6d29b4255da22b69c4a6916bb63a5f18d"} err="failed to get container status \"afaf414825f92c83d6de30df05d873b6d29b4255da22b69c4a6916bb63a5f18d\": rpc error: code = NotFound desc = could not find container \"afaf414825f92c83d6de30df05d873b6d29b4255da22b69c4a6916bb63a5f18d\": container with ID starting with afaf414825f92c83d6de30df05d873b6d29b4255da22b69c4a6916bb63a5f18d not found: ID does not exist"
Sep 10 19:18:27.727204 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.727178    2564 scope.go:117] "RemoveContainer" containerID="ab5a5945bd22dd2c7affd8044765602cbcba8ac88e49e6674edad189da741df6"
Sep 10 19:18:27.727411 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:18:27.727397    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab5a5945bd22dd2c7affd8044765602cbcba8ac88e49e6674edad189da741df6\": container with ID starting with ab5a5945bd22dd2c7affd8044765602cbcba8ac88e49e6674edad189da741df6 not found: ID does not exist" containerID="ab5a5945bd22dd2c7affd8044765602cbcba8ac88e49e6674edad189da741df6"
Sep 10 19:18:27.727468 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.727413    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab5a5945bd22dd2c7affd8044765602cbcba8ac88e49e6674edad189da741df6"} err="failed to get container status \"ab5a5945bd22dd2c7affd8044765602cbcba8ac88e49e6674edad189da741df6\": rpc error: code = NotFound desc = could not find container \"ab5a5945bd22dd2c7affd8044765602cbcba8ac88e49e6674edad189da741df6\": container with ID starting with ab5a5945bd22dd2c7affd8044765602cbcba8ac88e49e6674edad189da741df6 not found: ID does not exist"
Sep 10 19:18:27.727468 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.727424    2564 scope.go:117] "RemoveContainer" containerID="423ab8ab659fb962c232769b6954c08bb47d5a49477c475f9100b7d7e0f0d4fc"
Sep 10 19:18:27.727602 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:18:27.727585    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"423ab8ab659fb962c232769b6954c08bb47d5a49477c475f9100b7d7e0f0d4fc\": container with ID starting with 423ab8ab659fb962c232769b6954c08bb47d5a49477c475f9100b7d7e0f0d4fc not found: ID does not exist" containerID="423ab8ab659fb962c232769b6954c08bb47d5a49477c475f9100b7d7e0f0d4fc"
Sep 10 19:18:27.727669 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.727604    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"423ab8ab659fb962c232769b6954c08bb47d5a49477c475f9100b7d7e0f0d4fc"} err="failed to get container status \"423ab8ab659fb962c232769b6954c08bb47d5a49477c475f9100b7d7e0f0d4fc\": rpc error: code = NotFound desc = could not find container \"423ab8ab659fb962c232769b6954c08bb47d5a49477c475f9100b7d7e0f0d4fc\": container with ID starting with 423ab8ab659fb962c232769b6954c08bb47d5a49477c475f9100b7d7e0f0d4fc not found: ID does not exist"
Sep 10 19:18:27.727714 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.727615    2564 scope.go:117] "RemoveContainer" containerID="587658c07f740026e6cac9858f8b0fc15d119726bc1fa33486d85243916373c1"
Sep 10 19:18:27.727921 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:18:27.727901    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"587658c07f740026e6cac9858f8b0fc15d119726bc1fa33486d85243916373c1\": container with ID starting with 587658c07f740026e6cac9858f8b0fc15d119726bc1fa33486d85243916373c1 not found: ID does not exist" containerID="587658c07f740026e6cac9858f8b0fc15d119726bc1fa33486d85243916373c1"
Sep 10 19:18:27.727972 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.727924    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"587658c07f740026e6cac9858f8b0fc15d119726bc1fa33486d85243916373c1"} err="failed to get container status \"587658c07f740026e6cac9858f8b0fc15d119726bc1fa33486d85243916373c1\": rpc error: code = NotFound desc = could not find container \"587658c07f740026e6cac9858f8b0fc15d119726bc1fa33486d85243916373c1\": container with ID starting with 587658c07f740026e6cac9858f8b0fc15d119726bc1fa33486d85243916373c1 not found: ID does not exist"
Sep 10 19:18:27.727972 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.727937    2564 scope.go:117] "RemoveContainer" containerID="fd61e04df64a2ab106f9c39884abb3fd65b0aa621f809e09dba4ee49d6e7266f"
Sep 10 19:18:27.728164 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.728149    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd61e04df64a2ab106f9c39884abb3fd65b0aa621f809e09dba4ee49d6e7266f"} err="failed to get container status \"fd61e04df64a2ab106f9c39884abb3fd65b0aa621f809e09dba4ee49d6e7266f\": rpc error: code = NotFound desc = could not find container \"fd61e04df64a2ab106f9c39884abb3fd65b0aa621f809e09dba4ee49d6e7266f\": container with ID starting with fd61e04df64a2ab106f9c39884abb3fd65b0aa621f809e09dba4ee49d6e7266f not found: ID does not exist"
Sep 10 19:18:27.728208 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.728164    2564 scope.go:117] "RemoveContainer" containerID="1c2dd76d55247af36a5f7feab7a6bb31e482ea77ce6b0a61ef597647f032b381"
Sep 10 19:18:27.728388 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.728369    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c2dd76d55247af36a5f7feab7a6bb31e482ea77ce6b0a61ef597647f032b381"} err="failed to get container status \"1c2dd76d55247af36a5f7feab7a6bb31e482ea77ce6b0a61ef597647f032b381\": rpc error: code = NotFound desc = could not find container \"1c2dd76d55247af36a5f7feab7a6bb31e482ea77ce6b0a61ef597647f032b381\": container with ID starting with 1c2dd76d55247af36a5f7feab7a6bb31e482ea77ce6b0a61ef597647f032b381 not found: ID does not exist"
Sep 10 19:18:27.728452 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.728389    2564 scope.go:117] "RemoveContainer" containerID="6fe837d0189b954b19f649a4c5a5ae0553778f0dd1132df3d27fa9f944d14813"
Sep 10 19:18:27.728607 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.728592    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fe837d0189b954b19f649a4c5a5ae0553778f0dd1132df3d27fa9f944d14813"} err="failed to get container status \"6fe837d0189b954b19f649a4c5a5ae0553778f0dd1132df3d27fa9f944d14813\": rpc error: code = NotFound desc = could not find container \"6fe837d0189b954b19f649a4c5a5ae0553778f0dd1132df3d27fa9f944d14813\": container with ID starting with 6fe837d0189b954b19f649a4c5a5ae0553778f0dd1132df3d27fa9f944d14813 not found: ID does not exist"
Sep 10 19:18:27.728692 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.728606    2564 scope.go:117] "RemoveContainer" containerID="afaf414825f92c83d6de30df05d873b6d29b4255da22b69c4a6916bb63a5f18d"
Sep 10 19:18:27.728831 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.728814    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afaf414825f92c83d6de30df05d873b6d29b4255da22b69c4a6916bb63a5f18d"} err="failed to get container status \"afaf414825f92c83d6de30df05d873b6d29b4255da22b69c4a6916bb63a5f18d\": rpc error: code = NotFound desc = could not find container \"afaf414825f92c83d6de30df05d873b6d29b4255da22b69c4a6916bb63a5f18d\": container with ID starting with afaf414825f92c83d6de30df05d873b6d29b4255da22b69c4a6916bb63a5f18d not found: ID does not exist"
Sep 10 19:18:27.728881 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.728832    2564 scope.go:117] "RemoveContainer" containerID="ab5a5945bd22dd2c7affd8044765602cbcba8ac88e49e6674edad189da741df6"
Sep 10 19:18:27.729036 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.729014    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab5a5945bd22dd2c7affd8044765602cbcba8ac88e49e6674edad189da741df6"} err="failed to get container status \"ab5a5945bd22dd2c7affd8044765602cbcba8ac88e49e6674edad189da741df6\": rpc error: code = NotFound desc = could not find container \"ab5a5945bd22dd2c7affd8044765602cbcba8ac88e49e6674edad189da741df6\": container with ID starting with ab5a5945bd22dd2c7affd8044765602cbcba8ac88e49e6674edad189da741df6 not found: ID does not exist"
Sep 10 19:18:27.729036 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.729033    2564 scope.go:117] "RemoveContainer" containerID="423ab8ab659fb962c232769b6954c08bb47d5a49477c475f9100b7d7e0f0d4fc"
Sep 10 19:18:27.729250 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.729232    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"423ab8ab659fb962c232769b6954c08bb47d5a49477c475f9100b7d7e0f0d4fc"} err="failed to get container status \"423ab8ab659fb962c232769b6954c08bb47d5a49477c475f9100b7d7e0f0d4fc\": rpc error: code = NotFound desc = could not find container \"423ab8ab659fb962c232769b6954c08bb47d5a49477c475f9100b7d7e0f0d4fc\": container with ID starting with 423ab8ab659fb962c232769b6954c08bb47d5a49477c475f9100b7d7e0f0d4fc not found: ID does not exist"
Sep 10 19:18:27.729292 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.729251    2564 scope.go:117] "RemoveContainer" containerID="587658c07f740026e6cac9858f8b0fc15d119726bc1fa33486d85243916373c1"
Sep 10 19:18:27.729476 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.729459    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"587658c07f740026e6cac9858f8b0fc15d119726bc1fa33486d85243916373c1"} err="failed to get container status \"587658c07f740026e6cac9858f8b0fc15d119726bc1fa33486d85243916373c1\": rpc error: code = NotFound desc = could not find container \"587658c07f740026e6cac9858f8b0fc15d119726bc1fa33486d85243916373c1\": container with ID starting with 587658c07f740026e6cac9858f8b0fc15d119726bc1fa33486d85243916373c1 not found: ID does not exist"
Sep 10 19:18:27.729476 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.729475    2564 scope.go:117] "RemoveContainer" containerID="fd61e04df64a2ab106f9c39884abb3fd65b0aa621f809e09dba4ee49d6e7266f"
Sep 10 19:18:27.729692 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.729675    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd61e04df64a2ab106f9c39884abb3fd65b0aa621f809e09dba4ee49d6e7266f"} err="failed to get container status \"fd61e04df64a2ab106f9c39884abb3fd65b0aa621f809e09dba4ee49d6e7266f\": rpc error: code = NotFound desc = could not find container \"fd61e04df64a2ab106f9c39884abb3fd65b0aa621f809e09dba4ee49d6e7266f\": container with ID starting with fd61e04df64a2ab106f9c39884abb3fd65b0aa621f809e09dba4ee49d6e7266f not found: ID does not exist"
Sep 10 19:18:27.729731 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.729692    2564 scope.go:117] "RemoveContainer" containerID="1c2dd76d55247af36a5f7feab7a6bb31e482ea77ce6b0a61ef597647f032b381"
Sep 10 19:18:27.729885 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.729863    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c2dd76d55247af36a5f7feab7a6bb31e482ea77ce6b0a61ef597647f032b381"} err="failed to get container status \"1c2dd76d55247af36a5f7feab7a6bb31e482ea77ce6b0a61ef597647f032b381\": rpc error: code = NotFound desc = could not find container \"1c2dd76d55247af36a5f7feab7a6bb31e482ea77ce6b0a61ef597647f032b381\": container with ID starting with 1c2dd76d55247af36a5f7feab7a6bb31e482ea77ce6b0a61ef597647f032b381 not found: ID does not exist"
Sep 10 19:18:27.729885 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.729885    2564 scope.go:117] "RemoveContainer" containerID="6fe837d0189b954b19f649a4c5a5ae0553778f0dd1132df3d27fa9f944d14813"
Sep 10 19:18:27.730116 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.730100    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fe837d0189b954b19f649a4c5a5ae0553778f0dd1132df3d27fa9f944d14813"} err="failed to get container status \"6fe837d0189b954b19f649a4c5a5ae0553778f0dd1132df3d27fa9f944d14813\": rpc error: code = NotFound desc = could not find container \"6fe837d0189b954b19f649a4c5a5ae0553778f0dd1132df3d27fa9f944d14813\": container with ID starting with 6fe837d0189b954b19f649a4c5a5ae0553778f0dd1132df3d27fa9f944d14813 not found: ID does not exist"
Sep 10 19:18:27.730174 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.730115    2564 scope.go:117] "RemoveContainer" containerID="afaf414825f92c83d6de30df05d873b6d29b4255da22b69c4a6916bb63a5f18d"
Sep 10 19:18:27.730342 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.730323    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afaf414825f92c83d6de30df05d873b6d29b4255da22b69c4a6916bb63a5f18d"} err="failed to get container status \"afaf414825f92c83d6de30df05d873b6d29b4255da22b69c4a6916bb63a5f18d\": rpc error: code = NotFound desc = could not find container \"afaf414825f92c83d6de30df05d873b6d29b4255da22b69c4a6916bb63a5f18d\": container with ID starting with afaf414825f92c83d6de30df05d873b6d29b4255da22b69c4a6916bb63a5f18d not found: ID does not exist"
Sep 10 19:18:27.730407 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.730343    2564 scope.go:117] "RemoveContainer" containerID="ab5a5945bd22dd2c7affd8044765602cbcba8ac88e49e6674edad189da741df6"
Sep 10 19:18:27.730583 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.730562    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab5a5945bd22dd2c7affd8044765602cbcba8ac88e49e6674edad189da741df6"} err="failed to get container status \"ab5a5945bd22dd2c7affd8044765602cbcba8ac88e49e6674edad189da741df6\": rpc error: code = NotFound desc = could not find container \"ab5a5945bd22dd2c7affd8044765602cbcba8ac88e49e6674edad189da741df6\": container with ID starting with ab5a5945bd22dd2c7affd8044765602cbcba8ac88e49e6674edad189da741df6 not found: ID does not exist"
Sep 10 19:18:27.730583 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.730583    2564 scope.go:117] "RemoveContainer" containerID="423ab8ab659fb962c232769b6954c08bb47d5a49477c475f9100b7d7e0f0d4fc"
Sep 10 19:18:27.730858 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.730840    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"423ab8ab659fb962c232769b6954c08bb47d5a49477c475f9100b7d7e0f0d4fc"} err="failed to get container status \"423ab8ab659fb962c232769b6954c08bb47d5a49477c475f9100b7d7e0f0d4fc\": rpc error: code = NotFound desc = could not find container \"423ab8ab659fb962c232769b6954c08bb47d5a49477c475f9100b7d7e0f0d4fc\": container with ID starting with 423ab8ab659fb962c232769b6954c08bb47d5a49477c475f9100b7d7e0f0d4fc not found: ID does not exist"
Sep 10 19:18:27.730907 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.730860    2564 scope.go:117] "RemoveContainer" containerID="587658c07f740026e6cac9858f8b0fc15d119726bc1fa33486d85243916373c1"
Sep 10 19:18:27.731130 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.731113    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"587658c07f740026e6cac9858f8b0fc15d119726bc1fa33486d85243916373c1"} err="failed to get container status \"587658c07f740026e6cac9858f8b0fc15d119726bc1fa33486d85243916373c1\": rpc error: code = NotFound desc = could not find container \"587658c07f740026e6cac9858f8b0fc15d119726bc1fa33486d85243916373c1\": container with ID starting with 587658c07f740026e6cac9858f8b0fc15d119726bc1fa33486d85243916373c1 not found: ID does not exist"
Sep 10 19:18:27.731179 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.731131    2564 scope.go:117] "RemoveContainer" containerID="fd61e04df64a2ab106f9c39884abb3fd65b0aa621f809e09dba4ee49d6e7266f"
Sep 10 19:18:27.731377 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.731357    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd61e04df64a2ab106f9c39884abb3fd65b0aa621f809e09dba4ee49d6e7266f"} err="failed to get container status \"fd61e04df64a2ab106f9c39884abb3fd65b0aa621f809e09dba4ee49d6e7266f\": rpc error: code = NotFound desc = could not find container \"fd61e04df64a2ab106f9c39884abb3fd65b0aa621f809e09dba4ee49d6e7266f\": container with ID starting with fd61e04df64a2ab106f9c39884abb3fd65b0aa621f809e09dba4ee49d6e7266f not found: ID does not exist"
Sep 10 19:18:27.731434 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.731378    2564 scope.go:117] "RemoveContainer" containerID="1c2dd76d55247af36a5f7feab7a6bb31e482ea77ce6b0a61ef597647f032b381"
Sep 10 19:18:27.731597 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.731573    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c2dd76d55247af36a5f7feab7a6bb31e482ea77ce6b0a61ef597647f032b381"} err="failed to get container status \"1c2dd76d55247af36a5f7feab7a6bb31e482ea77ce6b0a61ef597647f032b381\": rpc error: code = NotFound desc = could not find container \"1c2dd76d55247af36a5f7feab7a6bb31e482ea77ce6b0a61ef597647f032b381\": container with ID starting with 1c2dd76d55247af36a5f7feab7a6bb31e482ea77ce6b0a61ef597647f032b381 not found: ID does not exist"
Sep 10 19:18:27.731597 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.731593    2564 scope.go:117] "RemoveContainer" containerID="6fe837d0189b954b19f649a4c5a5ae0553778f0dd1132df3d27fa9f944d14813"
Sep 10 19:18:27.731858 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.731841    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fe837d0189b954b19f649a4c5a5ae0553778f0dd1132df3d27fa9f944d14813"} err="failed to get container status \"6fe837d0189b954b19f649a4c5a5ae0553778f0dd1132df3d27fa9f944d14813\": rpc error: code = NotFound desc = could not find container \"6fe837d0189b954b19f649a4c5a5ae0553778f0dd1132df3d27fa9f944d14813\": container with ID starting with 6fe837d0189b954b19f649a4c5a5ae0553778f0dd1132df3d27fa9f944d14813 not found: ID does not exist"
Sep 10 19:18:27.731900 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.731860    2564 scope.go:117] "RemoveContainer" containerID="afaf414825f92c83d6de30df05d873b6d29b4255da22b69c4a6916bb63a5f18d"
Sep 10 19:18:27.732092 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.732073    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afaf414825f92c83d6de30df05d873b6d29b4255da22b69c4a6916bb63a5f18d"} err="failed to get container status \"afaf414825f92c83d6de30df05d873b6d29b4255da22b69c4a6916bb63a5f18d\": rpc error: code = NotFound desc = could not find container \"afaf414825f92c83d6de30df05d873b6d29b4255da22b69c4a6916bb63a5f18d\": container with ID starting with afaf414825f92c83d6de30df05d873b6d29b4255da22b69c4a6916bb63a5f18d not found: ID does not exist"
Sep 10 19:18:27.732150 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.732093    2564 scope.go:117] "RemoveContainer" containerID="ab5a5945bd22dd2c7affd8044765602cbcba8ac88e49e6674edad189da741df6"
Sep 10 19:18:27.732322 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.732306    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab5a5945bd22dd2c7affd8044765602cbcba8ac88e49e6674edad189da741df6"} err="failed to get container status \"ab5a5945bd22dd2c7affd8044765602cbcba8ac88e49e6674edad189da741df6\": rpc error: code = NotFound desc = could not find container \"ab5a5945bd22dd2c7affd8044765602cbcba8ac88e49e6674edad189da741df6\": container with ID starting with ab5a5945bd22dd2c7affd8044765602cbcba8ac88e49e6674edad189da741df6 not found: ID does not exist"
Sep 10 19:18:27.732391 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.732322    2564 scope.go:117] "RemoveContainer" containerID="423ab8ab659fb962c232769b6954c08bb47d5a49477c475f9100b7d7e0f0d4fc"
Sep 10 19:18:27.732520 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.732505    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"423ab8ab659fb962c232769b6954c08bb47d5a49477c475f9100b7d7e0f0d4fc"} err="failed to get container status \"423ab8ab659fb962c232769b6954c08bb47d5a49477c475f9100b7d7e0f0d4fc\": rpc error: code = NotFound desc = could not find container \"423ab8ab659fb962c232769b6954c08bb47d5a49477c475f9100b7d7e0f0d4fc\": container with ID starting with 423ab8ab659fb962c232769b6954c08bb47d5a49477c475f9100b7d7e0f0d4fc not found: ID does not exist"
Sep 10 19:18:27.732561 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.732519    2564 scope.go:117] "RemoveContainer" containerID="587658c07f740026e6cac9858f8b0fc15d119726bc1fa33486d85243916373c1"
Sep 10 19:18:27.732760 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.732742    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"587658c07f740026e6cac9858f8b0fc15d119726bc1fa33486d85243916373c1"} err="failed to get container status \"587658c07f740026e6cac9858f8b0fc15d119726bc1fa33486d85243916373c1\": rpc error: code = NotFound desc = could not find container \"587658c07f740026e6cac9858f8b0fc15d119726bc1fa33486d85243916373c1\": container with ID starting with 587658c07f740026e6cac9858f8b0fc15d119726bc1fa33486d85243916373c1 not found: ID does not exist"
Sep 10 19:18:27.732819 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.732761    2564 scope.go:117] "RemoveContainer" containerID="fd61e04df64a2ab106f9c39884abb3fd65b0aa621f809e09dba4ee49d6e7266f"
Sep 10 19:18:27.732991 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.732975    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd61e04df64a2ab106f9c39884abb3fd65b0aa621f809e09dba4ee49d6e7266f"} err="failed to get container status \"fd61e04df64a2ab106f9c39884abb3fd65b0aa621f809e09dba4ee49d6e7266f\": rpc error: code = NotFound desc = could not find container \"fd61e04df64a2ab106f9c39884abb3fd65b0aa621f809e09dba4ee49d6e7266f\": container with ID starting with fd61e04df64a2ab106f9c39884abb3fd65b0aa621f809e09dba4ee49d6e7266f not found: ID does not exist"
Sep 10 19:18:27.733041 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.732992    2564 scope.go:117] "RemoveContainer" containerID="1c2dd76d55247af36a5f7feab7a6bb31e482ea77ce6b0a61ef597647f032b381"
Sep 10 19:18:27.733213 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.733193    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c2dd76d55247af36a5f7feab7a6bb31e482ea77ce6b0a61ef597647f032b381"} err="failed to get container status \"1c2dd76d55247af36a5f7feab7a6bb31e482ea77ce6b0a61ef597647f032b381\": rpc error: code = NotFound desc = could not find container \"1c2dd76d55247af36a5f7feab7a6bb31e482ea77ce6b0a61ef597647f032b381\": container with ID starting with 1c2dd76d55247af36a5f7feab7a6bb31e482ea77ce6b0a61ef597647f032b381 not found: ID does not exist"
Sep 10 19:18:27.733254 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.733214    2564 scope.go:117] "RemoveContainer" containerID="6fe837d0189b954b19f649a4c5a5ae0553778f0dd1132df3d27fa9f944d14813"
Sep 10 19:18:27.733437 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.733419    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fe837d0189b954b19f649a4c5a5ae0553778f0dd1132df3d27fa9f944d14813"} err="failed to get container status \"6fe837d0189b954b19f649a4c5a5ae0553778f0dd1132df3d27fa9f944d14813\": rpc error: code = NotFound desc = could not find container \"6fe837d0189b954b19f649a4c5a5ae0553778f0dd1132df3d27fa9f944d14813\": container with ID starting with 6fe837d0189b954b19f649a4c5a5ae0553778f0dd1132df3d27fa9f944d14813 not found: ID does not exist"
Sep 10 19:18:27.733479 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.733438    2564 scope.go:117] "RemoveContainer" containerID="afaf414825f92c83d6de30df05d873b6d29b4255da22b69c4a6916bb63a5f18d"
Sep 10 19:18:27.733678 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.733663    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afaf414825f92c83d6de30df05d873b6d29b4255da22b69c4a6916bb63a5f18d"} err="failed to get container status \"afaf414825f92c83d6de30df05d873b6d29b4255da22b69c4a6916bb63a5f18d\": rpc error: code = NotFound desc = could not find container \"afaf414825f92c83d6de30df05d873b6d29b4255da22b69c4a6916bb63a5f18d\": container with ID starting with afaf414825f92c83d6de30df05d873b6d29b4255da22b69c4a6916bb63a5f18d not found: ID does not exist"
Sep 10 19:18:27.733678 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.733676    2564 scope.go:117] "RemoveContainer" containerID="ab5a5945bd22dd2c7affd8044765602cbcba8ac88e49e6674edad189da741df6"
Sep 10 19:18:27.733897 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.733881    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab5a5945bd22dd2c7affd8044765602cbcba8ac88e49e6674edad189da741df6"} err="failed to get container status \"ab5a5945bd22dd2c7affd8044765602cbcba8ac88e49e6674edad189da741df6\": rpc error: code = NotFound desc = could not find container \"ab5a5945bd22dd2c7affd8044765602cbcba8ac88e49e6674edad189da741df6\": container with ID starting with ab5a5945bd22dd2c7affd8044765602cbcba8ac88e49e6674edad189da741df6 not found: ID does not exist"
Sep 10 19:18:27.733897 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.733897    2564 scope.go:117] "RemoveContainer" containerID="423ab8ab659fb962c232769b6954c08bb47d5a49477c475f9100b7d7e0f0d4fc"
Sep 10 19:18:27.734109 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.734091    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"423ab8ab659fb962c232769b6954c08bb47d5a49477c475f9100b7d7e0f0d4fc"} err="failed to get container status \"423ab8ab659fb962c232769b6954c08bb47d5a49477c475f9100b7d7e0f0d4fc\": rpc error: code = NotFound desc = could not find container \"423ab8ab659fb962c232769b6954c08bb47d5a49477c475f9100b7d7e0f0d4fc\": container with ID starting with 423ab8ab659fb962c232769b6954c08bb47d5a49477c475f9100b7d7e0f0d4fc not found: ID does not exist"
Sep 10 19:18:27.734109 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.734108    2564 scope.go:117] "RemoveContainer" containerID="587658c07f740026e6cac9858f8b0fc15d119726bc1fa33486d85243916373c1"
Sep 10 19:18:27.734308 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.734293    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"587658c07f740026e6cac9858f8b0fc15d119726bc1fa33486d85243916373c1"} err="failed to get container status \"587658c07f740026e6cac9858f8b0fc15d119726bc1fa33486d85243916373c1\": rpc error: code = NotFound desc = could not find container \"587658c07f740026e6cac9858f8b0fc15d119726bc1fa33486d85243916373c1\": container with ID starting with 587658c07f740026e6cac9858f8b0fc15d119726bc1fa33486d85243916373c1 not found: ID does not exist"
Sep 10 19:18:27.734352 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.734308    2564 scope.go:117] "RemoveContainer" containerID="fd61e04df64a2ab106f9c39884abb3fd65b0aa621f809e09dba4ee49d6e7266f"
Sep 10 19:18:27.734527 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.734507    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd61e04df64a2ab106f9c39884abb3fd65b0aa621f809e09dba4ee49d6e7266f"} err="failed to get container status \"fd61e04df64a2ab106f9c39884abb3fd65b0aa621f809e09dba4ee49d6e7266f\": rpc error: code = NotFound desc = could not find container \"fd61e04df64a2ab106f9c39884abb3fd65b0aa621f809e09dba4ee49d6e7266f\": container with ID starting with fd61e04df64a2ab106f9c39884abb3fd65b0aa621f809e09dba4ee49d6e7266f not found: ID does not exist"
Sep 10 19:18:27.734580 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.734527    2564 scope.go:117] "RemoveContainer" containerID="1c2dd76d55247af36a5f7feab7a6bb31e482ea77ce6b0a61ef597647f032b381"
Sep 10 19:18:27.734780 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.734763    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c2dd76d55247af36a5f7feab7a6bb31e482ea77ce6b0a61ef597647f032b381"} err="failed to get container status \"1c2dd76d55247af36a5f7feab7a6bb31e482ea77ce6b0a61ef597647f032b381\": rpc error: code = NotFound desc = could not find container \"1c2dd76d55247af36a5f7feab7a6bb31e482ea77ce6b0a61ef597647f032b381\": container with ID starting with 1c2dd76d55247af36a5f7feab7a6bb31e482ea77ce6b0a61ef597647f032b381 not found: ID does not exist"
Sep 10 19:18:27.734835 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.734781    2564 scope.go:117] "RemoveContainer" containerID="6fe837d0189b954b19f649a4c5a5ae0553778f0dd1132df3d27fa9f944d14813"
Sep 10 19:18:27.735011 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.734985    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fe837d0189b954b19f649a4c5a5ae0553778f0dd1132df3d27fa9f944d14813"} err="failed to get container status \"6fe837d0189b954b19f649a4c5a5ae0553778f0dd1132df3d27fa9f944d14813\": rpc error: code = NotFound desc = could not find container \"6fe837d0189b954b19f649a4c5a5ae0553778f0dd1132df3d27fa9f944d14813\": container with ID starting with 6fe837d0189b954b19f649a4c5a5ae0553778f0dd1132df3d27fa9f944d14813 not found: ID does not exist"
Sep 10 19:18:27.735072 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.735011    2564 scope.go:117] "RemoveContainer" containerID="afaf414825f92c83d6de30df05d873b6d29b4255da22b69c4a6916bb63a5f18d"
Sep 10 19:18:27.735233 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.735214    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afaf414825f92c83d6de30df05d873b6d29b4255da22b69c4a6916bb63a5f18d"} err="failed to get container status \"afaf414825f92c83d6de30df05d873b6d29b4255da22b69c4a6916bb63a5f18d\": rpc error: code = NotFound desc = could not find container \"afaf414825f92c83d6de30df05d873b6d29b4255da22b69c4a6916bb63a5f18d\": container with ID starting with afaf414825f92c83d6de30df05d873b6d29b4255da22b69c4a6916bb63a5f18d not found: ID does not exist"
Sep 10 19:18:27.735301 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.735235    2564 scope.go:117] "RemoveContainer" containerID="ab5a5945bd22dd2c7affd8044765602cbcba8ac88e49e6674edad189da741df6"
Sep 10 19:18:27.735471 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.735454    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab5a5945bd22dd2c7affd8044765602cbcba8ac88e49e6674edad189da741df6"} err="failed to get container status \"ab5a5945bd22dd2c7affd8044765602cbcba8ac88e49e6674edad189da741df6\": rpc error: code = NotFound desc = could not find container \"ab5a5945bd22dd2c7affd8044765602cbcba8ac88e49e6674edad189da741df6\": container with ID starting with ab5a5945bd22dd2c7affd8044765602cbcba8ac88e49e6674edad189da741df6 not found: ID does not exist"
Sep 10 19:18:27.735533 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.735471    2564 scope.go:117] "RemoveContainer" containerID="423ab8ab659fb962c232769b6954c08bb47d5a49477c475f9100b7d7e0f0d4fc"
Sep 10 19:18:27.735693 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.735671    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"423ab8ab659fb962c232769b6954c08bb47d5a49477c475f9100b7d7e0f0d4fc"} err="failed to get container status \"423ab8ab659fb962c232769b6954c08bb47d5a49477c475f9100b7d7e0f0d4fc\": rpc error: code = NotFound desc = could not find container \"423ab8ab659fb962c232769b6954c08bb47d5a49477c475f9100b7d7e0f0d4fc\": container with ID starting with 423ab8ab659fb962c232769b6954c08bb47d5a49477c475f9100b7d7e0f0d4fc not found: ID does not exist"
Sep 10 19:18:27.735693 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.735687    2564 scope.go:117] "RemoveContainer" containerID="587658c07f740026e6cac9858f8b0fc15d119726bc1fa33486d85243916373c1"
Sep 10 19:18:27.735898 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.735881    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"587658c07f740026e6cac9858f8b0fc15d119726bc1fa33486d85243916373c1"} err="failed to get container status \"587658c07f740026e6cac9858f8b0fc15d119726bc1fa33486d85243916373c1\": rpc error: code = NotFound desc = could not find container \"587658c07f740026e6cac9858f8b0fc15d119726bc1fa33486d85243916373c1\": container with ID starting with 587658c07f740026e6cac9858f8b0fc15d119726bc1fa33486d85243916373c1 not found: ID does not exist"
Sep 10 19:18:27.752379 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.752344    2564 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-secret-grpc-tls\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:18:27.752379 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.752375    2564 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-config-out\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:18:27.752379 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.752386    2564 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-configmap-metrics-client-ca\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:18:27.752593 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.752395    2564 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-prometheus-k8s-db\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:18:27.752593 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.752406    2564 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-config\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:18:27.752593 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.752414    2564 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-prometheus-trusted-ca-bundle\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:18:27.752593 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.752422    2564 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:18:27.752593 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.752431    2564 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:18:27.752593 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.752440    2564 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:18:27.752593 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.752448    2564 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:18:27.752593 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.752458    2564 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-tls-assets\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:18:27.752593 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.752467    2564 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:18:27.752593 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.752475    2564 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-web-config\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:18:27.752593 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.752483    2564 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-secret-prometheus-k8s-tls\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:18:27.752593 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.752491    2564 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-thanos-prometheus-http-client-file\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:18:27.752593 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.752499    2564 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-secret-kube-rbac-proxy\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:18:27.752593 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.752508    2564 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-secret-metrics-client-certs\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:18:27.752593 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.752517    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lnb4r\" (UniqueName: \"kubernetes.io/projected/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-kube-api-access-lnb4r\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:18:27.752593 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:27.752525    2564 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:18:28.003940 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.003907    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 19:18:28.008435 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.008407    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 19:18:28.033970 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.033940    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 19:18:28.034223 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.034211    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a" containerName="kube-rbac-proxy-web"
Sep 10 19:18:28.034273 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.034224    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a" containerName="kube-rbac-proxy-web"
Sep 10 19:18:28.034273 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.034234    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c459a94-07c5-44fb-b155-8a8f7aa46d25" containerName="registry"
Sep 10 19:18:28.034273 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.034239    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c459a94-07c5-44fb-b155-8a8f7aa46d25" containerName="registry"
Sep 10 19:18:28.034273 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.034246    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a" containerName="thanos-sidecar"
Sep 10 19:18:28.034273 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.034252    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a" containerName="thanos-sidecar"
Sep 10 19:18:28.034273 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.034261    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a" containerName="prometheus"
Sep 10 19:18:28.034273 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.034266    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a" containerName="prometheus"
Sep 10 19:18:28.034273 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.034272    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13aa0d02-73a3-4f27-993b-46ab70dbed06" containerName="console"
Sep 10 19:18:28.034273 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.034278    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="13aa0d02-73a3-4f27-993b-46ab70dbed06" containerName="console"
Sep 10 19:18:28.034556 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.034285    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a" containerName="config-reloader"
Sep 10 19:18:28.034556 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.034290    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a" containerName="config-reloader"
Sep 10 19:18:28.034556 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.034297    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a" containerName="kube-rbac-proxy-thanos"
Sep 10 19:18:28.034556 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.034302    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a" containerName="kube-rbac-proxy-thanos"
Sep 10 19:18:28.034556 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.034307    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a" containerName="kube-rbac-proxy"
Sep 10 19:18:28.034556 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.034312    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a" containerName="kube-rbac-proxy"
Sep 10 19:18:28.034556 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.034317    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a" containerName="init-config-reloader"
Sep 10 19:18:28.034556 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.034322    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a" containerName="init-config-reloader"
Sep 10 19:18:28.034556 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.034359    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a" containerName="prometheus"
Sep 10 19:18:28.034556 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.034367    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a" containerName="thanos-sidecar"
Sep 10 19:18:28.034556 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.034374    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a" containerName="config-reloader"
Sep 10 19:18:28.034556 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.034380    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c459a94-07c5-44fb-b155-8a8f7aa46d25" containerName="registry"
Sep 10 19:18:28.034556 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.034384    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="13aa0d02-73a3-4f27-993b-46ab70dbed06" containerName="console"
Sep 10 19:18:28.034556 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.034389    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a" containerName="kube-rbac-proxy-thanos"
Sep 10 19:18:28.034556 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.034394    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a" containerName="kube-rbac-proxy-web"
Sep 10 19:18:28.034556 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.034399    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a" containerName="kube-rbac-proxy"
Sep 10 19:18:28.040048 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.040022    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:28.043050 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.043022    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 10 19:18:28.043205 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.043027    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-xd7q5\""
Sep 10 19:18:28.043205 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.043026    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 10 19:18:28.043205 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.043077    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 10 19:18:28.043429 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.043412    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 10 19:18:28.043553 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.043486    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-2alcjquntm8be\""
Sep 10 19:18:28.043553 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.043513    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 10 19:18:28.043817 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.043551    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 10 19:18:28.043817 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.043610    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 10 19:18:28.044036 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.043833    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 10 19:18:28.044036 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.043843    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 10 19:18:28.044036 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.043858    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 10 19:18:28.044036 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.043908    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 10 19:18:28.044266 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.044038    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 10 19:18:28.045853 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.045831    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 10 19:18:28.049084 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.049063    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 10 19:18:28.053862 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.053839    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 19:18:28.155596 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.155551    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c121a589-1848-4341-a3a0-5a4627dee824-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c121a589-1848-4341-a3a0-5a4627dee824\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:28.155596 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.155590    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c121a589-1848-4341-a3a0-5a4627dee824-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c121a589-1848-4341-a3a0-5a4627dee824\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:28.155835 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.155614    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c121a589-1848-4341-a3a0-5a4627dee824-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c121a589-1848-4341-a3a0-5a4627dee824\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:28.155835 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.155667    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c121a589-1848-4341-a3a0-5a4627dee824-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c121a589-1848-4341-a3a0-5a4627dee824\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:28.155835 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.155686    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c121a589-1848-4341-a3a0-5a4627dee824-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c121a589-1848-4341-a3a0-5a4627dee824\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:28.155835 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.155700    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c121a589-1848-4341-a3a0-5a4627dee824-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c121a589-1848-4341-a3a0-5a4627dee824\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:28.155835 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.155727    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c121a589-1848-4341-a3a0-5a4627dee824-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c121a589-1848-4341-a3a0-5a4627dee824\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:28.155835 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.155745    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvbhf\" (UniqueName: \"kubernetes.io/projected/c121a589-1848-4341-a3a0-5a4627dee824-kube-api-access-dvbhf\") pod \"prometheus-k8s-0\" (UID: \"c121a589-1848-4341-a3a0-5a4627dee824\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:28.155835 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.155764    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c121a589-1848-4341-a3a0-5a4627dee824-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c121a589-1848-4341-a3a0-5a4627dee824\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:28.156045 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.155831    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c121a589-1848-4341-a3a0-5a4627dee824-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c121a589-1848-4341-a3a0-5a4627dee824\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:28.156045 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.155863    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c121a589-1848-4341-a3a0-5a4627dee824-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c121a589-1848-4341-a3a0-5a4627dee824\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:28.156045 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.155881    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c121a589-1848-4341-a3a0-5a4627dee824-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c121a589-1848-4341-a3a0-5a4627dee824\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:28.156045 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.155959    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c121a589-1848-4341-a3a0-5a4627dee824-config-out\") pod \"prometheus-k8s-0\" (UID: \"c121a589-1848-4341-a3a0-5a4627dee824\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:28.156045 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.155987    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c121a589-1848-4341-a3a0-5a4627dee824-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c121a589-1848-4341-a3a0-5a4627dee824\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:28.156045 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.156011    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c121a589-1848-4341-a3a0-5a4627dee824-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c121a589-1848-4341-a3a0-5a4627dee824\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:28.156045 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.156030    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c121a589-1848-4341-a3a0-5a4627dee824-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c121a589-1848-4341-a3a0-5a4627dee824\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:28.156256 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.156049    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c121a589-1848-4341-a3a0-5a4627dee824-web-config\") pod \"prometheus-k8s-0\" (UID: \"c121a589-1848-4341-a3a0-5a4627dee824\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:28.156256 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.156068    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c121a589-1848-4341-a3a0-5a4627dee824-config\") pod \"prometheus-k8s-0\" (UID: \"c121a589-1848-4341-a3a0-5a4627dee824\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:28.156256 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.156104    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c121a589-1848-4341-a3a0-5a4627dee824-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c121a589-1848-4341-a3a0-5a4627dee824\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:28.156256 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.156126    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c121a589-1848-4341-a3a0-5a4627dee824-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c121a589-1848-4341-a3a0-5a4627dee824\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:28.256903 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.256869    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c121a589-1848-4341-a3a0-5a4627dee824-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c121a589-1848-4341-a3a0-5a4627dee824\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:28.257069 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.256911    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c121a589-1848-4341-a3a0-5a4627dee824-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c121a589-1848-4341-a3a0-5a4627dee824\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:28.257069 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.256943    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c121a589-1848-4341-a3a0-5a4627dee824-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c121a589-1848-4341-a3a0-5a4627dee824\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:28.257069 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.256978    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c121a589-1848-4341-a3a0-5a4627dee824-config-out\") pod \"prometheus-k8s-0\" (UID: \"c121a589-1848-4341-a3a0-5a4627dee824\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:28.257069 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.256996    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c121a589-1848-4341-a3a0-5a4627dee824-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c121a589-1848-4341-a3a0-5a4627dee824\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:28.257069 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.257010    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c121a589-1848-4341-a3a0-5a4627dee824-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c121a589-1848-4341-a3a0-5a4627dee824\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:28.257069 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.257027    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c121a589-1848-4341-a3a0-5a4627dee824-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c121a589-1848-4341-a3a0-5a4627dee824\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:28.257069 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.257045    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c121a589-1848-4341-a3a0-5a4627dee824-web-config\") pod \"prometheus-k8s-0\" (UID: \"c121a589-1848-4341-a3a0-5a4627dee824\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:28.257069 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.257071    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c121a589-1848-4341-a3a0-5a4627dee824-config\") pod \"prometheus-k8s-0\" (UID: \"c121a589-1848-4341-a3a0-5a4627dee824\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:28.257479 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.257097    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c121a589-1848-4341-a3a0-5a4627dee824-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c121a589-1848-4341-a3a0-5a4627dee824\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:28.257479 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.257121    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c121a589-1848-4341-a3a0-5a4627dee824-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c121a589-1848-4341-a3a0-5a4627dee824\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:28.257479 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.257145    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c121a589-1848-4341-a3a0-5a4627dee824-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c121a589-1848-4341-a3a0-5a4627dee824\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:28.257479 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.257173    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c121a589-1848-4341-a3a0-5a4627dee824-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c121a589-1848-4341-a3a0-5a4627dee824\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:28.257479 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.257201    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c121a589-1848-4341-a3a0-5a4627dee824-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c121a589-1848-4341-a3a0-5a4627dee824\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:28.257479 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.257229    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c121a589-1848-4341-a3a0-5a4627dee824-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c121a589-1848-4341-a3a0-5a4627dee824\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:28.257479 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.257258    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c121a589-1848-4341-a3a0-5a4627dee824-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c121a589-1848-4341-a3a0-5a4627dee824\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:28.257479 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.257281    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c121a589-1848-4341-a3a0-5a4627dee824-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c121a589-1848-4341-a3a0-5a4627dee824\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:28.257479 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.257313    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c121a589-1848-4341-a3a0-5a4627dee824-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c121a589-1848-4341-a3a0-5a4627dee824\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:28.257479 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.257339    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvbhf\" (UniqueName: \"kubernetes.io/projected/c121a589-1848-4341-a3a0-5a4627dee824-kube-api-access-dvbhf\") pod \"prometheus-k8s-0\" (UID: \"c121a589-1848-4341-a3a0-5a4627dee824\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:28.257479 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.257361    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c121a589-1848-4341-a3a0-5a4627dee824-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c121a589-1848-4341-a3a0-5a4627dee824\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:28.258055 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.257758    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c121a589-1848-4341-a3a0-5a4627dee824-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c121a589-1848-4341-a3a0-5a4627dee824\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:28.258055 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.257871    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c121a589-1848-4341-a3a0-5a4627dee824-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c121a589-1848-4341-a3a0-5a4627dee824\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:28.258380 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.258356    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c121a589-1848-4341-a3a0-5a4627dee824-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c121a589-1848-4341-a3a0-5a4627dee824\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:28.259987 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.259961    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c121a589-1848-4341-a3a0-5a4627dee824-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c121a589-1848-4341-a3a0-5a4627dee824\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:28.260112 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.260005    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c121a589-1848-4341-a3a0-5a4627dee824-config-out\") pod \"prometheus-k8s-0\" (UID: \"c121a589-1848-4341-a3a0-5a4627dee824\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:28.260112 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.260097    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c121a589-1848-4341-a3a0-5a4627dee824-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c121a589-1848-4341-a3a0-5a4627dee824\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:28.260225 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.260190    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c121a589-1848-4341-a3a0-5a4627dee824-web-config\") pod \"prometheus-k8s-0\" (UID: \"c121a589-1848-4341-a3a0-5a4627dee824\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:28.260280 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.260244    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c121a589-1848-4341-a3a0-5a4627dee824-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c121a589-1848-4341-a3a0-5a4627dee824\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:28.261398 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.260482    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c121a589-1848-4341-a3a0-5a4627dee824-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c121a589-1848-4341-a3a0-5a4627dee824\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:28.261398 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.260643    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c121a589-1848-4341-a3a0-5a4627dee824-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c121a589-1848-4341-a3a0-5a4627dee824\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:28.261398 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.261181    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c121a589-1848-4341-a3a0-5a4627dee824-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c121a589-1848-4341-a3a0-5a4627dee824\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:28.261398 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.261304    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c121a589-1848-4341-a3a0-5a4627dee824-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c121a589-1848-4341-a3a0-5a4627dee824\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:28.261398 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.261341    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c121a589-1848-4341-a3a0-5a4627dee824-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c121a589-1848-4341-a3a0-5a4627dee824\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:28.261721 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.261596    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c121a589-1848-4341-a3a0-5a4627dee824-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c121a589-1848-4341-a3a0-5a4627dee824\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:28.261721 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.261648    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c121a589-1848-4341-a3a0-5a4627dee824-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c121a589-1848-4341-a3a0-5a4627dee824\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:28.262423 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.262403    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c121a589-1848-4341-a3a0-5a4627dee824-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c121a589-1848-4341-a3a0-5a4627dee824\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:28.262492 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.262472    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c121a589-1848-4341-a3a0-5a4627dee824-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c121a589-1848-4341-a3a0-5a4627dee824\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:28.262746 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.262723    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/c121a589-1848-4341-a3a0-5a4627dee824-config\") pod \"prometheus-k8s-0\" (UID: \"c121a589-1848-4341-a3a0-5a4627dee824\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:28.263287 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.263261    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c121a589-1848-4341-a3a0-5a4627dee824-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c121a589-1848-4341-a3a0-5a4627dee824\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:28.269719 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.269694    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvbhf\" (UniqueName: \"kubernetes.io/projected/c121a589-1848-4341-a3a0-5a4627dee824-kube-api-access-dvbhf\") pod \"prometheus-k8s-0\" (UID: \"c121a589-1848-4341-a3a0-5a4627dee824\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:28.350988 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.350946    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:28.634223 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.634021    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 19:18:28.636800 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:18:28.636771    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc121a589_1848_4341_a3a0_5a4627dee824.slice/crio-9cf8978b01a589ed72882b696acf7c89fd80a3d62a6d3220472b8a065c8d2e5d WatchSource:0}: Error finding container 9cf8978b01a589ed72882b696acf7c89fd80a3d62a6d3220472b8a065c8d2e5d: Status 404 returned error can't find the container with id 9cf8978b01a589ed72882b696acf7c89fd80a3d62a6d3220472b8a065c8d2e5d
Sep 10 19:18:28.680924 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.680734    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c121a589-1848-4341-a3a0-5a4627dee824","Type":"ContainerStarted","Data":"9cf8978b01a589ed72882b696acf7c89fd80a3d62a6d3220472b8a065c8d2e5d"}
Sep 10 19:18:28.947762 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:28.947685    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a" path="/var/lib/kubelet/pods/f3dbc922-35d8-4cb0-a435-9b61a2ca7f3a/volumes"
Sep 10 19:18:29.686026 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:29.685939    2564 generic.go:356] "Generic (PLEG): container finished" podID="c121a589-1848-4341-a3a0-5a4627dee824" containerID="79818789cd41a37f1198f83c6d2719d2e2a53337e288bc3c0a9890a15a40ad94" exitCode=0
Sep 10 19:18:29.686026 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:29.686005    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c121a589-1848-4341-a3a0-5a4627dee824","Type":"ContainerDied","Data":"79818789cd41a37f1198f83c6d2719d2e2a53337e288bc3c0a9890a15a40ad94"}
Sep 10 19:18:30.695805 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:30.695771    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c121a589-1848-4341-a3a0-5a4627dee824","Type":"ContainerStarted","Data":"276cd6f097f49b9072e4252658cb87fc7763df8f1d015335419829df4417e642"}
Sep 10 19:18:30.696179 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:30.695835    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c121a589-1848-4341-a3a0-5a4627dee824","Type":"ContainerStarted","Data":"7e3e38bdaef8d6e4766839b924c562b7f9fbfb062dea0d3bd485491ba9aaee04"}
Sep 10 19:18:30.696179 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:30.695865    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c121a589-1848-4341-a3a0-5a4627dee824","Type":"ContainerStarted","Data":"fcd6d6b433ffb7f771e68fd9e18cc0ff68a7abcb03dc85f482efd59bc37a8d03"}
Sep 10 19:18:30.696179 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:30.695889    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c121a589-1848-4341-a3a0-5a4627dee824","Type":"ContainerStarted","Data":"2e1c8d8060b1ed500c012bd9be9cfdea48d2edf2c46cabeca260ba3e05ae0332"}
Sep 10 19:18:30.696179 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:30.695912    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c121a589-1848-4341-a3a0-5a4627dee824","Type":"ContainerStarted","Data":"384c7c012d1aa53b43675fc48037b1ba6be84452d8526b70ece846e476542f0b"}
Sep 10 19:18:30.696179 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:30.695943    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c121a589-1848-4341-a3a0-5a4627dee824","Type":"ContainerStarted","Data":"384207374cb0a9cd6f892b377aaa6fcdc169faa9bac31f37b0795f042df4ee41"}
Sep 10 19:18:30.730986 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:30.730921    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.7111492029999997 podStartE2EDuration="2.730897887s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.748684ms" isStatefulPod=false podCreationTimestamp="2026-09-10 19:18:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:18:30.728505152 +0000 UTC m=+166.329412402" watchObservedRunningTime="2026-09-10 19:18:30.730897887 +0000 UTC m=+166.331805127"
Sep 10 19:18:30.735049 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:30.735021    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:18:30.914049 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:30.914019    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:18:30.965115 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:30.965029    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:18:31.514480 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:31.514425    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:18:31.603238 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:31.603192    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:18:33.351364 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:33.351310    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:41.781445 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:41.781350    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:18:41.835828 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:41.835787    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:18:41.890145 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:18:41.890106    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:19:03.623226 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:19:03.623182    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:19:03.674330 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:19:03.674293    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:19:14.063955 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:19:14.063912    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:19:28.351768 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:19:28.351726    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:19:28.367360 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:19:28.367332    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:19:28.923642 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:19:28.923592    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:19:37.328908 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:19:37.328872    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:19:37.382483 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:19:37.382449    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:19:49.249436 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:19:49.249398    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:19:49.350507 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:19:49.350444    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:19:51.533204 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:19:51.533164    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:19:51.598993 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:19:51.598963    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:19:51.675842 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:19:51.675797    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:19:52.290716 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:19:52.290677    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:19:52.345997 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:19:52.345962    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:20:02.164103 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:20:02.164064    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:20:02.219213 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:20:02.219173    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:20:02.281808 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:20:02.281774    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:20:05.011880 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:20:05.011835    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:20:05.113199 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:20:05.113164    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:20:05.165007 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:20:05.164967    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:20:36.581527 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:20:36.581479    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:20:36.794313 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:20:36.794274    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:20:36.887932 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:20:36.887840    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:20:44.828961 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:20:44.828926    2564 kubelet.go:1648] "Image garbage collection succeeded"
Sep 10 19:21:03.325659 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:03.325602    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:04.648586 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:04.648550    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:06.924180 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:06.924143    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:07.024221 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:07.024187    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:08.746500 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:08.746444    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:08.837130 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:08.837093    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:09.157581 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:09.157491    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:09.249073 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:09.249038    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:11.578247 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:11.578213    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:11.999838 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:11.999807    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:13.912094 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:13.912059    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:13.971983 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:13.971950    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:18.933476 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:18.933440    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:18.989666 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:18.989632    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:19.201511 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:19.201429    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:21.625226 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:21.625184    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-ntj7p"]
Sep 10 19:21:21.628311 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:21.628287    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-ntj7p"
Sep 10 19:21:21.630723 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:21.630696    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 10 19:21:21.630832 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:21.630700    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 10 19:21:21.631478 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:21.631458    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-2h9sw\""
Sep 10 19:21:21.638027 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:21.638003    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-ntj7p"]
Sep 10 19:21:21.661391 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:21.661360    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8bx2\" (UniqueName: \"kubernetes.io/projected/9ee9accc-1fd2-47f0-8e92-fa2a2818e367-kube-api-access-r8bx2\") pod \"cert-manager-55b4d954d5-ntj7p\" (UID: \"9ee9accc-1fd2-47f0-8e92-fa2a2818e367\") " pod="cert-manager/cert-manager-55b4d954d5-ntj7p"
Sep 10 19:21:21.661544 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:21.661402    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9ee9accc-1fd2-47f0-8e92-fa2a2818e367-bound-sa-token\") pod \"cert-manager-55b4d954d5-ntj7p\" (UID: \"9ee9accc-1fd2-47f0-8e92-fa2a2818e367\") " pod="cert-manager/cert-manager-55b4d954d5-ntj7p"
Sep 10 19:21:21.761915 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:21.761881    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9ee9accc-1fd2-47f0-8e92-fa2a2818e367-bound-sa-token\") pod \"cert-manager-55b4d954d5-ntj7p\" (UID: \"9ee9accc-1fd2-47f0-8e92-fa2a2818e367\") " pod="cert-manager/cert-manager-55b4d954d5-ntj7p"
Sep 10 19:21:21.762046 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:21.761969    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8bx2\" (UniqueName: \"kubernetes.io/projected/9ee9accc-1fd2-47f0-8e92-fa2a2818e367-kube-api-access-r8bx2\") pod \"cert-manager-55b4d954d5-ntj7p\" (UID: \"9ee9accc-1fd2-47f0-8e92-fa2a2818e367\") " pod="cert-manager/cert-manager-55b4d954d5-ntj7p"
Sep 10 19:21:21.770900 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:21.770868    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9ee9accc-1fd2-47f0-8e92-fa2a2818e367-bound-sa-token\") pod \"cert-manager-55b4d954d5-ntj7p\" (UID: \"9ee9accc-1fd2-47f0-8e92-fa2a2818e367\") " pod="cert-manager/cert-manager-55b4d954d5-ntj7p"
Sep 10 19:21:21.771010 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:21.770984    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8bx2\" (UniqueName: \"kubernetes.io/projected/9ee9accc-1fd2-47f0-8e92-fa2a2818e367-kube-api-access-r8bx2\") pod \"cert-manager-55b4d954d5-ntj7p\" (UID: \"9ee9accc-1fd2-47f0-8e92-fa2a2818e367\") " pod="cert-manager/cert-manager-55b4d954d5-ntj7p"
Sep 10 19:21:21.938424 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:21.938331    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-ntj7p"
Sep 10 19:21:22.207879 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:22.207791    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-ntj7p"]
Sep 10 19:21:22.210045 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:21:22.210009    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9ee9accc_1fd2_47f0_8e92_fa2a2818e367.slice/crio-e68c01b0bd981977a0d401f5f3018437472a5531e68dc8198d9a1770502a20ec WatchSource:0}: Error finding container e68c01b0bd981977a0d401f5f3018437472a5531e68dc8198d9a1770502a20ec: Status 404 returned error can't find the container with id e68c01b0bd981977a0d401f5f3018437472a5531e68dc8198d9a1770502a20ec
Sep 10 19:21:22.211990 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:22.211972    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 19:21:22.303129 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:22.303092    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-ntj7p" event={"ID":"9ee9accc-1fd2-47f0-8e92-fa2a2818e367","Type":"ContainerStarted","Data":"e68c01b0bd981977a0d401f5f3018437472a5531e68dc8198d9a1770502a20ec"}
Sep 10 19:21:25.317422 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:25.317387    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-ntj7p" event={"ID":"9ee9accc-1fd2-47f0-8e92-fa2a2818e367","Type":"ContainerStarted","Data":"7364b521b329647be60c3490be924176b648439e19f0b7b7297a0d51d5e86947"}
Sep 10 19:21:25.335889 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:25.335836    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-ntj7p" podStartSLOduration=1.331704565 podStartE2EDuration="4.335821844s" totalImagesPullingTime="3.004117279s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:21:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:21:25.335205904 +0000 UTC m=+340.936113144" watchObservedRunningTime="2026-09-10 19:21:25.335821844 +0000 UTC m=+340.936729084"
Sep 10 19:21:31.656964 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:31.656924    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:31.754754 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:31.754716    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:33.476128 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:33.476087    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-s25nx"]
Sep 10 19:21:33.479017 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:33.478996    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-s25nx"
Sep 10 19:21:33.481481 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:33.481456    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 10 19:21:33.482370 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:33.482354    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 10 19:21:33.482426 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:33.482375    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-znv6l\""
Sep 10 19:21:33.487733 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:33.487710    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-s25nx"]
Sep 10 19:21:33.558777 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:33.558742    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ac59f2f5-eeae-4b22-a3e5-4d83179e9f0c-tmp\") pod \"openshift-lws-operator-fd8ccff4c-s25nx\" (UID: \"ac59f2f5-eeae-4b22-a3e5-4d83179e9f0c\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-s25nx"
Sep 10 19:21:33.558948 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:33.558796    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59kn5\" (UniqueName: \"kubernetes.io/projected/ac59f2f5-eeae-4b22-a3e5-4d83179e9f0c-kube-api-access-59kn5\") pod \"openshift-lws-operator-fd8ccff4c-s25nx\" (UID: \"ac59f2f5-eeae-4b22-a3e5-4d83179e9f0c\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-s25nx"
Sep 10 19:21:33.660105 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:33.660063    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ac59f2f5-eeae-4b22-a3e5-4d83179e9f0c-tmp\") pod \"openshift-lws-operator-fd8ccff4c-s25nx\" (UID: \"ac59f2f5-eeae-4b22-a3e5-4d83179e9f0c\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-s25nx"
Sep 10 19:21:33.660307 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:33.660129    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59kn5\" (UniqueName: \"kubernetes.io/projected/ac59f2f5-eeae-4b22-a3e5-4d83179e9f0c-kube-api-access-59kn5\") pod \"openshift-lws-operator-fd8ccff4c-s25nx\" (UID: \"ac59f2f5-eeae-4b22-a3e5-4d83179e9f0c\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-s25nx"
Sep 10 19:21:33.660503 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:33.660480    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ac59f2f5-eeae-4b22-a3e5-4d83179e9f0c-tmp\") pod \"openshift-lws-operator-fd8ccff4c-s25nx\" (UID: \"ac59f2f5-eeae-4b22-a3e5-4d83179e9f0c\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-s25nx"
Sep 10 19:21:33.669162 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:33.669136    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-59kn5\" (UniqueName: \"kubernetes.io/projected/ac59f2f5-eeae-4b22-a3e5-4d83179e9f0c-kube-api-access-59kn5\") pod \"openshift-lws-operator-fd8ccff4c-s25nx\" (UID: \"ac59f2f5-eeae-4b22-a3e5-4d83179e9f0c\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-s25nx"
Sep 10 19:21:33.789033 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:33.788997    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-s25nx"
Sep 10 19:21:34.059763 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:34.059735    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-s25nx"]
Sep 10 19:21:34.062058 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:21:34.062033    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podac59f2f5_eeae_4b22_a3e5_4d83179e9f0c.slice/crio-b17321e708ee8fa3f3adb0d6ed1c2cee85e811ade6f84cf3936df5612114240b WatchSource:0}: Error finding container b17321e708ee8fa3f3adb0d6ed1c2cee85e811ade6f84cf3936df5612114240b: Status 404 returned error can't find the container with id b17321e708ee8fa3f3adb0d6ed1c2cee85e811ade6f84cf3936df5612114240b
Sep 10 19:21:34.351276 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:34.351199    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-s25nx" event={"ID":"ac59f2f5-eeae-4b22-a3e5-4d83179e9f0c","Type":"ContainerStarted","Data":"b17321e708ee8fa3f3adb0d6ed1c2cee85e811ade6f84cf3936df5612114240b"}
Sep 10 19:21:37.365858 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:37.365775    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-s25nx" event={"ID":"ac59f2f5-eeae-4b22-a3e5-4d83179e9f0c","Type":"ContainerStarted","Data":"5eb587937d0acfc65627e48a471abb6440502114b443410634270b31599d5f12"}
Sep 10 19:21:37.388670 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:37.388580    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-s25nx" podStartSLOduration=1.958460848 podStartE2EDuration="4.388553066s" totalImagesPullingTime="2.430092218s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:21:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:21:37.38480214 +0000 UTC m=+352.985709382" watchObservedRunningTime="2026-09-10 19:21:37.388553066 +0000 UTC m=+352.989460308"
Sep 10 19:21:38.070220 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:38.070175    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:38.170165 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:38.170123    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:40.075944 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:40.075904    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:52.993474 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:52.993433    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:53.089296 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:53.089259    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:53.598227 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:53.598188    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:53.813564 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:53.813522    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:53.901730 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:53.901642    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:54.049255 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:54.049220    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-55556678b8-vpjt8"]
Sep 10 19:21:54.123197 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:54.123163    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-55556678b8-vpjt8"]
Sep 10 19:21:54.123379 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:54.123266    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-55556678b8-vpjt8"
Sep 10 19:21:54.125993 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:54.125957    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 10 19:21:54.125993 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:54.125981    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 10 19:21:54.126169 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:54.126085    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 10 19:21:54.126216 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:54.126161    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-4z6tt\""
Sep 10 19:21:54.126262 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:54.126226    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 10 19:21:54.170969 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:54.170877    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:54.245685 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:54.245645    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/99764670-0b23-4b19-b3fe-cffb58fd825b-tmp\") pod \"opendatahub-operator-controller-manager-55556678b8-vpjt8\" (UID: \"99764670-0b23-4b19-b3fe-cffb58fd825b\") " pod="opendatahub/opendatahub-operator-controller-manager-55556678b8-vpjt8"
Sep 10 19:21:54.245874 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:54.245697    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gh8s4\" (UniqueName: \"kubernetes.io/projected/99764670-0b23-4b19-b3fe-cffb58fd825b-kube-api-access-gh8s4\") pod \"opendatahub-operator-controller-manager-55556678b8-vpjt8\" (UID: \"99764670-0b23-4b19-b3fe-cffb58fd825b\") " pod="opendatahub/opendatahub-operator-controller-manager-55556678b8-vpjt8"
Sep 10 19:21:54.245874 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:54.245728    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/99764670-0b23-4b19-b3fe-cffb58fd825b-webhook-cert\") pod \"opendatahub-operator-controller-manager-55556678b8-vpjt8\" (UID: \"99764670-0b23-4b19-b3fe-cffb58fd825b\") " pod="opendatahub/opendatahub-operator-controller-manager-55556678b8-vpjt8"
Sep 10 19:21:54.245874 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:54.245779    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/99764670-0b23-4b19-b3fe-cffb58fd825b-manifests\") pod \"opendatahub-operator-controller-manager-55556678b8-vpjt8\" (UID: \"99764670-0b23-4b19-b3fe-cffb58fd825b\") " pod="opendatahub/opendatahub-operator-controller-manager-55556678b8-vpjt8"
Sep 10 19:21:54.245874 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:54.245812    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/99764670-0b23-4b19-b3fe-cffb58fd825b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-55556678b8-vpjt8\" (UID: \"99764670-0b23-4b19-b3fe-cffb58fd825b\") " pod="opendatahub/opendatahub-operator-controller-manager-55556678b8-vpjt8"
Sep 10 19:21:54.273882 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:54.273840    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:54.346256 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:54.346220    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/99764670-0b23-4b19-b3fe-cffb58fd825b-manifests\") pod \"opendatahub-operator-controller-manager-55556678b8-vpjt8\" (UID: \"99764670-0b23-4b19-b3fe-cffb58fd825b\") " pod="opendatahub/opendatahub-operator-controller-manager-55556678b8-vpjt8"
Sep 10 19:21:54.346455 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:54.346284    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/99764670-0b23-4b19-b3fe-cffb58fd825b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-55556678b8-vpjt8\" (UID: \"99764670-0b23-4b19-b3fe-cffb58fd825b\") " pod="opendatahub/opendatahub-operator-controller-manager-55556678b8-vpjt8"
Sep 10 19:21:54.346455 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:54.346313    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/99764670-0b23-4b19-b3fe-cffb58fd825b-tmp\") pod \"opendatahub-operator-controller-manager-55556678b8-vpjt8\" (UID: \"99764670-0b23-4b19-b3fe-cffb58fd825b\") " pod="opendatahub/opendatahub-operator-controller-manager-55556678b8-vpjt8"
Sep 10 19:21:54.346455 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:54.346352    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gh8s4\" (UniqueName: \"kubernetes.io/projected/99764670-0b23-4b19-b3fe-cffb58fd825b-kube-api-access-gh8s4\") pod \"opendatahub-operator-controller-manager-55556678b8-vpjt8\" (UID: \"99764670-0b23-4b19-b3fe-cffb58fd825b\") " pod="opendatahub/opendatahub-operator-controller-manager-55556678b8-vpjt8"
Sep 10 19:21:54.346455 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:54.346388    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/99764670-0b23-4b19-b3fe-cffb58fd825b-webhook-cert\") pod \"opendatahub-operator-controller-manager-55556678b8-vpjt8\" (UID: \"99764670-0b23-4b19-b3fe-cffb58fd825b\") " pod="opendatahub/opendatahub-operator-controller-manager-55556678b8-vpjt8"
Sep 10 19:21:54.346747 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:54.346723    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/99764670-0b23-4b19-b3fe-cffb58fd825b-manifests\") pod \"opendatahub-operator-controller-manager-55556678b8-vpjt8\" (UID: \"99764670-0b23-4b19-b3fe-cffb58fd825b\") " pod="opendatahub/opendatahub-operator-controller-manager-55556678b8-vpjt8"
Sep 10 19:21:54.348727 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:54.348698    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/99764670-0b23-4b19-b3fe-cffb58fd825b-tmp\") pod \"opendatahub-operator-controller-manager-55556678b8-vpjt8\" (UID: \"99764670-0b23-4b19-b3fe-cffb58fd825b\") " pod="opendatahub/opendatahub-operator-controller-manager-55556678b8-vpjt8"
Sep 10 19:21:54.348916 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:54.348899    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/99764670-0b23-4b19-b3fe-cffb58fd825b-webhook-cert\") pod \"opendatahub-operator-controller-manager-55556678b8-vpjt8\" (UID: \"99764670-0b23-4b19-b3fe-cffb58fd825b\") " pod="opendatahub/opendatahub-operator-controller-manager-55556678b8-vpjt8"
Sep 10 19:21:54.348969 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:54.348905    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/99764670-0b23-4b19-b3fe-cffb58fd825b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-55556678b8-vpjt8\" (UID: \"99764670-0b23-4b19-b3fe-cffb58fd825b\") " pod="opendatahub/opendatahub-operator-controller-manager-55556678b8-vpjt8"
Sep 10 19:21:54.356587 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:54.356560    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gh8s4\" (UniqueName: \"kubernetes.io/projected/99764670-0b23-4b19-b3fe-cffb58fd825b-kube-api-access-gh8s4\") pod \"opendatahub-operator-controller-manager-55556678b8-vpjt8\" (UID: \"99764670-0b23-4b19-b3fe-cffb58fd825b\") " pod="opendatahub/opendatahub-operator-controller-manager-55556678b8-vpjt8"
Sep 10 19:21:54.374517 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:54.374478    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:54.432612 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:54.432531    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-55556678b8-vpjt8"
Sep 10 19:21:54.474686 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:54.474585    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:54.573030 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:54.572998    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:54.651391 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:54.651363    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:54.711783 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:54.711707    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:54.731678 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:54.731653    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-55556678b8-vpjt8"]
Sep 10 19:21:54.733634 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:21:54.733591    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99764670_0b23_4b19_b3fe_cffb58fd825b.slice/crio-8b83e9b6bc053fed999c42307f6fa7945393e6e0850a26f958f2877f9e6dbb00 WatchSource:0}: Error finding container 8b83e9b6bc053fed999c42307f6fa7945393e6e0850a26f958f2877f9e6dbb00: Status 404 returned error can't find the container with id 8b83e9b6bc053fed999c42307f6fa7945393e6e0850a26f958f2877f9e6dbb00
Sep 10 19:21:54.812470 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:54.812423    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:55.442244 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:55.442200    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-55556678b8-vpjt8" event={"ID":"99764670-0b23-4b19-b3fe-cffb58fd825b","Type":"ContainerStarted","Data":"8b83e9b6bc053fed999c42307f6fa7945393e6e0850a26f958f2877f9e6dbb00"}
Sep 10 19:21:56.146691 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:56.146646    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:56.253361 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:56.253304    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:58.457812 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:58.457760    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-55556678b8-vpjt8" event={"ID":"99764670-0b23-4b19-b3fe-cffb58fd825b","Type":"ContainerStarted","Data":"6bd20bc30be4461f5b30ce78b325cccdf5a9bfd727eec38f61cc04a51e24441d"}
Sep 10 19:21:59.463854 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:59.463767    2564 generic.go:356] "Generic (PLEG): container finished" podID="99764670-0b23-4b19-b3fe-cffb58fd825b" containerID="6bd20bc30be4461f5b30ce78b325cccdf5a9bfd727eec38f61cc04a51e24441d" exitCode=0
Sep 10 19:21:59.464222 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:21:59.463853    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-55556678b8-vpjt8" event={"ID":"99764670-0b23-4b19-b3fe-cffb58fd825b","Type":"ContainerDied","Data":"6bd20bc30be4461f5b30ce78b325cccdf5a9bfd727eec38f61cc04a51e24441d"}
Sep 10 19:22:00.470148 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:00.470112    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-55556678b8-vpjt8" event={"ID":"99764670-0b23-4b19-b3fe-cffb58fd825b","Type":"ContainerStarted","Data":"3e945aab5417f584f6712724796ef306da73985b9fff2789a67dc3230c298df6"}
Sep 10 19:22:00.470527 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:00.470257    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-55556678b8-vpjt8"
Sep 10 19:22:00.492154 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:00.492097    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-55556678b8-vpjt8" podStartSLOduration=1.782060153 podStartE2EDuration="6.492073491s" totalImagesPullingTime="3.570231742s" totalInitContainerRuntime="1.139781596s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:21:54 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:22:00.490502479 +0000 UTC m=+376.091409720" watchObservedRunningTime="2026-09-10 19:22:00.492073491 +0000 UTC m=+376.092980731"
Sep 10 19:22:05.713580 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:05.713543    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:05.818262 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:05.818230    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:05.914102 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:05.914067    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:06.018751 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:06.018711    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:06.907260 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:06.907223    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:07.007687 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:07.007656    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:07.378963 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:07.378924    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:07.478860 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:07.478826    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:07.877489 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:07.877453    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:07.976918 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:07.976885    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:11.477060 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:11.477026    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-55556678b8-vpjt8"
Sep 10 19:22:13.201347 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:13.201304    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:13.301143 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:13.301096    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:18.859155 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:18.859114    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:18.960964 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:18.960927    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:19.016677 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:19.016638    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:23.851640 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:23.851581    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:23.991764 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:23.991729    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-64954c75b6-htzd4"]
Sep 10 19:22:23.995036 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:23.995017    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-64954c75b6-htzd4"
Sep 10 19:22:23.997535 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:23.997507    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 10 19:22:23.997665 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:23.997580    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 10 19:22:23.997665 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:23.997599    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 10 19:22:23.997665 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:23.997635    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 10 19:22:23.998048 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:23.998033    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-9j2pj\""
Sep 10 19:22:24.005114 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:24.005091    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-64954c75b6-htzd4"]
Sep 10 19:22:24.087206 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:24.087160    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/518a84e9-05e6-43d2-b478-1a2885a29dce-tmp\") pod \"kube-auth-proxy-64954c75b6-htzd4\" (UID: \"518a84e9-05e6-43d2-b478-1a2885a29dce\") " pod="openshift-ingress/kube-auth-proxy-64954c75b6-htzd4"
Sep 10 19:22:24.087387 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:24.087243    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9gll\" (UniqueName: \"kubernetes.io/projected/518a84e9-05e6-43d2-b478-1a2885a29dce-kube-api-access-x9gll\") pod \"kube-auth-proxy-64954c75b6-htzd4\" (UID: \"518a84e9-05e6-43d2-b478-1a2885a29dce\") " pod="openshift-ingress/kube-auth-proxy-64954c75b6-htzd4"
Sep 10 19:22:24.087387 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:24.087344    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/518a84e9-05e6-43d2-b478-1a2885a29dce-tls-certs\") pod \"kube-auth-proxy-64954c75b6-htzd4\" (UID: \"518a84e9-05e6-43d2-b478-1a2885a29dce\") " pod="openshift-ingress/kube-auth-proxy-64954c75b6-htzd4"
Sep 10 19:22:24.179667 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:24.177468    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-48fv6"]
Sep 10 19:22:24.181806 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:24.181762    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-48fv6"
Sep 10 19:22:24.184288 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:24.184171    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-dcmk4\""
Sep 10 19:22:24.184288 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:24.184195    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Sep 10 19:22:24.187119 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:24.187092    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-48fv6"]
Sep 10 19:22:24.188028 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:24.188002    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9gll\" (UniqueName: \"kubernetes.io/projected/518a84e9-05e6-43d2-b478-1a2885a29dce-kube-api-access-x9gll\") pod \"kube-auth-proxy-64954c75b6-htzd4\" (UID: \"518a84e9-05e6-43d2-b478-1a2885a29dce\") " pod="openshift-ingress/kube-auth-proxy-64954c75b6-htzd4"
Sep 10 19:22:24.188138 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:24.188103    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/518a84e9-05e6-43d2-b478-1a2885a29dce-tls-certs\") pod \"kube-auth-proxy-64954c75b6-htzd4\" (UID: \"518a84e9-05e6-43d2-b478-1a2885a29dce\") " pod="openshift-ingress/kube-auth-proxy-64954c75b6-htzd4"
Sep 10 19:22:24.188200 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:24.188156    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/518a84e9-05e6-43d2-b478-1a2885a29dce-tmp\") pod \"kube-auth-proxy-64954c75b6-htzd4\" (UID: \"518a84e9-05e6-43d2-b478-1a2885a29dce\") " pod="openshift-ingress/kube-auth-proxy-64954c75b6-htzd4"
Sep 10 19:22:24.188256 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:22:24.188213    2564 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Sep 10 19:22:24.188465 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:22:24.188291    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/518a84e9-05e6-43d2-b478-1a2885a29dce-tls-certs podName:518a84e9-05e6-43d2-b478-1a2885a29dce nodeName:}" failed. No retries permitted until 2026-09-10 19:22:24.688254173 +0000 UTC m=+400.289161397 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/518a84e9-05e6-43d2-b478-1a2885a29dce-tls-certs") pod "kube-auth-proxy-64954c75b6-htzd4" (UID: "518a84e9-05e6-43d2-b478-1a2885a29dce") : secret "kube-auth-proxy-tls" not found
Sep 10 19:22:24.190893 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:24.190868    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/518a84e9-05e6-43d2-b478-1a2885a29dce-tmp\") pod \"kube-auth-proxy-64954c75b6-htzd4\" (UID: \"518a84e9-05e6-43d2-b478-1a2885a29dce\") " pod="openshift-ingress/kube-auth-proxy-64954c75b6-htzd4"
Sep 10 19:22:24.197089 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:24.197063    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9gll\" (UniqueName: \"kubernetes.io/projected/518a84e9-05e6-43d2-b478-1a2885a29dce-kube-api-access-x9gll\") pod \"kube-auth-proxy-64954c75b6-htzd4\" (UID: \"518a84e9-05e6-43d2-b478-1a2885a29dce\") " pod="openshift-ingress/kube-auth-proxy-64954c75b6-htzd4"
Sep 10 19:22:24.205756 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:24.205721    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:24.288921 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:24.288877    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfshk\" (UniqueName: \"kubernetes.io/projected/5c5b1c30-dce1-463a-9739-3cc9b306dfe5-kube-api-access-nfshk\") pod \"odh-model-controller-7998c67dc4-48fv6\" (UID: \"5c5b1c30-dce1-463a-9739-3cc9b306dfe5\") " pod="opendatahub/odh-model-controller-7998c67dc4-48fv6"
Sep 10 19:22:24.288921 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:24.288926    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5c5b1c30-dce1-463a-9739-3cc9b306dfe5-cert\") pod \"odh-model-controller-7998c67dc4-48fv6\" (UID: \"5c5b1c30-dce1-463a-9739-3cc9b306dfe5\") " pod="opendatahub/odh-model-controller-7998c67dc4-48fv6"
Sep 10 19:22:24.309103 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:24.309068    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:24.390356 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:24.390317    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5c5b1c30-dce1-463a-9739-3cc9b306dfe5-cert\") pod \"odh-model-controller-7998c67dc4-48fv6\" (UID: \"5c5b1c30-dce1-463a-9739-3cc9b306dfe5\") " pod="opendatahub/odh-model-controller-7998c67dc4-48fv6"
Sep 10 19:22:24.390526 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:24.390455    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfshk\" (UniqueName: \"kubernetes.io/projected/5c5b1c30-dce1-463a-9739-3cc9b306dfe5-kube-api-access-nfshk\") pod \"odh-model-controller-7998c67dc4-48fv6\" (UID: \"5c5b1c30-dce1-463a-9739-3cc9b306dfe5\") " pod="opendatahub/odh-model-controller-7998c67dc4-48fv6"
Sep 10 19:22:24.390526 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:22:24.390467    2564 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Sep 10 19:22:24.390673 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:22:24.390537    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c5b1c30-dce1-463a-9739-3cc9b306dfe5-cert podName:5c5b1c30-dce1-463a-9739-3cc9b306dfe5 nodeName:}" failed. No retries permitted until 2026-09-10 19:22:24.890518984 +0000 UTC m=+400.491426201 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/5c5b1c30-dce1-463a-9739-3cc9b306dfe5-cert") pod "odh-model-controller-7998c67dc4-48fv6" (UID: "5c5b1c30-dce1-463a-9739-3cc9b306dfe5") : secret "odh-model-controller-webhook-cert" not found
Sep 10 19:22:24.399999 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:24.399971    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfshk\" (UniqueName: \"kubernetes.io/projected/5c5b1c30-dce1-463a-9739-3cc9b306dfe5-kube-api-access-nfshk\") pod \"odh-model-controller-7998c67dc4-48fv6\" (UID: \"5c5b1c30-dce1-463a-9739-3cc9b306dfe5\") " pod="opendatahub/odh-model-controller-7998c67dc4-48fv6"
Sep 10 19:22:24.406535 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:24.406493    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:24.608898 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:24.608862    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:24.646161 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:24.646124    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-96fp6"]
Sep 10 19:22:24.649706 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:24.649682    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-96fp6"
Sep 10 19:22:24.652281 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:24.652261    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-rgxkh\""
Sep 10 19:22:24.652281 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:24.652267    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Sep 10 19:22:24.658119 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:24.658095    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-96fp6"]
Sep 10 19:22:24.693858 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:24.693823    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/518a84e9-05e6-43d2-b478-1a2885a29dce-tls-certs\") pod \"kube-auth-proxy-64954c75b6-htzd4\" (UID: \"518a84e9-05e6-43d2-b478-1a2885a29dce\") " pod="openshift-ingress/kube-auth-proxy-64954c75b6-htzd4"
Sep 10 19:22:24.693858 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:24.693867    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f4e21f1-9f35-4dda-9b62-f32d6b4a23c2-tls-certs\") pod \"model-serving-api-54d89bd5fc-96fp6\" (UID: \"6f4e21f1-9f35-4dda-9b62-f32d6b4a23c2\") " pod="opendatahub/model-serving-api-54d89bd5fc-96fp6"
Sep 10 19:22:24.694073 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:24.693891    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ksc7c\" (UniqueName: \"kubernetes.io/projected/6f4e21f1-9f35-4dda-9b62-f32d6b4a23c2-kube-api-access-ksc7c\") pod \"model-serving-api-54d89bd5fc-96fp6\" (UID: \"6f4e21f1-9f35-4dda-9b62-f32d6b4a23c2\") " pod="opendatahub/model-serving-api-54d89bd5fc-96fp6"
Sep 10 19:22:24.696253 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:24.696232    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/518a84e9-05e6-43d2-b478-1a2885a29dce-tls-certs\") pod \"kube-auth-proxy-64954c75b6-htzd4\" (UID: \"518a84e9-05e6-43d2-b478-1a2885a29dce\") " pod="openshift-ingress/kube-auth-proxy-64954c75b6-htzd4"
Sep 10 19:22:24.707232 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:24.707203    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:24.794424 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:24.794394    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f4e21f1-9f35-4dda-9b62-f32d6b4a23c2-tls-certs\") pod \"model-serving-api-54d89bd5fc-96fp6\" (UID: \"6f4e21f1-9f35-4dda-9b62-f32d6b4a23c2\") " pod="opendatahub/model-serving-api-54d89bd5fc-96fp6"
Sep 10 19:22:24.794570 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:24.794430    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ksc7c\" (UniqueName: \"kubernetes.io/projected/6f4e21f1-9f35-4dda-9b62-f32d6b4a23c2-kube-api-access-ksc7c\") pod \"model-serving-api-54d89bd5fc-96fp6\" (UID: \"6f4e21f1-9f35-4dda-9b62-f32d6b4a23c2\") " pod="opendatahub/model-serving-api-54d89bd5fc-96fp6"
Sep 10 19:22:24.794570 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:22:24.794540    2564 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Sep 10 19:22:24.794701 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:22:24.794606    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f4e21f1-9f35-4dda-9b62-f32d6b4a23c2-tls-certs podName:6f4e21f1-9f35-4dda-9b62-f32d6b4a23c2 nodeName:}" failed. No retries permitted until 2026-09-10 19:22:25.294591673 +0000 UTC m=+400.895498890 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/6f4e21f1-9f35-4dda-9b62-f32d6b4a23c2-tls-certs") pod "model-serving-api-54d89bd5fc-96fp6" (UID: "6f4e21f1-9f35-4dda-9b62-f32d6b4a23c2") : secret "model-serving-api-tls" not found
Sep 10 19:22:24.806208 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:24.806176    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ksc7c\" (UniqueName: \"kubernetes.io/projected/6f4e21f1-9f35-4dda-9b62-f32d6b4a23c2-kube-api-access-ksc7c\") pod \"model-serving-api-54d89bd5fc-96fp6\" (UID: \"6f4e21f1-9f35-4dda-9b62-f32d6b4a23c2\") " pod="opendatahub/model-serving-api-54d89bd5fc-96fp6"
Sep 10 19:22:24.895543 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:24.895444    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5c5b1c30-dce1-463a-9739-3cc9b306dfe5-cert\") pod \"odh-model-controller-7998c67dc4-48fv6\" (UID: \"5c5b1c30-dce1-463a-9739-3cc9b306dfe5\") " pod="opendatahub/odh-model-controller-7998c67dc4-48fv6"
Sep 10 19:22:24.896431 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:24.896400    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:24.897942 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:24.897924    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5c5b1c30-dce1-463a-9739-3cc9b306dfe5-cert\") pod \"odh-model-controller-7998c67dc4-48fv6\" (UID: \"5c5b1c30-dce1-463a-9739-3cc9b306dfe5\") " pod="opendatahub/odh-model-controller-7998c67dc4-48fv6"
Sep 10 19:22:24.905786 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:24.905763    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-64954c75b6-htzd4"
Sep 10 19:22:25.066346 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:25.066311    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:25.082080 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:25.082046    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ngx68"]
Sep 10 19:22:25.085395 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:25.085374    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ngx68"
Sep 10 19:22:25.087564 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:25.087544    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 10 19:22:25.087835 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:25.087820    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 10 19:22:25.087895 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:25.087877    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 10 19:22:25.087937 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:25.087918    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 10 19:22:25.088139 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:25.088122    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-c7zwg\""
Sep 10 19:22:25.106708 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:25.106678    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ngx68"]
Sep 10 19:22:25.108343 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:25.108316    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-48fv6"
Sep 10 19:22:25.126125 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:25.126095    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:25.177845 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:25.177816    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-64954c75b6-htzd4"]
Sep 10 19:22:25.180691 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:22:25.180660    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod518a84e9_05e6_43d2_b478_1a2885a29dce.slice/crio-2217e379d208600812d7b295d2925876409808fcf32bcfbc297f1e7e9b5f352d WatchSource:0}: Error finding container 2217e379d208600812d7b295d2925876409808fcf32bcfbc297f1e7e9b5f352d: Status 404 returned error can't find the container with id 2217e379d208600812d7b295d2925876409808fcf32bcfbc297f1e7e9b5f352d
Sep 10 19:22:25.199182 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:25.199152    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-ngx68\" (UID: \"fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ngx68"
Sep 10 19:22:25.199326 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:25.199192    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-ngx68\" (UID: \"fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ngx68"
Sep 10 19:22:25.199326 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:25.199220    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-ngx68\" (UID: \"fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ngx68"
Sep 10 19:22:25.199326 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:25.199253    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-ngx68\" (UID: \"fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ngx68"
Sep 10 19:22:25.199487 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:25.199378    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-ngx68\" (UID: \"fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ngx68"
Sep 10 19:22:25.199487 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:25.199404    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-ngx68\" (UID: \"fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ngx68"
Sep 10 19:22:25.199575 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:25.199487    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hh4cs\" (UniqueName: \"kubernetes.io/projected/fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de-kube-api-access-hh4cs\") pod \"istiod-openshift-gateway-7b7995d5cd-ngx68\" (UID: \"fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ngx68"
Sep 10 19:22:25.300582 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:25.300545    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-ngx68\" (UID: \"fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ngx68"
Sep 10 19:22:25.300582 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:25.300580    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-ngx68\" (UID: \"fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ngx68"
Sep 10 19:22:25.300918 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:25.300599    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-ngx68\" (UID: \"fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ngx68"
Sep 10 19:22:25.300918 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:25.300618    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-ngx68\" (UID: \"fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ngx68"
Sep 10 19:22:25.300918 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:25.300694    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f4e21f1-9f35-4dda-9b62-f32d6b4a23c2-tls-certs\") pod \"model-serving-api-54d89bd5fc-96fp6\" (UID: \"6f4e21f1-9f35-4dda-9b62-f32d6b4a23c2\") " pod="opendatahub/model-serving-api-54d89bd5fc-96fp6"
Sep 10 19:22:25.300918 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:25.300713    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-ngx68\" (UID: \"fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ngx68"
Sep 10 19:22:25.300918 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:25.300729    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-ngx68\" (UID: \"fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ngx68"
Sep 10 19:22:25.300918 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:25.300790    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hh4cs\" (UniqueName: \"kubernetes.io/projected/fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de-kube-api-access-hh4cs\") pod \"istiod-openshift-gateway-7b7995d5cd-ngx68\" (UID: \"fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ngx68"
Sep 10 19:22:25.301460 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:25.301433    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-ngx68\" (UID: \"fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ngx68"
Sep 10 19:22:25.303171 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:25.303142    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-ngx68\" (UID: \"fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ngx68"
Sep 10 19:22:25.303503 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:25.303481    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-ngx68\" (UID: \"fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ngx68"
Sep 10 19:22:25.303589 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:25.303510    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-ngx68\" (UID: \"fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ngx68"
Sep 10 19:22:25.303589 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:25.303544    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-ngx68\" (UID: \"fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ngx68"
Sep 10 19:22:25.303709 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:25.303690    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f4e21f1-9f35-4dda-9b62-f32d6b4a23c2-tls-certs\") pod \"model-serving-api-54d89bd5fc-96fp6\" (UID: \"6f4e21f1-9f35-4dda-9b62-f32d6b4a23c2\") " pod="opendatahub/model-serving-api-54d89bd5fc-96fp6"
Sep 10 19:22:25.309464 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:25.309440    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-ngx68\" (UID: \"fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ngx68"
Sep 10 19:22:25.309675 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:25.309645    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hh4cs\" (UniqueName: \"kubernetes.io/projected/fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de-kube-api-access-hh4cs\") pod \"istiod-openshift-gateway-7b7995d5cd-ngx68\" (UID: \"fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ngx68"
Sep 10 19:22:25.385542 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:25.381032    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-48fv6"]
Sep 10 19:22:25.395699 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:25.395670    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ngx68"
Sep 10 19:22:25.561289 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:25.561242    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-96fp6"
Sep 10 19:22:25.570362 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:25.570330    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-48fv6" event={"ID":"5c5b1c30-dce1-463a-9739-3cc9b306dfe5","Type":"ContainerStarted","Data":"a3bb8370e6589134f12273bf60b4f4346d5b4aa8a402a4f13bf5a4bb98bc57a1"}
Sep 10 19:22:25.571732 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:25.571706    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-64954c75b6-htzd4" event={"ID":"518a84e9-05e6-43d2-b478-1a2885a29dce","Type":"ContainerStarted","Data":"2217e379d208600812d7b295d2925876409808fcf32bcfbc297f1e7e9b5f352d"}
Sep 10 19:22:25.678378 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:25.678350    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ngx68"]
Sep 10 19:22:25.680602 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:22:25.680576    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfff4cb9d_7eb3_4d4c_8531_6bd7b6da33de.slice/crio-4a8f634a9027869166d1859394c12ad488ebb39889b251af2e5b8dd144909659 WatchSource:0}: Error finding container 4a8f634a9027869166d1859394c12ad488ebb39889b251af2e5b8dd144909659: Status 404 returned error can't find the container with id 4a8f634a9027869166d1859394c12ad488ebb39889b251af2e5b8dd144909659
Sep 10 19:22:25.842999 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:25.842918    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-96fp6"]
Sep 10 19:22:25.845661 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:22:25.845613    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f4e21f1_9f35_4dda_9b62_f32d6b4a23c2.slice/crio-4d2bd4f8de232ba19b66aa4ff56904d6f094bab14ed69396c9c171accb469966 WatchSource:0}: Error finding container 4d2bd4f8de232ba19b66aa4ff56904d6f094bab14ed69396c9c171accb469966: Status 404 returned error can't find the container with id 4d2bd4f8de232ba19b66aa4ff56904d6f094bab14ed69396c9c171accb469966
Sep 10 19:22:26.089830 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:26.089747    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:26.591826 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:26.588050    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ngx68" event={"ID":"fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de","Type":"ContainerStarted","Data":"4a8f634a9027869166d1859394c12ad488ebb39889b251af2e5b8dd144909659"}
Sep 10 19:22:26.593908 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:26.593866    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-96fp6" event={"ID":"6f4e21f1-9f35-4dda-9b62-f32d6b4a23c2","Type":"ContainerStarted","Data":"4d2bd4f8de232ba19b66aa4ff56904d6f094bab14ed69396c9c171accb469966"}
Sep 10 19:22:28.397834 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:28.397789    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:28.464437 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:28.464392    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:29.767431 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:29.767393    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:29.828304 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:29.828258    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:31.394914 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:31.394865    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:22:31.395261 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:31.394955    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:22:31.621118 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:31.621033    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ngx68" event={"ID":"fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de","Type":"ContainerStarted","Data":"c769e6e2136fd922a667ed8ad7379bac0a2c315ca9a691367bf3be049f48f9a5"}
Sep 10 19:22:31.621279 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:31.621185    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ngx68"
Sep 10 19:22:31.623223 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:31.623118    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-48fv6" event={"ID":"5c5b1c30-dce1-463a-9739-3cc9b306dfe5","Type":"ContainerStarted","Data":"689e146d83b2f12ec6ab175ace9bae7f36f3002c81d3040f2bc388ff0a5f0e35"}
Sep 10 19:22:31.623223 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:31.623191    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-48fv6"
Sep 10 19:22:31.625270 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:31.625245    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-64954c75b6-htzd4" event={"ID":"518a84e9-05e6-43d2-b478-1a2885a29dce","Type":"ContainerStarted","Data":"7ce6d66dd0f7f3d85f51d8cf0e1a9c51c7b0ebb287b9936092ad0834d61345fe"}
Sep 10 19:22:31.626872 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:31.626847    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-96fp6" event={"ID":"6f4e21f1-9f35-4dda-9b62-f32d6b4a23c2","Type":"ContainerStarted","Data":"18afc452c46eed174086f4d455c86a4ba446b50d91160912c800698badf4724f"}
Sep 10 19:22:31.627070 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:31.627050    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-96fp6"
Sep 10 19:22:31.680184 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:31.680120    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ngx68" podStartSLOduration=0.968291863 podStartE2EDuration="6.680102834s" totalImagesPullingTime="5.711810971s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:22:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:22:31.679171884 +0000 UTC m=+407.280079126" watchObservedRunningTime="2026-09-10 19:22:31.680102834 +0000 UTC m=+407.281010073"
Sep 10 19:22:31.706943 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:31.706880    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-64954c75b6-htzd4" podStartSLOduration=2.512453538 podStartE2EDuration="8.70685966s" totalImagesPullingTime="6.194406122s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:22:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:22:31.705518403 +0000 UTC m=+407.306425643" watchObservedRunningTime="2026-09-10 19:22:31.70685966 +0000 UTC m=+407.307766899"
Sep 10 19:22:31.730447 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:31.730382    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-96fp6" podStartSLOduration=2.197173914 podStartE2EDuration="7.730365221s" totalImagesPullingTime="5.533191307s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:22:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:22:31.728200263 +0000 UTC m=+407.329107504" watchObservedRunningTime="2026-09-10 19:22:31.730365221 +0000 UTC m=+407.331272460"
Sep 10 19:22:31.761955 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:31.761893    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-48fv6" podStartSLOduration=1.749300833 podStartE2EDuration="7.761871509s" totalImagesPullingTime="6.012570676s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:22:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:22:31.759794593 +0000 UTC m=+407.360701836" watchObservedRunningTime="2026-09-10 19:22:31.761871509 +0000 UTC m=+407.362778755"
Sep 10 19:22:32.090937 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.090897    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:32.197687 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.197643    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:32.377389 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.377314    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck29xb"]
Sep 10 19:22:32.387324 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.387297    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck29xb"
Sep 10 19:22:32.387818 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.387788    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:32.399134 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.398859    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-8f987\""
Sep 10 19:22:32.400416 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.400389    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck29xb"]
Sep 10 19:22:32.473023 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.472984    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/87991782-f257-48e3-8674-ee7097bb0a1f-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bck29xb\" (UID: \"87991782-f257-48e3-8674-ee7097bb0a1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck29xb"
Sep 10 19:22:32.473023 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.473027    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/87991782-f257-48e3-8674-ee7097bb0a1f-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bck29xb\" (UID: \"87991782-f257-48e3-8674-ee7097bb0a1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck29xb"
Sep 10 19:22:32.473232 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.473048    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/87991782-f257-48e3-8674-ee7097bb0a1f-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bck29xb\" (UID: \"87991782-f257-48e3-8674-ee7097bb0a1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck29xb"
Sep 10 19:22:32.473232 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.473118    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/87991782-f257-48e3-8674-ee7097bb0a1f-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bck29xb\" (UID: \"87991782-f257-48e3-8674-ee7097bb0a1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck29xb"
Sep 10 19:22:32.473232 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.473155    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/87991782-f257-48e3-8674-ee7097bb0a1f-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bck29xb\" (UID: \"87991782-f257-48e3-8674-ee7097bb0a1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck29xb"
Sep 10 19:22:32.473328 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.473237    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/87991782-f257-48e3-8674-ee7097bb0a1f-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bck29xb\" (UID: \"87991782-f257-48e3-8674-ee7097bb0a1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck29xb"
Sep 10 19:22:32.473328 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.473259    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cvr4\" (UniqueName: \"kubernetes.io/projected/87991782-f257-48e3-8674-ee7097bb0a1f-kube-api-access-2cvr4\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bck29xb\" (UID: \"87991782-f257-48e3-8674-ee7097bb0a1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck29xb"
Sep 10 19:22:32.473328 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.473281    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/87991782-f257-48e3-8674-ee7097bb0a1f-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bck29xb\" (UID: \"87991782-f257-48e3-8674-ee7097bb0a1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck29xb"
Sep 10 19:22:32.473328 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.473298    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/87991782-f257-48e3-8674-ee7097bb0a1f-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bck29xb\" (UID: \"87991782-f257-48e3-8674-ee7097bb0a1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck29xb"
Sep 10 19:22:32.574798 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.574760    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/87991782-f257-48e3-8674-ee7097bb0a1f-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bck29xb\" (UID: \"87991782-f257-48e3-8674-ee7097bb0a1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck29xb"
Sep 10 19:22:32.575012 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.574809    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cvr4\" (UniqueName: \"kubernetes.io/projected/87991782-f257-48e3-8674-ee7097bb0a1f-kube-api-access-2cvr4\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bck29xb\" (UID: \"87991782-f257-48e3-8674-ee7097bb0a1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck29xb"
Sep 10 19:22:32.575012 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.574849    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/87991782-f257-48e3-8674-ee7097bb0a1f-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bck29xb\" (UID: \"87991782-f257-48e3-8674-ee7097bb0a1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck29xb"
Sep 10 19:22:32.575012 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.574875    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/87991782-f257-48e3-8674-ee7097bb0a1f-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bck29xb\" (UID: \"87991782-f257-48e3-8674-ee7097bb0a1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck29xb"
Sep 10 19:22:32.575012 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.574921    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/87991782-f257-48e3-8674-ee7097bb0a1f-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bck29xb\" (UID: \"87991782-f257-48e3-8674-ee7097bb0a1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck29xb"
Sep 10 19:22:32.575012 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.574946    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/87991782-f257-48e3-8674-ee7097bb0a1f-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bck29xb\" (UID: \"87991782-f257-48e3-8674-ee7097bb0a1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck29xb"
Sep 10 19:22:32.575012 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.574970    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/87991782-f257-48e3-8674-ee7097bb0a1f-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bck29xb\" (UID: \"87991782-f257-48e3-8674-ee7097bb0a1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck29xb"
Sep 10 19:22:32.575012 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.575001    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/87991782-f257-48e3-8674-ee7097bb0a1f-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bck29xb\" (UID: \"87991782-f257-48e3-8674-ee7097bb0a1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck29xb"
Sep 10 19:22:32.575388 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.575042    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/87991782-f257-48e3-8674-ee7097bb0a1f-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bck29xb\" (UID: \"87991782-f257-48e3-8674-ee7097bb0a1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck29xb"
Sep 10 19:22:32.575388 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.575287    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/87991782-f257-48e3-8674-ee7097bb0a1f-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bck29xb\" (UID: \"87991782-f257-48e3-8674-ee7097bb0a1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck29xb"
Sep 10 19:22:32.575500 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.575444    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/87991782-f257-48e3-8674-ee7097bb0a1f-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bck29xb\" (UID: \"87991782-f257-48e3-8674-ee7097bb0a1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck29xb"
Sep 10 19:22:32.575554 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.575519    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/87991782-f257-48e3-8674-ee7097bb0a1f-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bck29xb\" (UID: \"87991782-f257-48e3-8674-ee7097bb0a1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck29xb"
Sep 10 19:22:32.575679 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.575654    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/87991782-f257-48e3-8674-ee7097bb0a1f-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bck29xb\" (UID: \"87991782-f257-48e3-8674-ee7097bb0a1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck29xb"
Sep 10 19:22:32.576042 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.576006    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/87991782-f257-48e3-8674-ee7097bb0a1f-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bck29xb\" (UID: \"87991782-f257-48e3-8674-ee7097bb0a1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck29xb"
Sep 10 19:22:32.577592 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.577573    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/87991782-f257-48e3-8674-ee7097bb0a1f-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bck29xb\" (UID: \"87991782-f257-48e3-8674-ee7097bb0a1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck29xb"
Sep 10 19:22:32.577988 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.577959    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/87991782-f257-48e3-8674-ee7097bb0a1f-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bck29xb\" (UID: \"87991782-f257-48e3-8674-ee7097bb0a1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck29xb"
Sep 10 19:22:32.595044 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.591467    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j"]
Sep 10 19:22:32.595044 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.594513    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cvr4\" (UniqueName: \"kubernetes.io/projected/87991782-f257-48e3-8674-ee7097bb0a1f-kube-api-access-2cvr4\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bck29xb\" (UID: \"87991782-f257-48e3-8674-ee7097bb0a1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck29xb"
Sep 10 19:22:32.596496 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.596470    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/87991782-f257-48e3-8674-ee7097bb0a1f-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bck29xb\" (UID: \"87991782-f257-48e3-8674-ee7097bb0a1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck29xb"
Sep 10 19:22:32.601331 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.601306    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j"
Sep 10 19:22:32.614748 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.614721    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j"]
Sep 10 19:22:32.634119 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.634026    2564 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-ngx68 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 19:22:32.634119 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.634084    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ngx68" podUID="fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 19:22:32.675903 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.675846    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b34759e7-f5de-46c2-9f9f-9d3b3dd3200f-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j\" (UID: \"b34759e7-f5de-46c2-9f9f-9d3b3dd3200f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j"
Sep 10 19:22:32.676082 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.675925    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b34759e7-f5de-46c2-9f9f-9d3b3dd3200f-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j\" (UID: \"b34759e7-f5de-46c2-9f9f-9d3b3dd3200f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j"
Sep 10 19:22:32.676082 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.675942    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b34759e7-f5de-46c2-9f9f-9d3b3dd3200f-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j\" (UID: \"b34759e7-f5de-46c2-9f9f-9d3b3dd3200f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j"
Sep 10 19:22:32.676082 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.676020    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b34759e7-f5de-46c2-9f9f-9d3b3dd3200f-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j\" (UID: \"b34759e7-f5de-46c2-9f9f-9d3b3dd3200f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j"
Sep 10 19:22:32.676262 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.676117    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b34759e7-f5de-46c2-9f9f-9d3b3dd3200f-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j\" (UID: \"b34759e7-f5de-46c2-9f9f-9d3b3dd3200f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j"
Sep 10 19:22:32.676262 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.676153    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b34759e7-f5de-46c2-9f9f-9d3b3dd3200f-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j\" (UID: \"b34759e7-f5de-46c2-9f9f-9d3b3dd3200f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j"
Sep 10 19:22:32.676262 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.676255    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b34759e7-f5de-46c2-9f9f-9d3b3dd3200f-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j\" (UID: \"b34759e7-f5de-46c2-9f9f-9d3b3dd3200f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j"
Sep 10 19:22:32.676419 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.676299    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b34759e7-f5de-46c2-9f9f-9d3b3dd3200f-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j\" (UID: \"b34759e7-f5de-46c2-9f9f-9d3b3dd3200f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j"
Sep 10 19:22:32.676419 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.676384    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sl5qj\" (UniqueName: \"kubernetes.io/projected/b34759e7-f5de-46c2-9f9f-9d3b3dd3200f-kube-api-access-sl5qj\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j\" (UID: \"b34759e7-f5de-46c2-9f9f-9d3b3dd3200f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j"
Sep 10 19:22:32.698610 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.698568    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck29xb"
Sep 10 19:22:32.715593 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.715555    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:32.777553 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.777219    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b34759e7-f5de-46c2-9f9f-9d3b3dd3200f-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j\" (UID: \"b34759e7-f5de-46c2-9f9f-9d3b3dd3200f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j"
Sep 10 19:22:32.777553 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.777311    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sl5qj\" (UniqueName: \"kubernetes.io/projected/b34759e7-f5de-46c2-9f9f-9d3b3dd3200f-kube-api-access-sl5qj\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j\" (UID: \"b34759e7-f5de-46c2-9f9f-9d3b3dd3200f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j"
Sep 10 19:22:32.777553 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.777362    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b34759e7-f5de-46c2-9f9f-9d3b3dd3200f-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j\" (UID: \"b34759e7-f5de-46c2-9f9f-9d3b3dd3200f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j"
Sep 10 19:22:32.777553 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.777410    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b34759e7-f5de-46c2-9f9f-9d3b3dd3200f-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j\" (UID: \"b34759e7-f5de-46c2-9f9f-9d3b3dd3200f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j"
Sep 10 19:22:32.777553 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.777439    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b34759e7-f5de-46c2-9f9f-9d3b3dd3200f-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j\" (UID: \"b34759e7-f5de-46c2-9f9f-9d3b3dd3200f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j"
Sep 10 19:22:32.777553 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.777474    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b34759e7-f5de-46c2-9f9f-9d3b3dd3200f-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j\" (UID: \"b34759e7-f5de-46c2-9f9f-9d3b3dd3200f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j"
Sep 10 19:22:32.777553 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.777511    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b34759e7-f5de-46c2-9f9f-9d3b3dd3200f-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j\" (UID: \"b34759e7-f5de-46c2-9f9f-9d3b3dd3200f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j"
Sep 10 19:22:32.777553 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.777546    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b34759e7-f5de-46c2-9f9f-9d3b3dd3200f-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j\" (UID: \"b34759e7-f5de-46c2-9f9f-9d3b3dd3200f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j"
Sep 10 19:22:32.778067 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.777599    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b34759e7-f5de-46c2-9f9f-9d3b3dd3200f-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j\" (UID: \"b34759e7-f5de-46c2-9f9f-9d3b3dd3200f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j"
Sep 10 19:22:32.778067 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.777809    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b34759e7-f5de-46c2-9f9f-9d3b3dd3200f-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j\" (UID: \"b34759e7-f5de-46c2-9f9f-9d3b3dd3200f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j"
Sep 10 19:22:32.778372 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.778186    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b34759e7-f5de-46c2-9f9f-9d3b3dd3200f-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j\" (UID: \"b34759e7-f5de-46c2-9f9f-9d3b3dd3200f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j"
Sep 10 19:22:32.778504 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.778379    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b34759e7-f5de-46c2-9f9f-9d3b3dd3200f-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j\" (UID: \"b34759e7-f5de-46c2-9f9f-9d3b3dd3200f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j"
Sep 10 19:22:32.778736 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.778693    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b34759e7-f5de-46c2-9f9f-9d3b3dd3200f-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j\" (UID: \"b34759e7-f5de-46c2-9f9f-9d3b3dd3200f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j"
Sep 10 19:22:32.778736 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.778700    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b34759e7-f5de-46c2-9f9f-9d3b3dd3200f-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j\" (UID: \"b34759e7-f5de-46c2-9f9f-9d3b3dd3200f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j"
Sep 10 19:22:32.781055 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.781022    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b34759e7-f5de-46c2-9f9f-9d3b3dd3200f-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j\" (UID: \"b34759e7-f5de-46c2-9f9f-9d3b3dd3200f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j"
Sep 10 19:22:32.781544 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.781497    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b34759e7-f5de-46c2-9f9f-9d3b3dd3200f-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j\" (UID: \"b34759e7-f5de-46c2-9f9f-9d3b3dd3200f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j"
Sep 10 19:22:32.788879 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.788833    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b34759e7-f5de-46c2-9f9f-9d3b3dd3200f-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j\" (UID: \"b34759e7-f5de-46c2-9f9f-9d3b3dd3200f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j"
Sep 10 19:22:32.791435 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.791391    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sl5qj\" (UniqueName: \"kubernetes.io/projected/b34759e7-f5de-46c2-9f9f-9d3b3dd3200f-kube-api-access-sl5qj\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j\" (UID: \"b34759e7-f5de-46c2-9f9f-9d3b3dd3200f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j"
Sep 10 19:22:32.815771 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.815737    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:32.917954 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:32.917869    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j"
Sep 10 19:22:33.002391 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:22:33.002349    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87991782_f257_48e3_8674_ee7097bb0a1f.slice/crio-c297b0211d8bae2399fbe9988877cd920f2a6b87dff3efe3bc12e2e1d084d25f WatchSource:0}: Error finding container c297b0211d8bae2399fbe9988877cd920f2a6b87dff3efe3bc12e2e1d084d25f: Status 404 returned error can't find the container with id c297b0211d8bae2399fbe9988877cd920f2a6b87dff3efe3bc12e2e1d084d25f
Sep 10 19:22:33.004750 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:33.004725    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck29xb"]
Sep 10 19:22:33.196401 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:33.196261    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j"]
Sep 10 19:22:33.198979 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:22:33.198955    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb34759e7_f5de_46c2_9f9f_9d3b3dd3200f.slice/crio-d44a35e517faf83af1f14f52c0d496af352dd85ae6169bc6a4a187e8fb028992 WatchSource:0}: Error finding container d44a35e517faf83af1f14f52c0d496af352dd85ae6169bc6a4a187e8fb028992: Status 404 returned error can't find the container with id d44a35e517faf83af1f14f52c0d496af352dd85ae6169bc6a4a187e8fb028992
Sep 10 19:22:33.637352 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:33.637314    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck29xb" event={"ID":"87991782-f257-48e3-8674-ee7097bb0a1f","Type":"ContainerStarted","Data":"c297b0211d8bae2399fbe9988877cd920f2a6b87dff3efe3bc12e2e1d084d25f"}
Sep 10 19:22:33.638548 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:33.638526    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j" event={"ID":"b34759e7-f5de-46c2-9f9f-9d3b3dd3200f","Type":"ContainerStarted","Data":"d44a35e517faf83af1f14f52c0d496af352dd85ae6169bc6a4a187e8fb028992"}
Sep 10 19:22:34.426259 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:34.426199    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:34.533365 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:34.533335    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-tv6rf"]
Sep 10 19:22:34.535245 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:34.535132    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:34.537101 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:34.537078    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-tv6rf"
Sep 10 19:22:34.544791 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:34.544761    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 10 19:22:34.545170 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:34.544960    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-rvfs8\""
Sep 10 19:22:34.563090 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:34.563048    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-k55hx"]
Sep 10 19:22:34.566692 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:34.566666    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-tv6rf"]
Sep 10 19:22:34.566797 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:34.566788    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-k55hx"
Sep 10 19:22:34.577902 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:34.577872    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 10 19:22:34.578043 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:34.577977    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-7sd4s\""
Sep 10 19:22:34.598038 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:34.598003    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gpdm\" (UniqueName: \"kubernetes.io/projected/b046a9c4-10fd-4387-b304-a57f379d8314-kube-api-access-8gpdm\") pod \"kserve-controller-manager-d6f4b65b8-tv6rf\" (UID: \"b046a9c4-10fd-4387-b304-a57f379d8314\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-tv6rf"
Sep 10 19:22:34.598354 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:34.598059    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b046a9c4-10fd-4387-b304-a57f379d8314-cert\") pod \"kserve-controller-manager-d6f4b65b8-tv6rf\" (UID: \"b046a9c4-10fd-4387-b304-a57f379d8314\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-tv6rf"
Sep 10 19:22:34.621599 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:34.621517    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-k55hx"]
Sep 10 19:22:34.624787 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:34.624670    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:34.698758 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:34.698667    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gpdm\" (UniqueName: \"kubernetes.io/projected/b046a9c4-10fd-4387-b304-a57f379d8314-kube-api-access-8gpdm\") pod \"kserve-controller-manager-d6f4b65b8-tv6rf\" (UID: \"b046a9c4-10fd-4387-b304-a57f379d8314\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-tv6rf"
Sep 10 19:22:34.698758 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:34.698723    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b046a9c4-10fd-4387-b304-a57f379d8314-cert\") pod \"kserve-controller-manager-d6f4b65b8-tv6rf\" (UID: \"b046a9c4-10fd-4387-b304-a57f379d8314\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-tv6rf"
Sep 10 19:22:34.699257 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:34.698815    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpc6w\" (UniqueName: \"kubernetes.io/projected/437ff782-378c-4a6e-a7ce-3ec34f48a472-kube-api-access-hpc6w\") pod \"llmisvc-controller-manager-777c6bd7d5-k55hx\" (UID: \"437ff782-378c-4a6e-a7ce-3ec34f48a472\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-k55hx"
Sep 10 19:22:34.699257 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:34.698848    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/437ff782-378c-4a6e-a7ce-3ec34f48a472-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-k55hx\" (UID: \"437ff782-378c-4a6e-a7ce-3ec34f48a472\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-k55hx"
Sep 10 19:22:34.699257 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:22:34.698991    2564 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Sep 10 19:22:34.699257 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:22:34.699080    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b046a9c4-10fd-4387-b304-a57f379d8314-cert podName:b046a9c4-10fd-4387-b304-a57f379d8314 nodeName:}" failed. No retries permitted until 2026-09-10 19:22:35.199029108 +0000 UTC m=+410.799936343 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/b046a9c4-10fd-4387-b304-a57f379d8314-cert") pod "kserve-controller-manager-d6f4b65b8-tv6rf" (UID: "b046a9c4-10fd-4387-b304-a57f379d8314") : secret "kserve-webhook-server-cert" not found
Sep 10 19:22:34.712388 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:34.712350    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gpdm\" (UniqueName: \"kubernetes.io/projected/b046a9c4-10fd-4387-b304-a57f379d8314-kube-api-access-8gpdm\") pod \"kserve-controller-manager-d6f4b65b8-tv6rf\" (UID: \"b046a9c4-10fd-4387-b304-a57f379d8314\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-tv6rf"
Sep 10 19:22:34.799872 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:34.799829    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpc6w\" (UniqueName: \"kubernetes.io/projected/437ff782-378c-4a6e-a7ce-3ec34f48a472-kube-api-access-hpc6w\") pod \"llmisvc-controller-manager-777c6bd7d5-k55hx\" (UID: \"437ff782-378c-4a6e-a7ce-3ec34f48a472\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-k55hx"
Sep 10 19:22:34.800072 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:34.799885    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/437ff782-378c-4a6e-a7ce-3ec34f48a472-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-k55hx\" (UID: \"437ff782-378c-4a6e-a7ce-3ec34f48a472\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-k55hx"
Sep 10 19:22:34.800072 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:22:34.800065    2564 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Sep 10 19:22:34.800190 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:22:34.800133    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/437ff782-378c-4a6e-a7ce-3ec34f48a472-cert podName:437ff782-378c-4a6e-a7ce-3ec34f48a472 nodeName:}" failed. No retries permitted until 2026-09-10 19:22:35.300112138 +0000 UTC m=+410.901019358 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/437ff782-378c-4a6e-a7ce-3ec34f48a472-cert") pod "llmisvc-controller-manager-777c6bd7d5-k55hx" (UID: "437ff782-378c-4a6e-a7ce-3ec34f48a472") : secret "llmisvc-webhook-server-cert" not found
Sep 10 19:22:34.817597 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:34.817561    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpc6w\" (UniqueName: \"kubernetes.io/projected/437ff782-378c-4a6e-a7ce-3ec34f48a472-kube-api-access-hpc6w\") pod \"llmisvc-controller-manager-777c6bd7d5-k55hx\" (UID: \"437ff782-378c-4a6e-a7ce-3ec34f48a472\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-k55hx"
Sep 10 19:22:34.848454 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:34.848392    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:34.932943 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:34.932883    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:35.207588 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:35.206355    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b046a9c4-10fd-4387-b304-a57f379d8314-cert\") pod \"kserve-controller-manager-d6f4b65b8-tv6rf\" (UID: \"b046a9c4-10fd-4387-b304-a57f379d8314\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-tv6rf"
Sep 10 19:22:35.210486 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:35.210418    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b046a9c4-10fd-4387-b304-a57f379d8314-cert\") pod \"kserve-controller-manager-d6f4b65b8-tv6rf\" (UID: \"b046a9c4-10fd-4387-b304-a57f379d8314\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-tv6rf"
Sep 10 19:22:35.309374 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:35.309325    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/437ff782-378c-4a6e-a7ce-3ec34f48a472-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-k55hx\" (UID: \"437ff782-378c-4a6e-a7ce-3ec34f48a472\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-k55hx"
Sep 10 19:22:35.312270 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:35.312223    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/437ff782-378c-4a6e-a7ce-3ec34f48a472-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-k55hx\" (UID: \"437ff782-378c-4a6e-a7ce-3ec34f48a472\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-k55hx"
Sep 10 19:22:35.453058 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:35.453017    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-tv6rf"
Sep 10 19:22:35.480289 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:35.480211    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-k55hx"
Sep 10 19:22:35.633259 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:35.633229    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ngx68"
Sep 10 19:22:35.792578 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:35.792538    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:35.891202 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:35.891169    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:35.947748 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:35.947713    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:36.277542 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:36.277497    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:22:36.277676 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:36.277596    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:22:36.277676 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:36.277660    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:22:36.285325 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:36.285290    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:22:36.285397 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:36.285379    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:22:36.285440 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:36.285428    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:22:36.340242 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:22:36.340148    2564 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb34759e7_f5de_46c2_9f9f_9d3b3dd3200f.slice/crio-fa328d691f130e31a7320a8d5b1b3309f4f73e3c95c5d00f8a03eab7400042e8.scope\": RecentStats: unable to find data in memory cache]"
Sep 10 19:22:36.458532 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:36.457754    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ngx68"]
Sep 10 19:22:36.458532 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:36.457978    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ngx68" podUID="fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de" containerName="discovery" containerID="cri-o://c769e6e2136fd922a667ed8ad7379bac0a2c315ca9a691367bf3be049f48f9a5" gracePeriod=30
Sep 10 19:22:36.490187 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:36.490160    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-k55hx"]
Sep 10 19:22:36.494031 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:22:36.493994    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod437ff782_378c_4a6e_a7ce_3ec34f48a472.slice/crio-3b74a7bd347f7aa241e547a94448dc194dc34fecda970e3a6bf4bf223069a3c4 WatchSource:0}: Error finding container 3b74a7bd347f7aa241e547a94448dc194dc34fecda970e3a6bf4bf223069a3c4: Status 404 returned error can't find the container with id 3b74a7bd347f7aa241e547a94448dc194dc34fecda970e3a6bf4bf223069a3c4
Sep 10 19:22:36.504488 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:36.504398    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-tv6rf"]
Sep 10 19:22:36.529572 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:22:36.529538    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb046a9c4_10fd_4387_b304_a57f379d8314.slice/crio-a8daf02d937e17cf54778952abeaa61aa30d7fd83960611b02cbb07d3e2162d5 WatchSource:0}: Error finding container a8daf02d937e17cf54778952abeaa61aa30d7fd83960611b02cbb07d3e2162d5: Status 404 returned error can't find the container with id a8daf02d937e17cf54778952abeaa61aa30d7fd83960611b02cbb07d3e2162d5
Sep 10 19:22:36.654810 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:36.654774    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-k55hx" event={"ID":"437ff782-378c-4a6e-a7ce-3ec34f48a472","Type":"ContainerStarted","Data":"3b74a7bd347f7aa241e547a94448dc194dc34fecda970e3a6bf4bf223069a3c4"}
Sep 10 19:22:36.656643 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:36.656589    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck29xb" event={"ID":"87991782-f257-48e3-8674-ee7097bb0a1f","Type":"ContainerStarted","Data":"8f7f96a9abd4e5b732b315e4cfa26d9f9897a13569ab066a98a21d8bb3051c03"}
Sep 10 19:22:36.658417 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:36.658393    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j" event={"ID":"b34759e7-f5de-46c2-9f9f-9d3b3dd3200f","Type":"ContainerStarted","Data":"fa328d691f130e31a7320a8d5b1b3309f4f73e3c95c5d00f8a03eab7400042e8"}
Sep 10 19:22:36.659828 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:36.659792    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-tv6rf" event={"ID":"b046a9c4-10fd-4387-b304-a57f379d8314","Type":"ContainerStarted","Data":"a8daf02d937e17cf54778952abeaa61aa30d7fd83960611b02cbb07d3e2162d5"}
Sep 10 19:22:36.661463 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:36.661440    2564 generic.go:356] "Generic (PLEG): container finished" podID="fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de" containerID="c769e6e2136fd922a667ed8ad7379bac0a2c315ca9a691367bf3be049f48f9a5" exitCode=0
Sep 10 19:22:36.661552 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:36.661488    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ngx68" event={"ID":"fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de","Type":"ContainerDied","Data":"c769e6e2136fd922a667ed8ad7379bac0a2c315ca9a691367bf3be049f48f9a5"}
Sep 10 19:22:36.684456 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:36.684377    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck29xb" podStartSLOduration=1.404095709 podStartE2EDuration="4.684360331s" totalImagesPullingTime="3.280264622s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:22:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:22:36.680546342 +0000 UTC m=+412.281453582" watchObservedRunningTime="2026-09-10 19:22:36.684360331 +0000 UTC m=+412.285267570"
Sep 10 19:22:36.699593 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:36.699561    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck29xb"
Sep 10 19:22:36.699807 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:36.699789    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ngx68"
Sep 10 19:22:36.708925 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:36.708683    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j" podStartSLOduration=1.632575061 podStartE2EDuration="4.708664271s" totalImagesPullingTime="3.07608921s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:22:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:22:36.707983389 +0000 UTC m=+412.308890626" watchObservedRunningTime="2026-09-10 19:22:36.708664271 +0000 UTC m=+412.309571511"
Sep 10 19:22:36.824009 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:36.823972    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de-istio-kubeconfig\") pod \"fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de\" (UID: \"fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de\") "
Sep 10 19:22:36.824009 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:36.824016    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de-cacerts\") pod \"fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de\" (UID: \"fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de\") "
Sep 10 19:22:36.824485 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:36.824081    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de-local-certs\") pod \"fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de\" (UID: \"fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de\") "
Sep 10 19:22:36.824485 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:36.824116    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de-istio-csr-dns-cert\") pod \"fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de\" (UID: \"fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de\") "
Sep 10 19:22:36.824485 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:36.824146    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de-istio-csr-ca-configmap\") pod \"fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de\" (UID: \"fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de\") "
Sep 10 19:22:36.824485 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:36.824185    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hh4cs\" (UniqueName: \"kubernetes.io/projected/fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de-kube-api-access-hh4cs\") pod \"fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de\" (UID: \"fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de\") "
Sep 10 19:22:36.824485 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:36.824205    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de-istio-token\") pod \"fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de\" (UID: \"fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de\") "
Sep 10 19:22:36.824789 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:36.824562    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de" (UID: "fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:22:36.826603 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:36.826568    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de-istio-token" (OuterVolumeSpecName: "istio-token") pod "fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de" (UID: "fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:22:36.826747 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:36.826637    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de-kube-api-access-hh4cs" (OuterVolumeSpecName: "kube-api-access-hh4cs") pod "fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de" (UID: "fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de"). InnerVolumeSpecName "kube-api-access-hh4cs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:22:36.826747 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:36.826723    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de-cacerts" (OuterVolumeSpecName: "cacerts") pod "fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de" (UID: "fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:22:36.826852 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:36.826743    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de" (UID: "fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:22:36.826852 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:36.826820    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de" (UID: "fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:22:36.826852 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:36.826845    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de-local-certs" (OuterVolumeSpecName: "local-certs") pod "fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de" (UID: "fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:22:36.918961 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:36.918921    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j"
Sep 10 19:22:36.925398 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:36.925369    2564 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de-istio-csr-dns-cert\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:22:36.925398 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:36.925397    2564 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de-istio-csr-ca-configmap\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:22:36.925491 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:36.925408    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hh4cs\" (UniqueName: \"kubernetes.io/projected/fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de-kube-api-access-hh4cs\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:22:36.925491 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:36.925418    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de-istio-token\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:22:36.925491 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:36.925426    2564 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de-istio-kubeconfig\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:22:36.925491 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:36.925434    2564 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de-cacerts\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:22:36.925491 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:36.925442    2564 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de-local-certs\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:22:37.670543 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:37.670513    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ngx68"
Sep 10 19:22:37.670823 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:37.670539    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ngx68" event={"ID":"fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de","Type":"ContainerDied","Data":"4a8f634a9027869166d1859394c12ad488ebb39889b251af2e5b8dd144909659"}
Sep 10 19:22:37.671416 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:37.671211    2564 scope.go:117] "RemoveContainer" containerID="c769e6e2136fd922a667ed8ad7379bac0a2c315ca9a691367bf3be049f48f9a5"
Sep 10 19:22:37.700696 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:37.700659    2564 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-6745bb66bck29xb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" start-of-body=
Sep 10 19:22:37.700864 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:37.700723    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck29xb" podUID="87991782-f257-48e3-8674-ee7097bb0a1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused"
Sep 10 19:22:37.712366 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:37.712331    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ngx68"]
Sep 10 19:22:37.716984 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:37.716957    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ngx68"]
Sep 10 19:22:37.920505 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:37.920470    2564 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" start-of-body=
Sep 10 19:22:37.920988 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:37.920536    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j" podUID="b34759e7-f5de-46c2-9f9f-9d3b3dd3200f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused"
Sep 10 19:22:38.699524 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:38.699486    2564 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-6745bb66bck29xb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" start-of-body=
Sep 10 19:22:38.699738 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:38.699547    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck29xb" podUID="87991782-f257-48e3-8674-ee7097bb0a1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused"
Sep 10 19:22:38.922752 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:38.922715    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j"
Sep 10 19:22:38.949477 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:38.949439    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de" path="/var/lib/kubelet/pods/fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de/volumes"
Sep 10 19:22:39.681067 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:39.681038    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j"
Sep 10 19:22:39.681960 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:39.681939    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j"
Sep 10 19:22:39.698931 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:39.698902    2564 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-6745bb66bck29xb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" start-of-body=
Sep 10 19:22:39.699045 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:39.698955    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck29xb" podUID="87991782-f257-48e3-8674-ee7097bb0a1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused"
Sep 10 19:22:39.753078 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:39.753045    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck29xb"]
Sep 10 19:22:39.753266 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:39.753240    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck29xb" podUID="87991782-f257-48e3-8674-ee7097bb0a1f" containerName="istio-proxy" containerID="cri-o://8f7f96a9abd4e5b732b315e4cfa26d9f9897a13569ab066a98a21d8bb3051c03" gracePeriod=30
Sep 10 19:22:40.687014 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:40.686975    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-tv6rf" event={"ID":"b046a9c4-10fd-4387-b304-a57f379d8314","Type":"ContainerStarted","Data":"528e3c9b28031213baabfe7a12b2dc2a4ee6a3db252352ab36848f0688a7aecf"}
Sep 10 19:22:40.687477 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:40.687051    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-tv6rf"
Sep 10 19:22:40.688440 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:40.688419    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-k55hx" event={"ID":"437ff782-378c-4a6e-a7ce-3ec34f48a472","Type":"ContainerStarted","Data":"c46ead2218611aaff91632c341b2794107953a5fe687233a0815ebeccf40e901"}
Sep 10 19:22:40.688561 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:40.688458    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-k55hx"
Sep 10 19:22:40.716583 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:40.716526    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-tv6rf" podStartSLOduration=3.552785886 podStartE2EDuration="6.71650721s" totalImagesPullingTime="3.163721324s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:22:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:22:40.716125017 +0000 UTC m=+416.317032256" watchObservedRunningTime="2026-09-10 19:22:40.71650721 +0000 UTC m=+416.317414449"
Sep 10 19:22:40.744069 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:40.744019    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-k55hx" podStartSLOduration=3.5758974610000003 podStartE2EDuration="6.744003169s" totalImagesPullingTime="3.168105708s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:22:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:22:40.741351516 +0000 UTC m=+416.342258756" watchObservedRunningTime="2026-09-10 19:22:40.744003169 +0000 UTC m=+416.344910410"
Sep 10 19:22:42.634552 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:42.634521    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-48fv6"
Sep 10 19:22:42.637841 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:42.637818    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-96fp6"
Sep 10 19:22:45.008075 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:45.008049    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck29xb"
Sep 10 19:22:45.103836 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:45.103732    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/87991782-f257-48e3-8674-ee7097bb0a1f-istio-podinfo\") pod \"87991782-f257-48e3-8674-ee7097bb0a1f\" (UID: \"87991782-f257-48e3-8674-ee7097bb0a1f\") "
Sep 10 19:22:45.103836 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:45.103791    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/87991782-f257-48e3-8674-ee7097bb0a1f-istio-data\") pod \"87991782-f257-48e3-8674-ee7097bb0a1f\" (UID: \"87991782-f257-48e3-8674-ee7097bb0a1f\") "
Sep 10 19:22:45.103836 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:45.103813    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/87991782-f257-48e3-8674-ee7097bb0a1f-workload-socket\") pod \"87991782-f257-48e3-8674-ee7097bb0a1f\" (UID: \"87991782-f257-48e3-8674-ee7097bb0a1f\") "
Sep 10 19:22:45.104116 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:45.103845    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/87991782-f257-48e3-8674-ee7097bb0a1f-istiod-ca-cert\") pod \"87991782-f257-48e3-8674-ee7097bb0a1f\" (UID: \"87991782-f257-48e3-8674-ee7097bb0a1f\") "
Sep 10 19:22:45.104116 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:45.104013    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/87991782-f257-48e3-8674-ee7097bb0a1f-workload-certs\") pod \"87991782-f257-48e3-8674-ee7097bb0a1f\" (UID: \"87991782-f257-48e3-8674-ee7097bb0a1f\") "
Sep 10 19:22:45.104116 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:45.104062    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/87991782-f257-48e3-8674-ee7097bb0a1f-credential-socket\") pod \"87991782-f257-48e3-8674-ee7097bb0a1f\" (UID: \"87991782-f257-48e3-8674-ee7097bb0a1f\") "
Sep 10 19:22:45.104270 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:45.104138    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87991782-f257-48e3-8674-ee7097bb0a1f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "87991782-f257-48e3-8674-ee7097bb0a1f" (UID: "87991782-f257-48e3-8674-ee7097bb0a1f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:22:45.104270 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:45.104151    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87991782-f257-48e3-8674-ee7097bb0a1f-istio-data" (OuterVolumeSpecName: "istio-data") pod "87991782-f257-48e3-8674-ee7097bb0a1f" (UID: "87991782-f257-48e3-8674-ee7097bb0a1f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:22:45.104270 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:45.104163    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2cvr4\" (UniqueName: \"kubernetes.io/projected/87991782-f257-48e3-8674-ee7097bb0a1f-kube-api-access-2cvr4\") pod \"87991782-f257-48e3-8674-ee7097bb0a1f\" (UID: \"87991782-f257-48e3-8674-ee7097bb0a1f\") "
Sep 10 19:22:45.104270 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:45.104213    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/87991782-f257-48e3-8674-ee7097bb0a1f-istio-token\") pod \"87991782-f257-48e3-8674-ee7097bb0a1f\" (UID: \"87991782-f257-48e3-8674-ee7097bb0a1f\") "
Sep 10 19:22:45.104270 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:45.104237    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87991782-f257-48e3-8674-ee7097bb0a1f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "87991782-f257-48e3-8674-ee7097bb0a1f" (UID: "87991782-f257-48e3-8674-ee7097bb0a1f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:22:45.104270 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:45.104244    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/87991782-f257-48e3-8674-ee7097bb0a1f-istio-envoy\") pod \"87991782-f257-48e3-8674-ee7097bb0a1f\" (UID: \"87991782-f257-48e3-8674-ee7097bb0a1f\") "
Sep 10 19:22:45.104270 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:45.104254    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87991782-f257-48e3-8674-ee7097bb0a1f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "87991782-f257-48e3-8674-ee7097bb0a1f" (UID: "87991782-f257-48e3-8674-ee7097bb0a1f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:22:45.104614 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:45.104344    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87991782-f257-48e3-8674-ee7097bb0a1f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "87991782-f257-48e3-8674-ee7097bb0a1f" (UID: "87991782-f257-48e3-8674-ee7097bb0a1f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:22:45.104614 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:45.104560    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/87991782-f257-48e3-8674-ee7097bb0a1f-istio-data\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:22:45.104614 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:45.104580    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/87991782-f257-48e3-8674-ee7097bb0a1f-workload-socket\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:22:45.104614 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:45.104596    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/87991782-f257-48e3-8674-ee7097bb0a1f-istiod-ca-cert\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:22:45.104614 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:45.104610    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/87991782-f257-48e3-8674-ee7097bb0a1f-workload-certs\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:22:45.104898 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:45.104648    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/87991782-f257-48e3-8674-ee7097bb0a1f-credential-socket\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:22:45.106269 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:45.106226    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/87991782-f257-48e3-8674-ee7097bb0a1f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "87991782-f257-48e3-8674-ee7097bb0a1f" (UID: "87991782-f257-48e3-8674-ee7097bb0a1f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 19:22:45.106269 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:45.106227    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87991782-f257-48e3-8674-ee7097bb0a1f-kube-api-access-2cvr4" (OuterVolumeSpecName: "kube-api-access-2cvr4") pod "87991782-f257-48e3-8674-ee7097bb0a1f" (UID: "87991782-f257-48e3-8674-ee7097bb0a1f"). InnerVolumeSpecName "kube-api-access-2cvr4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:22:45.106424 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:45.106315    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87991782-f257-48e3-8674-ee7097bb0a1f-istio-token" (OuterVolumeSpecName: "istio-token") pod "87991782-f257-48e3-8674-ee7097bb0a1f" (UID: "87991782-f257-48e3-8674-ee7097bb0a1f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:22:45.106424 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:45.106335    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87991782-f257-48e3-8674-ee7097bb0a1f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "87991782-f257-48e3-8674-ee7097bb0a1f" (UID: "87991782-f257-48e3-8674-ee7097bb0a1f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:22:45.205915 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:45.205874    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/87991782-f257-48e3-8674-ee7097bb0a1f-istio-token\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:22:45.205915 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:45.205906    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/87991782-f257-48e3-8674-ee7097bb0a1f-istio-envoy\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:22:45.205915 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:45.205915    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/87991782-f257-48e3-8674-ee7097bb0a1f-istio-podinfo\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:22:45.205915 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:45.205923    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2cvr4\" (UniqueName: \"kubernetes.io/projected/87991782-f257-48e3-8674-ee7097bb0a1f-kube-api-access-2cvr4\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:22:45.711049 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:45.711016    2564 generic.go:356] "Generic (PLEG): container finished" podID="87991782-f257-48e3-8674-ee7097bb0a1f" containerID="8f7f96a9abd4e5b732b315e4cfa26d9f9897a13569ab066a98a21d8bb3051c03" exitCode=0
Sep 10 19:22:45.711225 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:45.711059    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck29xb"
Sep 10 19:22:45.711225 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:45.711080    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck29xb" event={"ID":"87991782-f257-48e3-8674-ee7097bb0a1f","Type":"ContainerDied","Data":"8f7f96a9abd4e5b732b315e4cfa26d9f9897a13569ab066a98a21d8bb3051c03"}
Sep 10 19:22:45.711225 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:45.711120    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck29xb" event={"ID":"87991782-f257-48e3-8674-ee7097bb0a1f","Type":"ContainerDied","Data":"c297b0211d8bae2399fbe9988877cd920f2a6b87dff3efe3bc12e2e1d084d25f"}
Sep 10 19:22:45.711225 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:45.711142    2564 scope.go:117] "RemoveContainer" containerID="8f7f96a9abd4e5b732b315e4cfa26d9f9897a13569ab066a98a21d8bb3051c03"
Sep 10 19:22:45.720447 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:45.720431    2564 scope.go:117] "RemoveContainer" containerID="8f7f96a9abd4e5b732b315e4cfa26d9f9897a13569ab066a98a21d8bb3051c03"
Sep 10 19:22:45.720730 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:22:45.720711    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f7f96a9abd4e5b732b315e4cfa26d9f9897a13569ab066a98a21d8bb3051c03\": container with ID starting with 8f7f96a9abd4e5b732b315e4cfa26d9f9897a13569ab066a98a21d8bb3051c03 not found: ID does not exist" containerID="8f7f96a9abd4e5b732b315e4cfa26d9f9897a13569ab066a98a21d8bb3051c03"
Sep 10 19:22:45.720783 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:45.720737    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f7f96a9abd4e5b732b315e4cfa26d9f9897a13569ab066a98a21d8bb3051c03"} err="failed to get container status \"8f7f96a9abd4e5b732b315e4cfa26d9f9897a13569ab066a98a21d8bb3051c03\": rpc error: code = NotFound desc = could not find container \"8f7f96a9abd4e5b732b315e4cfa26d9f9897a13569ab066a98a21d8bb3051c03\": container with ID starting with 8f7f96a9abd4e5b732b315e4cfa26d9f9897a13569ab066a98a21d8bb3051c03 not found: ID does not exist"
Sep 10 19:22:45.736460 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:45.736414    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck29xb"]
Sep 10 19:22:45.738041 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:45.738018    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bck29xb"]
Sep 10 19:22:46.948715 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:22:46.948678    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87991782-f257-48e3-8674-ee7097bb0a1f" path="/var/lib/kubelet/pods/87991782-f257-48e3-8674-ee7097bb0a1f/volumes"
Sep 10 19:23:11.694940 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:11.694863    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-k55hx"
Sep 10 19:23:11.697708 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:11.697686    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-tv6rf"
Sep 10 19:23:12.799072 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:12.799042    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:23:12.854769 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:12.854704    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:23:14.650897 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:14.650850    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:23:25.367757 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:25.367709    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:23:25.466733 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:25.466689    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:23:38.051239 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:38.051196    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:23:40.124417 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:40.124382    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-cbqrk"]
Sep 10 19:23:40.124812 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:40.124748    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de" containerName="discovery"
Sep 10 19:23:40.124812 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:40.124760    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de" containerName="discovery"
Sep 10 19:23:40.124812 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:40.124768    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87991782-f257-48e3-8674-ee7097bb0a1f" containerName="istio-proxy"
Sep 10 19:23:40.124812 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:40.124773    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="87991782-f257-48e3-8674-ee7097bb0a1f" containerName="istio-proxy"
Sep 10 19:23:40.124937 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:40.124838    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="fff4cb9d-7eb3-4d4c-8531-6bd7b6da33de" containerName="discovery"
Sep 10 19:23:40.124937 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:40.124846    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="87991782-f257-48e3-8674-ee7097bb0a1f" containerName="istio-proxy"
Sep 10 19:23:40.127885 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:40.127862    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-cbqrk"
Sep 10 19:23:40.130225 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:40.130198    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 10 19:23:40.130362 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:40.130197    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 10 19:23:40.130362 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:40.130243    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 10 19:23:40.130474 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:40.130370    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-r47hc\""
Sep 10 19:23:40.143675 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:40.143650    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-cbqrk"]
Sep 10 19:23:40.281555 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:40.281515    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2x6bv\" (UniqueName: \"kubernetes.io/projected/834ae1eb-c324-493f-9105-a3d5c10d3c1d-kube-api-access-2x6bv\") pod \"dns-operator-controller-manager-7b84cd7c9c-cbqrk\" (UID: \"834ae1eb-c324-493f-9105-a3d5c10d3c1d\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-cbqrk"
Sep 10 19:23:40.382605 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:40.382512    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2x6bv\" (UniqueName: \"kubernetes.io/projected/834ae1eb-c324-493f-9105-a3d5c10d3c1d-kube-api-access-2x6bv\") pod \"dns-operator-controller-manager-7b84cd7c9c-cbqrk\" (UID: \"834ae1eb-c324-493f-9105-a3d5c10d3c1d\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-cbqrk"
Sep 10 19:23:40.395476 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:40.395441    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2x6bv\" (UniqueName: \"kubernetes.io/projected/834ae1eb-c324-493f-9105-a3d5c10d3c1d-kube-api-access-2x6bv\") pod \"dns-operator-controller-manager-7b84cd7c9c-cbqrk\" (UID: \"834ae1eb-c324-493f-9105-a3d5c10d3c1d\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-cbqrk"
Sep 10 19:23:40.438733 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:40.438691    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-cbqrk"
Sep 10 19:23:40.723945 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:40.723870    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-cbqrk"]
Sep 10 19:23:40.726998 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:23:40.726972    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod834ae1eb_c324_493f_9105_a3d5c10d3c1d.slice/crio-b6d023a6727773db4185e44c6058faa2c8cd31e2f31310224ca13a2ab8740d26 WatchSource:0}: Error finding container b6d023a6727773db4185e44c6058faa2c8cd31e2f31310224ca13a2ab8740d26: Status 404 returned error can't find the container with id b6d023a6727773db4185e44c6058faa2c8cd31e2f31310224ca13a2ab8740d26
Sep 10 19:23:40.849658 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:40.849601    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:23:40.952886 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:40.952854    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-cbqrk" event={"ID":"834ae1eb-c324-493f-9105-a3d5c10d3c1d","Type":"ContainerStarted","Data":"b6d023a6727773db4185e44c6058faa2c8cd31e2f31310224ca13a2ab8740d26"}
Sep 10 19:23:42.132161 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:42.132124    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-rqkcc"]
Sep 10 19:23:42.135743 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:42.135717    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-rqkcc"
Sep 10 19:23:42.138166 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:42.138146    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-6fbrx\""
Sep 10 19:23:42.148668 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:42.148641    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-rqkcc"]
Sep 10 19:23:42.302012 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:42.301966    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcjtp\" (UniqueName: \"kubernetes.io/projected/12a848f5-5b73-4e62-bea5-39f47d8e97f9-kube-api-access-lcjtp\") pod \"limitador-operator-controller-manager-74495c69f8-rqkcc\" (UID: \"12a848f5-5b73-4e62-bea5-39f47d8e97f9\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-rqkcc"
Sep 10 19:23:42.403518 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:42.403431    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcjtp\" (UniqueName: \"kubernetes.io/projected/12a848f5-5b73-4e62-bea5-39f47d8e97f9-kube-api-access-lcjtp\") pod \"limitador-operator-controller-manager-74495c69f8-rqkcc\" (UID: \"12a848f5-5b73-4e62-bea5-39f47d8e97f9\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-rqkcc"
Sep 10 19:23:42.414128 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:42.414069    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcjtp\" (UniqueName: \"kubernetes.io/projected/12a848f5-5b73-4e62-bea5-39f47d8e97f9-kube-api-access-lcjtp\") pod \"limitador-operator-controller-manager-74495c69f8-rqkcc\" (UID: \"12a848f5-5b73-4e62-bea5-39f47d8e97f9\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-rqkcc"
Sep 10 19:23:42.449347 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:42.449310    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-rqkcc"
Sep 10 19:23:42.455912 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:42.455870    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:23:42.697155 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:42.697061    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:23:42.859274 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:42.859243    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:23:43.515031 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:43.514988    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-rqkcc"]
Sep 10 19:23:43.519608 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:23:43.519567    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12a848f5_5b73_4e62_bea5_39f47d8e97f9.slice/crio-192fd28bc5866dadca1d2122d10594d719f8d11aa40c9f6f88be8b78e4e9f36d WatchSource:0}: Error finding container 192fd28bc5866dadca1d2122d10594d719f8d11aa40c9f6f88be8b78e4e9f36d: Status 404 returned error can't find the container with id 192fd28bc5866dadca1d2122d10594d719f8d11aa40c9f6f88be8b78e4e9f36d
Sep 10 19:23:43.970762 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:43.970660    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-rqkcc" event={"ID":"12a848f5-5b73-4e62-bea5-39f47d8e97f9","Type":"ContainerStarted","Data":"192fd28bc5866dadca1d2122d10594d719f8d11aa40c9f6f88be8b78e4e9f36d"}
Sep 10 19:23:43.972332 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:43.972305    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-cbqrk" event={"ID":"834ae1eb-c324-493f-9105-a3d5c10d3c1d","Type":"ContainerStarted","Data":"c4cd14882a0573f4ee56f1938520362ea2635b166de209134bf1047960cd7509"}
Sep 10 19:23:43.972462 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:43.972373    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-cbqrk"
Sep 10 19:23:43.994042 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:43.993985    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-cbqrk" podStartSLOduration=1.414161798 podStartE2EDuration="3.993971307s" totalImagesPullingTime="2.579809509s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:23:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:23:43.991568319 +0000 UTC m=+479.592475560" watchObservedRunningTime="2026-09-10 19:23:43.993971307 +0000 UTC m=+479.594878546"
Sep 10 19:23:44.095064 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:44.095002    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:23:45.133001 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:45.132916    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-zgt99"]
Sep 10 19:23:45.136713 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:45.136687    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-zgt99"
Sep 10 19:23:45.138864 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:45.138842    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-j9bhw\""
Sep 10 19:23:45.151079 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:45.151052    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-zgt99"]
Sep 10 19:23:45.229631 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:45.229592    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7xnd\" (UniqueName: \"kubernetes.io/projected/e66da5a2-eef7-4e40-b4db-46f69585732e-kube-api-access-b7xnd\") pod \"authorino-operator-76b478bd78-zgt99\" (UID: \"e66da5a2-eef7-4e40-b4db-46f69585732e\") " pod="kuadrant-system/authorino-operator-76b478bd78-zgt99"
Sep 10 19:23:45.331077 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:45.331034    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7xnd\" (UniqueName: \"kubernetes.io/projected/e66da5a2-eef7-4e40-b4db-46f69585732e-kube-api-access-b7xnd\") pod \"authorino-operator-76b478bd78-zgt99\" (UID: \"e66da5a2-eef7-4e40-b4db-46f69585732e\") " pod="kuadrant-system/authorino-operator-76b478bd78-zgt99"
Sep 10 19:23:45.340059 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:45.340030    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7xnd\" (UniqueName: \"kubernetes.io/projected/e66da5a2-eef7-4e40-b4db-46f69585732e-kube-api-access-b7xnd\") pod \"authorino-operator-76b478bd78-zgt99\" (UID: \"e66da5a2-eef7-4e40-b4db-46f69585732e\") " pod="kuadrant-system/authorino-operator-76b478bd78-zgt99"
Sep 10 19:23:45.448536 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:45.448511    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-zgt99"
Sep 10 19:23:45.463744 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:45.463694    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:23:45.726685 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:45.726451    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-zgt99"]
Sep 10 19:23:45.729299 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:23:45.729266    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode66da5a2_eef7_4e40_b4db_46f69585732e.slice/crio-87ca2babcf3399815528be4e7bb0d1b7f9db8c000e57aef959f011c564d43e4c WatchSource:0}: Error finding container 87ca2babcf3399815528be4e7bb0d1b7f9db8c000e57aef959f011c564d43e4c: Status 404 returned error can't find the container with id 87ca2babcf3399815528be4e7bb0d1b7f9db8c000e57aef959f011c564d43e4c
Sep 10 19:23:45.860577 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:45.860541    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:23:45.983036 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:45.982949    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-zgt99" event={"ID":"e66da5a2-eef7-4e40-b4db-46f69585732e","Type":"ContainerStarted","Data":"87ca2babcf3399815528be4e7bb0d1b7f9db8c000e57aef959f011c564d43e4c"}
Sep 10 19:23:45.986862 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:45.986836    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-rqkcc" event={"ID":"12a848f5-5b73-4e62-bea5-39f47d8e97f9","Type":"ContainerStarted","Data":"473e07229860218e4ed9e48cd47304016e15f2e1191ed7157c60db12f2689da3"}
Sep 10 19:23:45.987001 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:45.986983    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-rqkcc"
Sep 10 19:23:46.025681 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:46.025609    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-rqkcc" podStartSLOduration=2.173755408 podStartE2EDuration="4.025590283s" totalImagesPullingTime="1.851834875s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:23:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:23:46.023301649 +0000 UTC m=+481.624208890" watchObservedRunningTime="2026-09-10 19:23:46.025590283 +0000 UTC m=+481.626497523"
Sep 10 19:23:46.143642 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:46.143588    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:23:46.240722 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:46.240644    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:23:46.341477 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:46.341433    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:23:47.684519 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:47.684480    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:23:47.790112 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:47.790062    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:23:47.999187 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:47.999154    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-zgt99" event={"ID":"e66da5a2-eef7-4e40-b4db-46f69585732e","Type":"ContainerStarted","Data":"908fda13aecf82baa7135fbc11f56273abde07e4c1507814353c6182d857833d"}
Sep 10 19:23:47.999354 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:47.999264    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-zgt99"
Sep 10 19:23:49.541977 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:49.541926    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-zgt99" podStartSLOduration=3.090064562 podStartE2EDuration="4.541907853s" totalImagesPullingTime="1.451843291s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:23:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:23:48.019393938 +0000 UTC m=+483.620301174" watchObservedRunningTime="2026-09-10 19:23:49.541907853 +0000 UTC m=+485.142815093"
Sep 10 19:23:49.543658 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:49.543637    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j7vjw"]
Sep 10 19:23:49.547134 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:49.547118    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j7vjw"
Sep 10 19:23:49.549416 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:49.549397    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-qpnj5\""
Sep 10 19:23:49.558672 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:49.558650    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j7vjw"]
Sep 10 19:23:49.670917 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:49.670880    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5nwm\" (UniqueName: \"kubernetes.io/projected/c8d87cc4-77f4-4081-95b1-357ad2a4fbbd-kube-api-access-h5nwm\") pod \"kuadrant-operator-controller-manager-76f8f686bf-j7vjw\" (UID: \"c8d87cc4-77f4-4081-95b1-357ad2a4fbbd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j7vjw"
Sep 10 19:23:49.671089 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:49.671035    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c8d87cc4-77f4-4081-95b1-357ad2a4fbbd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-j7vjw\" (UID: \"c8d87cc4-77f4-4081-95b1-357ad2a4fbbd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j7vjw"
Sep 10 19:23:49.771606 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:49.771569    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c8d87cc4-77f4-4081-95b1-357ad2a4fbbd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-j7vjw\" (UID: \"c8d87cc4-77f4-4081-95b1-357ad2a4fbbd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j7vjw"
Sep 10 19:23:49.771829 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:49.771615    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5nwm\" (UniqueName: \"kubernetes.io/projected/c8d87cc4-77f4-4081-95b1-357ad2a4fbbd-kube-api-access-h5nwm\") pod \"kuadrant-operator-controller-manager-76f8f686bf-j7vjw\" (UID: \"c8d87cc4-77f4-4081-95b1-357ad2a4fbbd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j7vjw"
Sep 10 19:23:49.772047 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:49.772026    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c8d87cc4-77f4-4081-95b1-357ad2a4fbbd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-j7vjw\" (UID: \"c8d87cc4-77f4-4081-95b1-357ad2a4fbbd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j7vjw"
Sep 10 19:23:49.781145 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:49.781114    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5nwm\" (UniqueName: \"kubernetes.io/projected/c8d87cc4-77f4-4081-95b1-357ad2a4fbbd-kube-api-access-h5nwm\") pod \"kuadrant-operator-controller-manager-76f8f686bf-j7vjw\" (UID: \"c8d87cc4-77f4-4081-95b1-357ad2a4fbbd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j7vjw"
Sep 10 19:23:49.857189 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:49.857088    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j7vjw"
Sep 10 19:23:50.133464 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:50.133425    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j7vjw"]
Sep 10 19:23:50.136063 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:23:50.136034    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8d87cc4_77f4_4081_95b1_357ad2a4fbbd.slice/crio-2f545346feb2d4ad3b0a2b3954f651cd1f4b3d6b4c91b2745846eb896b96edd8 WatchSource:0}: Error finding container 2f545346feb2d4ad3b0a2b3954f651cd1f4b3d6b4c91b2745846eb896b96edd8: Status 404 returned error can't find the container with id 2f545346feb2d4ad3b0a2b3954f651cd1f4b3d6b4c91b2745846eb896b96edd8
Sep 10 19:23:50.485900 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:50.485823    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:23:50.595293 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:50.595259    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:23:51.015337 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:51.015298    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j7vjw" event={"ID":"c8d87cc4-77f4-4081-95b1-357ad2a4fbbd","Type":"ContainerStarted","Data":"2f545346feb2d4ad3b0a2b3954f651cd1f4b3d6b4c91b2745846eb896b96edd8"}
Sep 10 19:23:53.867881 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:53.867841    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:23:53.969218 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:53.969193    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:23:54.034842 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:54.034803    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j7vjw" event={"ID":"c8d87cc4-77f4-4081-95b1-357ad2a4fbbd","Type":"ContainerStarted","Data":"880f7173aeab201e6ebd3b4cef93854f7e08a7d1a6770a53c1e20b2179147802"}
Sep 10 19:23:54.034999 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:54.034892    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j7vjw"
Sep 10 19:23:54.057963 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:54.057913    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j7vjw" podStartSLOduration=1.252923732 podStartE2EDuration="5.057898453s" totalImagesPullingTime="3.804974721s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:23:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:23:54.055443209 +0000 UTC m=+489.656350460" watchObservedRunningTime="2026-09-10 19:23:54.057898453 +0000 UTC m=+489.658805692"
Sep 10 19:23:54.979776 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:54.979748    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-cbqrk"
Sep 10 19:23:56.700213 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:56.700154    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:23:56.854117 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:56.854064    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:23:56.994709 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:56.994609    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-rqkcc"
Sep 10 19:23:58.277836 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:58.277798    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:23:58.379508 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:58.379471    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:23:59.007130 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:23:59.007094    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-zgt99"
Sep 10 19:24:03.003521 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:03.003486    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:24:03.103273 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:03.103225    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:24:03.171342 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:03.171299    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:24:05.042368 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:05.042326    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j7vjw"
Sep 10 19:24:06.022881 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:06.022847    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-jrrrf"]
Sep 10 19:24:06.026562 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:06.026543    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-jrrrf"
Sep 10 19:24:06.039254 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:06.039226    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-jrrrf"]
Sep 10 19:24:06.087468 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:06.087429    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpfgc\" (UniqueName: \"kubernetes.io/projected/dff1fae8-bd43-4e76-8b18-425481205937-kube-api-access-lpfgc\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-jrrrf\" (UID: \"dff1fae8-bd43-4e76-8b18-425481205937\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-jrrrf"
Sep 10 19:24:06.087918 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:06.087489    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dff1fae8-bd43-4e76-8b18-425481205937-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-jrrrf\" (UID: \"dff1fae8-bd43-4e76-8b18-425481205937\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-jrrrf"
Sep 10 19:24:06.188503 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:06.188465    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpfgc\" (UniqueName: \"kubernetes.io/projected/dff1fae8-bd43-4e76-8b18-425481205937-kube-api-access-lpfgc\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-jrrrf\" (UID: \"dff1fae8-bd43-4e76-8b18-425481205937\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-jrrrf"
Sep 10 19:24:06.188734 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:06.188520    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dff1fae8-bd43-4e76-8b18-425481205937-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-jrrrf\" (UID: \"dff1fae8-bd43-4e76-8b18-425481205937\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-jrrrf"
Sep 10 19:24:06.188904 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:06.188887    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dff1fae8-bd43-4e76-8b18-425481205937-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-jrrrf\" (UID: \"dff1fae8-bd43-4e76-8b18-425481205937\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-jrrrf"
Sep 10 19:24:06.197989 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:06.197959    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpfgc\" (UniqueName: \"kubernetes.io/projected/dff1fae8-bd43-4e76-8b18-425481205937-kube-api-access-lpfgc\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-jrrrf\" (UID: \"dff1fae8-bd43-4e76-8b18-425481205937\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-jrrrf"
Sep 10 19:24:06.336886 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:06.336792    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-jrrrf"
Sep 10 19:24:06.617120 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:06.616805    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-jrrrf"]
Sep 10 19:24:06.619377 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:24:06.619340    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddff1fae8_bd43_4e76_8b18_425481205937.slice/crio-3185200b5b8c112e750e93b0d0aad89fe1ad405f3b2ed82cfb3e41ede65122ba WatchSource:0}: Error finding container 3185200b5b8c112e750e93b0d0aad89fe1ad405f3b2ed82cfb3e41ede65122ba: Status 404 returned error can't find the container with id 3185200b5b8c112e750e93b0d0aad89fe1ad405f3b2ed82cfb3e41ede65122ba
Sep 10 19:24:07.097422 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:07.097390    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-jrrrf" event={"ID":"dff1fae8-bd43-4e76-8b18-425481205937","Type":"ContainerStarted","Data":"20a8941783b921e303781ea0538290cbc44d314b3946e644eb7110e6cfa83604"}
Sep 10 19:24:07.097834 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:07.097440    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-jrrrf" event={"ID":"dff1fae8-bd43-4e76-8b18-425481205937","Type":"ContainerStarted","Data":"3185200b5b8c112e750e93b0d0aad89fe1ad405f3b2ed82cfb3e41ede65122ba"}
Sep 10 19:24:07.097834 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:07.097476    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-jrrrf"
Sep 10 19:24:07.120015 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:07.119959    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-jrrrf" podStartSLOduration=1.119943094 podStartE2EDuration="1.119943094s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:24:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:24:07.11792664 +0000 UTC m=+502.718833882" watchObservedRunningTime="2026-09-10 19:24:07.119943094 +0000 UTC m=+502.720850334"
Sep 10 19:24:07.159783 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:07.159742    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-jrrrf"]
Sep 10 19:24:07.170437 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:07.170402    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-jrrrf"]
Sep 10 19:24:07.183940 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:07.183908    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-qqb2z"]
Sep 10 19:24:07.184327 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:07.184310    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dff1fae8-bd43-4e76-8b18-425481205937" containerName="manager"
Sep 10 19:24:07.184424 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:07.184330    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="dff1fae8-bd43-4e76-8b18-425481205937" containerName="manager"
Sep 10 19:24:07.184478 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:07.184442    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="dff1fae8-bd43-4e76-8b18-425481205937" containerName="manager"
Sep 10 19:24:07.187688 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:07.187664    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-qqb2z"
Sep 10 19:24:07.191169 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:07.191142    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j7vjw"]
Sep 10 19:24:07.191343 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:07.191318    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j7vjw" podUID="c8d87cc4-77f4-4081-95b1-357ad2a4fbbd" containerName="manager" containerID="cri-o://880f7173aeab201e6ebd3b4cef93854f7e08a7d1a6770a53c1e20b2179147802" gracePeriod=2
Sep 10 19:24:07.196890 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:07.196860    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4d786de9-def7-49df-8390-d22d7f32bdfa-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-qqb2z\" (UID: \"4d786de9-def7-49df-8390-d22d7f32bdfa\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-qqb2z"
Sep 10 19:24:07.197526 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:07.197504    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ksrw\" (UniqueName: \"kubernetes.io/projected/4d786de9-def7-49df-8390-d22d7f32bdfa-kube-api-access-4ksrw\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-qqb2z\" (UID: \"4d786de9-def7-49df-8390-d22d7f32bdfa\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-qqb2z"
Sep 10 19:24:07.201609 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:07.201577    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j7vjw"]
Sep 10 19:24:07.204157 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:07.204061    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-qqb2z"]
Sep 10 19:24:07.207037 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:07.207013    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-rqkcc"]
Sep 10 19:24:07.207230 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:07.207208    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-rqkcc" podUID="12a848f5-5b73-4e62-bea5-39f47d8e97f9" containerName="manager" containerID="cri-o://473e07229860218e4ed9e48cd47304016e15f2e1191ed7157c60db12f2689da3" gracePeriod=2
Sep 10 19:24:07.217801 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:07.217754    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-z5xdl"]
Sep 10 19:24:07.218125 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:07.218112    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8d87cc4-77f4-4081-95b1-357ad2a4fbbd" containerName="manager"
Sep 10 19:24:07.218172 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:07.218129    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8d87cc4-77f4-4081-95b1-357ad2a4fbbd" containerName="manager"
Sep 10 19:24:07.218204 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:07.218191    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8d87cc4-77f4-4081-95b1-357ad2a4fbbd" containerName="manager"
Sep 10 19:24:07.221616 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:07.221593    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-z5xdl"
Sep 10 19:24:07.222370 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:07.222346    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-rqkcc"]
Sep 10 19:24:07.223788 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:24:07.223757    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-rqkcc\" is forbidden: User \"system:node:ip-10-0-129-230.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-230.ec2.internal' and this object" podUID="12a848f5-5b73-4e62-bea5-39f47d8e97f9" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-rqkcc"
Sep 10 19:24:07.225652 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:24:07.225609    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-j7vjw\" is forbidden: User \"system:node:ip-10-0-129-230.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-230.ec2.internal' and this object" podUID="c8d87cc4-77f4-4081-95b1-357ad2a4fbbd" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j7vjw"
Sep 10 19:24:07.230665 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:07.230639    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-psz7d"]
Sep 10 19:24:07.231150 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:07.231129    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12a848f5-5b73-4e62-bea5-39f47d8e97f9" containerName="manager"
Sep 10 19:24:07.231291 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:07.231277    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="12a848f5-5b73-4e62-bea5-39f47d8e97f9" containerName="manager"
Sep 10 19:24:07.231541 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:07.231527    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="12a848f5-5b73-4e62-bea5-39f47d8e97f9" containerName="manager"
Sep 10 19:24:07.234930 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:07.234909    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-z5xdl"]
Sep 10 19:24:07.235084 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:07.235012    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-psz7d"
Sep 10 19:24:07.246379 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:07.246352    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-psz7d"]
Sep 10 19:24:07.267922 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:24:07.267882    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-rqkcc\" is forbidden: User \"system:node:ip-10-0-129-230.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-230.ec2.internal' and this object" podUID="12a848f5-5b73-4e62-bea5-39f47d8e97f9" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-rqkcc"
Sep 10 19:24:07.269790 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:24:07.269752    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-j7vjw\" is forbidden: User \"system:node:ip-10-0-129-230.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-230.ec2.internal' and this object" podUID="c8d87cc4-77f4-4081-95b1-357ad2a4fbbd" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j7vjw"
Sep 10 19:24:07.299743 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:07.299712    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ksrw\" (UniqueName: \"kubernetes.io/projected/4d786de9-def7-49df-8390-d22d7f32bdfa-kube-api-access-4ksrw\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-qqb2z\" (UID: \"4d786de9-def7-49df-8390-d22d7f32bdfa\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-qqb2z"
Sep 10 19:24:07.299877 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:07.299795    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chcxf\" (UniqueName: \"kubernetes.io/projected/9be52691-ec26-45cf-80e9-fa75c08b47fb-kube-api-access-chcxf\") pod \"limitador-operator-controller-manager-74495c69f8-psz7d\" (UID: \"9be52691-ec26-45cf-80e9-fa75c08b47fb\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-psz7d"
Sep 10 19:24:07.299877 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:07.299837    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpfmp\" (UniqueName: \"kubernetes.io/projected/57f17807-ae0d-4829-ac5d-00b338ca5c6d-kube-api-access-dpfmp\") pod \"kuadrant-operator-controller-manager-76f8f686bf-z5xdl\" (UID: \"57f17807-ae0d-4829-ac5d-00b338ca5c6d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-z5xdl"
Sep 10 19:24:07.299877 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:07.299867    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4d786de9-def7-49df-8390-d22d7f32bdfa-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-qqb2z\" (UID: \"4d786de9-def7-49df-8390-d22d7f32bdfa\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-qqb2z"
Sep 10 19:24:07.299992 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:07.299931    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/57f17807-ae0d-4829-ac5d-00b338ca5c6d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-z5xdl\" (UID: \"57f17807-ae0d-4829-ac5d-00b338ca5c6d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-z5xdl"
Sep 10 19:24:07.300235 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:07.300214    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4d786de9-def7-49df-8390-d22d7f32bdfa-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-qqb2z\" (UID: \"4d786de9-def7-49df-8390-d22d7f32bdfa\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-qqb2z"
Sep 10 19:24:07.310694 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:07.310668    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ksrw\" (UniqueName: \"kubernetes.io/projected/4d786de9-def7-49df-8390-d22d7f32bdfa-kube-api-access-4ksrw\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-qqb2z\" (UID: \"4d786de9-def7-49df-8390-d22d7f32bdfa\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-qqb2z"
Sep 10 19:24:07.401149 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:07.401110    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chcxf\" (UniqueName: \"kubernetes.io/projected/9be52691-ec26-45cf-80e9-fa75c08b47fb-kube-api-access-chcxf\") pod \"limitador-operator-controller-manager-74495c69f8-psz7d\" (UID: \"9be52691-ec26-45cf-80e9-fa75c08b47fb\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-psz7d"
Sep 10 19:24:07.401890 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:07.401178    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpfmp\" (UniqueName: \"kubernetes.io/projected/57f17807-ae0d-4829-ac5d-00b338ca5c6d-kube-api-access-dpfmp\") pod \"kuadrant-operator-controller-manager-76f8f686bf-z5xdl\" (UID: \"57f17807-ae0d-4829-ac5d-00b338ca5c6d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-z5xdl"
Sep 10 19:24:07.401890 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:07.401217    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/57f17807-ae0d-4829-ac5d-00b338ca5c6d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-z5xdl\" (UID: \"57f17807-ae0d-4829-ac5d-00b338ca5c6d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-z5xdl"
Sep 10 19:24:07.401890 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:07.401701    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/57f17807-ae0d-4829-ac5d-00b338ca5c6d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-z5xdl\" (UID: \"57f17807-ae0d-4829-ac5d-00b338ca5c6d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-z5xdl"
Sep 10 19:24:07.411974 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:07.411935    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-chcxf\" (UniqueName: \"kubernetes.io/projected/9be52691-ec26-45cf-80e9-fa75c08b47fb-kube-api-access-chcxf\") pod \"limitador-operator-controller-manager-74495c69f8-psz7d\" (UID: \"9be52691-ec26-45cf-80e9-fa75c08b47fb\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-psz7d"
Sep 10 19:24:07.412333 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:07.412309    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpfmp\" (UniqueName: \"kubernetes.io/projected/57f17807-ae0d-4829-ac5d-00b338ca5c6d-kube-api-access-dpfmp\") pod \"kuadrant-operator-controller-manager-76f8f686bf-z5xdl\" (UID: \"57f17807-ae0d-4829-ac5d-00b338ca5c6d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-z5xdl"
Sep 10 19:24:07.441970 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:07.441948    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j7vjw"
Sep 10 19:24:07.443990 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:07.443973    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-rqkcc"
Sep 10 19:24:07.444609 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:24:07.444582    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-rqkcc\" is forbidden: User \"system:node:ip-10-0-129-230.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-230.ec2.internal' and this object" podUID="12a848f5-5b73-4e62-bea5-39f47d8e97f9" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-rqkcc"
Sep 10 19:24:07.446470 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:24:07.446444    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-j7vjw\" is forbidden: User \"system:node:ip-10-0-129-230.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-230.ec2.internal' and this object" podUID="c8d87cc4-77f4-4081-95b1-357ad2a4fbbd" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j7vjw"
Sep 10 19:24:07.448071 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:24:07.448052    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-rqkcc\" is forbidden: User \"system:node:ip-10-0-129-230.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-230.ec2.internal' and this object" podUID="12a848f5-5b73-4e62-bea5-39f47d8e97f9" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-rqkcc"
Sep 10 19:24:07.449693 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:24:07.449674    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-j7vjw\" is forbidden: User \"system:node:ip-10-0-129-230.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-230.ec2.internal' and this object" podUID="c8d87cc4-77f4-4081-95b1-357ad2a4fbbd" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j7vjw"
Sep 10 19:24:07.502337 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:07.502299    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h5nwm\" (UniqueName: \"kubernetes.io/projected/c8d87cc4-77f4-4081-95b1-357ad2a4fbbd-kube-api-access-h5nwm\") pod \"c8d87cc4-77f4-4081-95b1-357ad2a4fbbd\" (UID: \"c8d87cc4-77f4-4081-95b1-357ad2a4fbbd\") "
Sep 10 19:24:07.502337 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:07.502343    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lcjtp\" (UniqueName: \"kubernetes.io/projected/12a848f5-5b73-4e62-bea5-39f47d8e97f9-kube-api-access-lcjtp\") pod \"12a848f5-5b73-4e62-bea5-39f47d8e97f9\" (UID: \"12a848f5-5b73-4e62-bea5-39f47d8e97f9\") "
Sep 10 19:24:07.502586 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:07.502410    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c8d87cc4-77f4-4081-95b1-357ad2a4fbbd-extensions-socket-volume\") pod \"c8d87cc4-77f4-4081-95b1-357ad2a4fbbd\" (UID: \"c8d87cc4-77f4-4081-95b1-357ad2a4fbbd\") "
Sep 10 19:24:07.502748 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:07.502722    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-qqb2z"
Sep 10 19:24:07.503084 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:07.503050    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8d87cc4-77f4-4081-95b1-357ad2a4fbbd-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "c8d87cc4-77f4-4081-95b1-357ad2a4fbbd" (UID: "c8d87cc4-77f4-4081-95b1-357ad2a4fbbd"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:24:07.504684 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:07.504658    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12a848f5-5b73-4e62-bea5-39f47d8e97f9-kube-api-access-lcjtp" (OuterVolumeSpecName: "kube-api-access-lcjtp") pod "12a848f5-5b73-4e62-bea5-39f47d8e97f9" (UID: "12a848f5-5b73-4e62-bea5-39f47d8e97f9"). InnerVolumeSpecName "kube-api-access-lcjtp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:24:07.504769 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:07.504694    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8d87cc4-77f4-4081-95b1-357ad2a4fbbd-kube-api-access-h5nwm" (OuterVolumeSpecName: "kube-api-access-h5nwm") pod "c8d87cc4-77f4-4081-95b1-357ad2a4fbbd" (UID: "c8d87cc4-77f4-4081-95b1-357ad2a4fbbd"). InnerVolumeSpecName "kube-api-access-h5nwm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:24:07.600035 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:07.599991    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-z5xdl"
Sep 10 19:24:07.603359 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:07.603323    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h5nwm\" (UniqueName: \"kubernetes.io/projected/c8d87cc4-77f4-4081-95b1-357ad2a4fbbd-kube-api-access-h5nwm\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:24:07.603359 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:07.603356    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lcjtp\" (UniqueName: \"kubernetes.io/projected/12a848f5-5b73-4e62-bea5-39f47d8e97f9-kube-api-access-lcjtp\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:24:07.603532 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:07.603371    2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c8d87cc4-77f4-4081-95b1-357ad2a4fbbd-extensions-socket-volume\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:24:07.606429 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:07.606408    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-psz7d"
Sep 10 19:24:07.776995 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:07.776968    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-qqb2z"]
Sep 10 19:24:07.778926 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:24:07.778899    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d786de9_def7_49df_8390_d22d7f32bdfa.slice/crio-af25a6a42fdeb29d18764b884d6673b142a09ab2747c99caec14418fa06c9f23 WatchSource:0}: Error finding container af25a6a42fdeb29d18764b884d6673b142a09ab2747c99caec14418fa06c9f23: Status 404 returned error can't find the container with id af25a6a42fdeb29d18764b884d6673b142a09ab2747c99caec14418fa06c9f23
Sep 10 19:24:07.878338 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:07.878280    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-z5xdl"]
Sep 10 19:24:07.881372 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:24:07.881342    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57f17807_ae0d_4829_ac5d_00b338ca5c6d.slice/crio-70b57bc1770a4f10eeffedc20f93a7b59c23c4299f6a921df98c17b9e10efa8c WatchSource:0}: Error finding container 70b57bc1770a4f10eeffedc20f93a7b59c23c4299f6a921df98c17b9e10efa8c: Status 404 returned error can't find the container with id 70b57bc1770a4f10eeffedc20f93a7b59c23c4299f6a921df98c17b9e10efa8c
Sep 10 19:24:07.881998 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:07.881976    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-psz7d"]
Sep 10 19:24:07.884819 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:24:07.884790    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9be52691_ec26_45cf_80e9_fa75c08b47fb.slice/crio-f12f7ce7c02b3d95ddfc19ce82c0b692a6d05f0c9b5b9fdcc60ab622d4783b85 WatchSource:0}: Error finding container f12f7ce7c02b3d95ddfc19ce82c0b692a6d05f0c9b5b9fdcc60ab622d4783b85: Status 404 returned error can't find the container with id f12f7ce7c02b3d95ddfc19ce82c0b692a6d05f0c9b5b9fdcc60ab622d4783b85
Sep 10 19:24:08.103512 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:08.103472    2564 generic.go:356] "Generic (PLEG): container finished" podID="c8d87cc4-77f4-4081-95b1-357ad2a4fbbd" containerID="880f7173aeab201e6ebd3b4cef93854f7e08a7d1a6770a53c1e20b2179147802" exitCode=0
Sep 10 19:24:08.104010 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:08.103533    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j7vjw"
Sep 10 19:24:08.104010 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:08.103570    2564 scope.go:117] "RemoveContainer" containerID="880f7173aeab201e6ebd3b4cef93854f7e08a7d1a6770a53c1e20b2179147802"
Sep 10 19:24:08.105580 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:08.105553    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-psz7d" event={"ID":"9be52691-ec26-45cf-80e9-fa75c08b47fb","Type":"ContainerStarted","Data":"5b952dcf4c6bed0f9854d588288ec83497a1088a56ba0ed78d8b9bbf765fa1da"}
Sep 10 19:24:08.105886 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:08.105863    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-psz7d" event={"ID":"9be52691-ec26-45cf-80e9-fa75c08b47fb","Type":"ContainerStarted","Data":"f12f7ce7c02b3d95ddfc19ce82c0b692a6d05f0c9b5b9fdcc60ab622d4783b85"}
Sep 10 19:24:08.105970 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:08.105904    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-psz7d"
Sep 10 19:24:08.106038 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:24:08.105953    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-rqkcc\" is forbidden: User \"system:node:ip-10-0-129-230.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-230.ec2.internal' and this object" podUID="12a848f5-5b73-4e62-bea5-39f47d8e97f9" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-rqkcc"
Sep 10 19:24:08.107599 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:08.107576    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-qqb2z" event={"ID":"4d786de9-def7-49df-8390-d22d7f32bdfa","Type":"ContainerStarted","Data":"9a14673a16e5f5c010686635fc4649aa4f9d2c0a8c631624c839c7db82010be7"}
Sep 10 19:24:08.107709 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:08.107640    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-qqb2z" event={"ID":"4d786de9-def7-49df-8390-d22d7f32bdfa","Type":"ContainerStarted","Data":"af25a6a42fdeb29d18764b884d6673b142a09ab2747c99caec14418fa06c9f23"}
Sep 10 19:24:08.107709 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:08.107673    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-qqb2z"
Sep 10 19:24:08.107965 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:24:08.107938    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-j7vjw\" is forbidden: User \"system:node:ip-10-0-129-230.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-230.ec2.internal' and this object" podUID="c8d87cc4-77f4-4081-95b1-357ad2a4fbbd" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j7vjw"
Sep 10 19:24:08.109197 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:08.109172    2564 generic.go:356] "Generic (PLEG): container finished" podID="12a848f5-5b73-4e62-bea5-39f47d8e97f9" containerID="473e07229860218e4ed9e48cd47304016e15f2e1191ed7157c60db12f2689da3" exitCode=0
Sep 10 19:24:08.109280 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:08.109224    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-rqkcc"
Sep 10 19:24:08.109637 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:24:08.109597    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-rqkcc\" is forbidden: User \"system:node:ip-10-0-129-230.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-230.ec2.internal' and this object" podUID="12a848f5-5b73-4e62-bea5-39f47d8e97f9" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-rqkcc"
Sep 10 19:24:08.111129 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:08.111108    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-z5xdl" event={"ID":"57f17807-ae0d-4829-ac5d-00b338ca5c6d","Type":"ContainerStarted","Data":"e1348cfd8d535f61d77b0d5aaf0cd853e41307721c0172f90cf0821e8a098fcd"}
Sep 10 19:24:08.111220 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:08.111147    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-z5xdl" event={"ID":"57f17807-ae0d-4829-ac5d-00b338ca5c6d","Type":"ContainerStarted","Data":"70b57bc1770a4f10eeffedc20f93a7b59c23c4299f6a921df98c17b9e10efa8c"}
Sep 10 19:24:08.111283 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:08.111263    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-z5xdl"
Sep 10 19:24:08.111330 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:24:08.111297    2564 kuberuntime_manager.go:649] "Missing actuated resource record" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-jrrrf" container="manager"
Sep 10 19:24:08.111799 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:24:08.111777    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-j7vjw\" is forbidden: User \"system:node:ip-10-0-129-230.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-230.ec2.internal' and this object" podUID="c8d87cc4-77f4-4081-95b1-357ad2a4fbbd" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j7vjw"
Sep 10 19:24:08.114547 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:08.114528    2564 scope.go:117] "RemoveContainer" containerID="880f7173aeab201e6ebd3b4cef93854f7e08a7d1a6770a53c1e20b2179147802"
Sep 10 19:24:08.114928 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:24:08.114838    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"880f7173aeab201e6ebd3b4cef93854f7e08a7d1a6770a53c1e20b2179147802\": container with ID starting with 880f7173aeab201e6ebd3b4cef93854f7e08a7d1a6770a53c1e20b2179147802 not found: ID does not exist" containerID="880f7173aeab201e6ebd3b4cef93854f7e08a7d1a6770a53c1e20b2179147802"
Sep 10 19:24:08.114928 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:08.114876    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"880f7173aeab201e6ebd3b4cef93854f7e08a7d1a6770a53c1e20b2179147802"} err="failed to get container status \"880f7173aeab201e6ebd3b4cef93854f7e08a7d1a6770a53c1e20b2179147802\": rpc error: code = NotFound desc = could not find container \"880f7173aeab201e6ebd3b4cef93854f7e08a7d1a6770a53c1e20b2179147802\": container with ID starting with 880f7173aeab201e6ebd3b4cef93854f7e08a7d1a6770a53c1e20b2179147802 not found: ID does not exist"
Sep 10 19:24:08.114928 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:08.114900    2564 scope.go:117] "RemoveContainer" containerID="473e07229860218e4ed9e48cd47304016e15f2e1191ed7157c60db12f2689da3"
Sep 10 19:24:08.125254 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:08.125234    2564 scope.go:117] "RemoveContainer" containerID="473e07229860218e4ed9e48cd47304016e15f2e1191ed7157c60db12f2689da3"
Sep 10 19:24:08.125547 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:24:08.125530    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"473e07229860218e4ed9e48cd47304016e15f2e1191ed7157c60db12f2689da3\": container with ID starting with 473e07229860218e4ed9e48cd47304016e15f2e1191ed7157c60db12f2689da3 not found: ID does not exist" containerID="473e07229860218e4ed9e48cd47304016e15f2e1191ed7157c60db12f2689da3"
Sep 10 19:24:08.125593 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:08.125554    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"473e07229860218e4ed9e48cd47304016e15f2e1191ed7157c60db12f2689da3"} err="failed to get container status \"473e07229860218e4ed9e48cd47304016e15f2e1191ed7157c60db12f2689da3\": rpc error: code = NotFound desc = could not find container \"473e07229860218e4ed9e48cd47304016e15f2e1191ed7157c60db12f2689da3\": container with ID starting with 473e07229860218e4ed9e48cd47304016e15f2e1191ed7157c60db12f2689da3 not found: ID does not exist"
Sep 10 19:24:08.132448 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:08.132399    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-qqb2z" podStartSLOduration=1.132385385 podStartE2EDuration="1.132385385s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:24:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:24:08.131543993 +0000 UTC m=+503.732451234" watchObservedRunningTime="2026-09-10 19:24:08.132385385 +0000 UTC m=+503.733292619"
Sep 10 19:24:08.157919 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:08.157813    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-psz7d" podStartSLOduration=1.157800897 podStartE2EDuration="1.157800897s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:24:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:24:08.156262461 +0000 UTC m=+503.757169703" watchObservedRunningTime="2026-09-10 19:24:08.157800897 +0000 UTC m=+503.758708137"
Sep 10 19:24:08.158215 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:24:08.158189    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-67c5dcb48c-jrrrf\" is forbidden: User \"system:node:ip-10-0-129-230.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-230.ec2.internal' and this object" podUID="dff1fae8-bd43-4e76-8b18-425481205937" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-jrrrf"
Sep 10 19:24:08.179887 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:08.179833    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-z5xdl" podStartSLOduration=1.179815798 podStartE2EDuration="1.179815798s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:24:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:24:08.178809214 +0000 UTC m=+503.779716456" watchObservedRunningTime="2026-09-10 19:24:08.179815798 +0000 UTC m=+503.780723037"
Sep 10 19:24:08.180720 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:24:08.180693    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-rqkcc\" is forbidden: User \"system:node:ip-10-0-129-230.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-230.ec2.internal' and this object" podUID="12a848f5-5b73-4e62-bea5-39f47d8e97f9" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-rqkcc"
Sep 10 19:24:08.182572 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:24:08.182539    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-j7vjw\" is forbidden: User \"system:node:ip-10-0-129-230.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-230.ec2.internal' and this object" podUID="c8d87cc4-77f4-4081-95b1-357ad2a4fbbd" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j7vjw"
Sep 10 19:24:08.948651 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:08.948592    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12a848f5-5b73-4e62-bea5-39f47d8e97f9" path="/var/lib/kubelet/pods/12a848f5-5b73-4e62-bea5-39f47d8e97f9/volumes"
Sep 10 19:24:08.949068 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:08.949046    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8d87cc4-77f4-4081-95b1-357ad2a4fbbd" path="/var/lib/kubelet/pods/c8d87cc4-77f4-4081-95b1-357ad2a4fbbd/volumes"
Sep 10 19:24:09.117514 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:09.117478    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-jrrrf" podUID="dff1fae8-bd43-4e76-8b18-425481205937" containerName="manager" containerID="cri-o://20a8941783b921e303781ea0538290cbc44d314b3946e644eb7110e6cfa83604" gracePeriod=2
Sep 10 19:24:09.356857 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:09.356830    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-jrrrf"
Sep 10 19:24:09.419042 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:09.419006    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dff1fae8-bd43-4e76-8b18-425481205937-extensions-socket-volume\") pod \"dff1fae8-bd43-4e76-8b18-425481205937\" (UID: \"dff1fae8-bd43-4e76-8b18-425481205937\") "
Sep 10 19:24:09.419202 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:09.419122    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lpfgc\" (UniqueName: \"kubernetes.io/projected/dff1fae8-bd43-4e76-8b18-425481205937-kube-api-access-lpfgc\") pod \"dff1fae8-bd43-4e76-8b18-425481205937\" (UID: \"dff1fae8-bd43-4e76-8b18-425481205937\") "
Sep 10 19:24:09.419431 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:09.419409    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dff1fae8-bd43-4e76-8b18-425481205937-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "dff1fae8-bd43-4e76-8b18-425481205937" (UID: "dff1fae8-bd43-4e76-8b18-425481205937"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:24:09.421137 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:09.421111    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dff1fae8-bd43-4e76-8b18-425481205937-kube-api-access-lpfgc" (OuterVolumeSpecName: "kube-api-access-lpfgc") pod "dff1fae8-bd43-4e76-8b18-425481205937" (UID: "dff1fae8-bd43-4e76-8b18-425481205937"). InnerVolumeSpecName "kube-api-access-lpfgc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:24:09.519876 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:09.519840    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lpfgc\" (UniqueName: \"kubernetes.io/projected/dff1fae8-bd43-4e76-8b18-425481205937-kube-api-access-lpfgc\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:24:09.519876 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:09.519877    2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dff1fae8-bd43-4e76-8b18-425481205937-extensions-socket-volume\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:24:09.650240 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:09.650206    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:24:09.752166 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:09.752135    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:24:10.126120 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:10.126084    2564 generic.go:356] "Generic (PLEG): container finished" podID="dff1fae8-bd43-4e76-8b18-425481205937" containerID="20a8941783b921e303781ea0538290cbc44d314b3946e644eb7110e6cfa83604" exitCode=0
Sep 10 19:24:10.126527 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:10.126135    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-jrrrf"
Sep 10 19:24:10.126527 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:10.126184    2564 scope.go:117] "RemoveContainer" containerID="20a8941783b921e303781ea0538290cbc44d314b3946e644eb7110e6cfa83604"
Sep 10 19:24:10.128366 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:24:10.128334    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-67c5dcb48c-jrrrf\" is forbidden: User \"system:node:ip-10-0-129-230.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-230.ec2.internal' and this object" podUID="dff1fae8-bd43-4e76-8b18-425481205937" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-jrrrf"
Sep 10 19:24:10.136010 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:10.135985    2564 scope.go:117] "RemoveContainer" containerID="20a8941783b921e303781ea0538290cbc44d314b3946e644eb7110e6cfa83604"
Sep 10 19:24:10.136345 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:24:10.136323    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20a8941783b921e303781ea0538290cbc44d314b3946e644eb7110e6cfa83604\": container with ID starting with 20a8941783b921e303781ea0538290cbc44d314b3946e644eb7110e6cfa83604 not found: ID does not exist" containerID="20a8941783b921e303781ea0538290cbc44d314b3946e644eb7110e6cfa83604"
Sep 10 19:24:10.136387 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:10.136356    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20a8941783b921e303781ea0538290cbc44d314b3946e644eb7110e6cfa83604"} err="failed to get container status \"20a8941783b921e303781ea0538290cbc44d314b3946e644eb7110e6cfa83604\": rpc error: code = NotFound desc = could not find container \"20a8941783b921e303781ea0538290cbc44d314b3946e644eb7110e6cfa83604\": container with ID starting with 20a8941783b921e303781ea0538290cbc44d314b3946e644eb7110e6cfa83604 not found: ID does not exist"
Sep 10 19:24:10.137014 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:24:10.136989    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-67c5dcb48c-jrrrf\" is forbidden: User \"system:node:ip-10-0-129-230.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-230.ec2.internal' and this object" podUID="dff1fae8-bd43-4e76-8b18-425481205937" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-jrrrf"
Sep 10 19:24:10.949520 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:10.949489    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dff1fae8-bd43-4e76-8b18-425481205937" path="/var/lib/kubelet/pods/dff1fae8-bd43-4e76-8b18-425481205937/volumes"
Sep 10 19:24:12.828875 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:12.828838    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:24:12.931831 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:12.931796    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:24:17.301179 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:17.301138    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:24:17.403248 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:17.403207    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:24:18.234866 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:18.234832    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:24:18.289548 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:18.289513    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:24:19.121700 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:19.121665    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-qqb2z"
Sep 10 19:24:19.122135 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:19.122003    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-z5xdl"
Sep 10 19:24:19.122135 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:19.122059    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-psz7d"
Sep 10 19:24:19.206878 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:19.206835    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-z5xdl"]
Sep 10 19:24:19.207106 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:19.207078    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-z5xdl" podUID="57f17807-ae0d-4829-ac5d-00b338ca5c6d" containerName="manager" containerID="cri-o://e1348cfd8d535f61d77b0d5aaf0cd853e41307721c0172f90cf0821e8a098fcd" gracePeriod=10
Sep 10 19:24:19.456846 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:19.456817    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-z5xdl"
Sep 10 19:24:19.511085 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:19.511052    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dpfmp\" (UniqueName: \"kubernetes.io/projected/57f17807-ae0d-4829-ac5d-00b338ca5c6d-kube-api-access-dpfmp\") pod \"57f17807-ae0d-4829-ac5d-00b338ca5c6d\" (UID: \"57f17807-ae0d-4829-ac5d-00b338ca5c6d\") "
Sep 10 19:24:19.511246 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:19.511184    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/57f17807-ae0d-4829-ac5d-00b338ca5c6d-extensions-socket-volume\") pod \"57f17807-ae0d-4829-ac5d-00b338ca5c6d\" (UID: \"57f17807-ae0d-4829-ac5d-00b338ca5c6d\") "
Sep 10 19:24:19.511521 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:19.511494    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57f17807-ae0d-4829-ac5d-00b338ca5c6d-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "57f17807-ae0d-4829-ac5d-00b338ca5c6d" (UID: "57f17807-ae0d-4829-ac5d-00b338ca5c6d"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:24:19.513175 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:19.513151    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57f17807-ae0d-4829-ac5d-00b338ca5c6d-kube-api-access-dpfmp" (OuterVolumeSpecName: "kube-api-access-dpfmp") pod "57f17807-ae0d-4829-ac5d-00b338ca5c6d" (UID: "57f17807-ae0d-4829-ac5d-00b338ca5c6d"). InnerVolumeSpecName "kube-api-access-dpfmp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:24:19.612584 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:19.612547    2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/57f17807-ae0d-4829-ac5d-00b338ca5c6d-extensions-socket-volume\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:24:19.612584 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:19.612575    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dpfmp\" (UniqueName: \"kubernetes.io/projected/57f17807-ae0d-4829-ac5d-00b338ca5c6d-kube-api-access-dpfmp\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:24:19.666244 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:19.666170    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tjnq2"]
Sep 10 19:24:19.666529 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:19.666516    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57f17807-ae0d-4829-ac5d-00b338ca5c6d" containerName="manager"
Sep 10 19:24:19.666582 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:19.666530    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="57f17807-ae0d-4829-ac5d-00b338ca5c6d" containerName="manager"
Sep 10 19:24:19.666617 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:19.666591    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="57f17807-ae0d-4829-ac5d-00b338ca5c6d" containerName="manager"
Sep 10 19:24:19.669339 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:19.669318    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tjnq2"
Sep 10 19:24:19.701901 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:19.701868    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tjnq2"]
Sep 10 19:24:19.713022 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:19.712972    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d352d9a4-e806-48c7-9cba-98af37878cef-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-tjnq2\" (UID: \"d352d9a4-e806-48c7-9cba-98af37878cef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tjnq2"
Sep 10 19:24:19.713189 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:19.713024    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sx694\" (UniqueName: \"kubernetes.io/projected/d352d9a4-e806-48c7-9cba-98af37878cef-kube-api-access-sx694\") pod \"kuadrant-operator-controller-manager-5769f99d9f-tjnq2\" (UID: \"d352d9a4-e806-48c7-9cba-98af37878cef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tjnq2"
Sep 10 19:24:19.813994 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:19.813953    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d352d9a4-e806-48c7-9cba-98af37878cef-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-tjnq2\" (UID: \"d352d9a4-e806-48c7-9cba-98af37878cef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tjnq2"
Sep 10 19:24:19.813994 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:19.813998    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sx694\" (UniqueName: \"kubernetes.io/projected/d352d9a4-e806-48c7-9cba-98af37878cef-kube-api-access-sx694\") pod \"kuadrant-operator-controller-manager-5769f99d9f-tjnq2\" (UID: \"d352d9a4-e806-48c7-9cba-98af37878cef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tjnq2"
Sep 10 19:24:19.814400 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:19.814376    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d352d9a4-e806-48c7-9cba-98af37878cef-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-tjnq2\" (UID: \"d352d9a4-e806-48c7-9cba-98af37878cef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tjnq2"
Sep 10 19:24:19.825928 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:19.825897    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sx694\" (UniqueName: \"kubernetes.io/projected/d352d9a4-e806-48c7-9cba-98af37878cef-kube-api-access-sx694\") pod \"kuadrant-operator-controller-manager-5769f99d9f-tjnq2\" (UID: \"d352d9a4-e806-48c7-9cba-98af37878cef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tjnq2"
Sep 10 19:24:19.978913 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:19.978816    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tjnq2"
Sep 10 19:24:20.187779 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:20.187744    2564 generic.go:356] "Generic (PLEG): container finished" podID="57f17807-ae0d-4829-ac5d-00b338ca5c6d" containerID="e1348cfd8d535f61d77b0d5aaf0cd853e41307721c0172f90cf0821e8a098fcd" exitCode=0
Sep 10 19:24:20.188205 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:20.187801    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-z5xdl"
Sep 10 19:24:20.188205 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:20.187836    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-z5xdl" event={"ID":"57f17807-ae0d-4829-ac5d-00b338ca5c6d","Type":"ContainerDied","Data":"e1348cfd8d535f61d77b0d5aaf0cd853e41307721c0172f90cf0821e8a098fcd"}
Sep 10 19:24:20.188205 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:20.187892    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-z5xdl" event={"ID":"57f17807-ae0d-4829-ac5d-00b338ca5c6d","Type":"ContainerDied","Data":"70b57bc1770a4f10eeffedc20f93a7b59c23c4299f6a921df98c17b9e10efa8c"}
Sep 10 19:24:20.188205 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:20.187921    2564 scope.go:117] "RemoveContainer" containerID="e1348cfd8d535f61d77b0d5aaf0cd853e41307721c0172f90cf0821e8a098fcd"
Sep 10 19:24:20.197509 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:20.197493    2564 scope.go:117] "RemoveContainer" containerID="e1348cfd8d535f61d77b0d5aaf0cd853e41307721c0172f90cf0821e8a098fcd"
Sep 10 19:24:20.197829 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:24:20.197808    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1348cfd8d535f61d77b0d5aaf0cd853e41307721c0172f90cf0821e8a098fcd\": container with ID starting with e1348cfd8d535f61d77b0d5aaf0cd853e41307721c0172f90cf0821e8a098fcd not found: ID does not exist" containerID="e1348cfd8d535f61d77b0d5aaf0cd853e41307721c0172f90cf0821e8a098fcd"
Sep 10 19:24:20.197890 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:20.197838    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1348cfd8d535f61d77b0d5aaf0cd853e41307721c0172f90cf0821e8a098fcd"} err="failed to get container status \"e1348cfd8d535f61d77b0d5aaf0cd853e41307721c0172f90cf0821e8a098fcd\": rpc error: code = NotFound desc = could not find container \"e1348cfd8d535f61d77b0d5aaf0cd853e41307721c0172f90cf0821e8a098fcd\": container with ID starting with e1348cfd8d535f61d77b0d5aaf0cd853e41307721c0172f90cf0821e8a098fcd not found: ID does not exist"
Sep 10 19:24:20.217757 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:20.217720    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-z5xdl"]
Sep 10 19:24:20.231173 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:20.231110    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-z5xdl"]
Sep 10 19:24:20.259751 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:20.259726    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tjnq2"]
Sep 10 19:24:20.261666 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:24:20.261635    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd352d9a4_e806_48c7_9cba_98af37878cef.slice/crio-3335910fdc55bd202ae4170f298724840268a0938c4ee0eddcf6e6341d0f9f52 WatchSource:0}: Error finding container 3335910fdc55bd202ae4170f298724840268a0938c4ee0eddcf6e6341d0f9f52: Status 404 returned error can't find the container with id 3335910fdc55bd202ae4170f298724840268a0938c4ee0eddcf6e6341d0f9f52
Sep 10 19:24:20.949709 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:20.949668    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="57f17807-ae0d-4829-ac5d-00b338ca5c6d" path="/var/lib/kubelet/pods/57f17807-ae0d-4829-ac5d-00b338ca5c6d/volumes"
Sep 10 19:24:21.196614 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:21.196580    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tjnq2" event={"ID":"d352d9a4-e806-48c7-9cba-98af37878cef","Type":"ContainerStarted","Data":"f167490d9b410226d9257e3b4e80f6f6f925a900519ea6b70a7c75158aac2596"}
Sep 10 19:24:21.197016 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:21.196644    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tjnq2" event={"ID":"d352d9a4-e806-48c7-9cba-98af37878cef","Type":"ContainerStarted","Data":"3335910fdc55bd202ae4170f298724840268a0938c4ee0eddcf6e6341d0f9f52"}
Sep 10 19:24:21.197016 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:21.196706    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tjnq2"
Sep 10 19:24:21.219486 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:21.219380    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tjnq2" podStartSLOduration=2.219360638 podStartE2EDuration="2.219360638s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:24:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:24:21.216601344 +0000 UTC m=+516.817508620" watchObservedRunningTime="2026-09-10 19:24:21.219360638 +0000 UTC m=+516.820267879"
Sep 10 19:24:22.428772 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:22.428732    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:24:22.523738 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:22.523701    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:24:22.623444 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:22.623410    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:24:32.209769 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:32.209730    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tjnq2"
Sep 10 19:24:32.269597 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:32.269566    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-qqb2z"]
Sep 10 19:24:32.269850 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:32.269822    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-qqb2z" podUID="4d786de9-def7-49df-8390-d22d7f32bdfa" containerName="manager" containerID="cri-o://9a14673a16e5f5c010686635fc4649aa4f9d2c0a8c631624c839c7db82010be7" gracePeriod=10
Sep 10 19:24:32.519262 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:32.519237    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-qqb2z"
Sep 10 19:24:32.628136 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:32.628102    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4ksrw\" (UniqueName: \"kubernetes.io/projected/4d786de9-def7-49df-8390-d22d7f32bdfa-kube-api-access-4ksrw\") pod \"4d786de9-def7-49df-8390-d22d7f32bdfa\" (UID: \"4d786de9-def7-49df-8390-d22d7f32bdfa\") "
Sep 10 19:24:32.628295 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:32.628160    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4d786de9-def7-49df-8390-d22d7f32bdfa-extensions-socket-volume\") pod \"4d786de9-def7-49df-8390-d22d7f32bdfa\" (UID: \"4d786de9-def7-49df-8390-d22d7f32bdfa\") "
Sep 10 19:24:32.628529 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:32.628501    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d786de9-def7-49df-8390-d22d7f32bdfa-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "4d786de9-def7-49df-8390-d22d7f32bdfa" (UID: "4d786de9-def7-49df-8390-d22d7f32bdfa"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:24:32.630186 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:32.630152    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d786de9-def7-49df-8390-d22d7f32bdfa-kube-api-access-4ksrw" (OuterVolumeSpecName: "kube-api-access-4ksrw") pod "4d786de9-def7-49df-8390-d22d7f32bdfa" (UID: "4d786de9-def7-49df-8390-d22d7f32bdfa"). InnerVolumeSpecName "kube-api-access-4ksrw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:24:32.729487 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:32.729452    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4ksrw\" (UniqueName: \"kubernetes.io/projected/4d786de9-def7-49df-8390-d22d7f32bdfa-kube-api-access-4ksrw\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:24:32.729487 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:32.729480    2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4d786de9-def7-49df-8390-d22d7f32bdfa-extensions-socket-volume\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:24:33.262864 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:33.262833    2564 generic.go:356] "Generic (PLEG): container finished" podID="4d786de9-def7-49df-8390-d22d7f32bdfa" containerID="9a14673a16e5f5c010686635fc4649aa4f9d2c0a8c631624c839c7db82010be7" exitCode=0
Sep 10 19:24:33.263335 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:33.262879    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-qqb2z"
Sep 10 19:24:33.263335 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:33.262888    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-qqb2z" event={"ID":"4d786de9-def7-49df-8390-d22d7f32bdfa","Type":"ContainerDied","Data":"9a14673a16e5f5c010686635fc4649aa4f9d2c0a8c631624c839c7db82010be7"}
Sep 10 19:24:33.263335 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:33.262928    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-qqb2z" event={"ID":"4d786de9-def7-49df-8390-d22d7f32bdfa","Type":"ContainerDied","Data":"af25a6a42fdeb29d18764b884d6673b142a09ab2747c99caec14418fa06c9f23"}
Sep 10 19:24:33.263335 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:33.262949    2564 scope.go:117] "RemoveContainer" containerID="9a14673a16e5f5c010686635fc4649aa4f9d2c0a8c631624c839c7db82010be7"
Sep 10 19:24:33.271575 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:33.271386    2564 scope.go:117] "RemoveContainer" containerID="9a14673a16e5f5c010686635fc4649aa4f9d2c0a8c631624c839c7db82010be7"
Sep 10 19:24:33.271706 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:24:33.271683    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a14673a16e5f5c010686635fc4649aa4f9d2c0a8c631624c839c7db82010be7\": container with ID starting with 9a14673a16e5f5c010686635fc4649aa4f9d2c0a8c631624c839c7db82010be7 not found: ID does not exist" containerID="9a14673a16e5f5c010686635fc4649aa4f9d2c0a8c631624c839c7db82010be7"
Sep 10 19:24:33.271752 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:33.271714    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a14673a16e5f5c010686635fc4649aa4f9d2c0a8c631624c839c7db82010be7"} err="failed to get container status \"9a14673a16e5f5c010686635fc4649aa4f9d2c0a8c631624c839c7db82010be7\": rpc error: code = NotFound desc = could not find container \"9a14673a16e5f5c010686635fc4649aa4f9d2c0a8c631624c839c7db82010be7\": container with ID starting with 9a14673a16e5f5c010686635fc4649aa4f9d2c0a8c631624c839c7db82010be7 not found: ID does not exist"
Sep 10 19:24:33.287561 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:33.287529    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-qqb2z"]
Sep 10 19:24:33.293964 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:33.293942    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-qqb2z"]
Sep 10 19:24:34.948607 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:34.948570    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d786de9-def7-49df-8390-d22d7f32bdfa" path="/var/lib/kubelet/pods/4d786de9-def7-49df-8390-d22d7f32bdfa/volumes"
Sep 10 19:24:35.539262 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:35.539225    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8r27k"]
Sep 10 19:24:35.539574 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:35.539560    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d786de9-def7-49df-8390-d22d7f32bdfa" containerName="manager"
Sep 10 19:24:35.539574 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:35.539574    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d786de9-def7-49df-8390-d22d7f32bdfa" containerName="manager"
Sep 10 19:24:35.539717 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:35.539660    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d786de9-def7-49df-8390-d22d7f32bdfa" containerName="manager"
Sep 10 19:24:35.543636 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:35.543605    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8r27k"
Sep 10 19:24:35.546072 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:35.546052    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-v62pl\""
Sep 10 19:24:35.554784 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:35.554757    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8r27k"]
Sep 10 19:24:35.560759 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:35.560735    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:24:35.654663 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:35.654602    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c5fe5daa-4a4f-45c8-9aaf-e83382f78471-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8r27k\" (UID: \"c5fe5daa-4a4f-45c8-9aaf-e83382f78471\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8r27k"
Sep 10 19:24:35.654825 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:35.654678    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5fe5daa-4a4f-45c8-9aaf-e83382f78471-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8r27k\" (UID: \"c5fe5daa-4a4f-45c8-9aaf-e83382f78471\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8r27k"
Sep 10 19:24:35.654825 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:35.654710    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c5fe5daa-4a4f-45c8-9aaf-e83382f78471-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8r27k\" (UID: \"c5fe5daa-4a4f-45c8-9aaf-e83382f78471\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8r27k"
Sep 10 19:24:35.654825 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:35.654729    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c5fe5daa-4a4f-45c8-9aaf-e83382f78471-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8r27k\" (UID: \"c5fe5daa-4a4f-45c8-9aaf-e83382f78471\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8r27k"
Sep 10 19:24:35.654825 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:35.654760    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c5fe5daa-4a4f-45c8-9aaf-e83382f78471-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8r27k\" (UID: \"c5fe5daa-4a4f-45c8-9aaf-e83382f78471\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8r27k"
Sep 10 19:24:35.654825 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:35.654806    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c5fe5daa-4a4f-45c8-9aaf-e83382f78471-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8r27k\" (UID: \"c5fe5daa-4a4f-45c8-9aaf-e83382f78471\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8r27k"
Sep 10 19:24:35.654825 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:35.654822    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5fe5daa-4a4f-45c8-9aaf-e83382f78471-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8r27k\" (UID: \"c5fe5daa-4a4f-45c8-9aaf-e83382f78471\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8r27k"
Sep 10 19:24:35.655159 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:35.654878    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c5fe5daa-4a4f-45c8-9aaf-e83382f78471-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8r27k\" (UID: \"c5fe5daa-4a4f-45c8-9aaf-e83382f78471\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8r27k"
Sep 10 19:24:35.655159 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:35.654893    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rbjn\" (UniqueName: \"kubernetes.io/projected/c5fe5daa-4a4f-45c8-9aaf-e83382f78471-kube-api-access-8rbjn\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8r27k\" (UID: \"c5fe5daa-4a4f-45c8-9aaf-e83382f78471\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8r27k"
Sep 10 19:24:35.756206 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:35.756154    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c5fe5daa-4a4f-45c8-9aaf-e83382f78471-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8r27k\" (UID: \"c5fe5daa-4a4f-45c8-9aaf-e83382f78471\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8r27k"
Sep 10 19:24:35.756419 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:35.756216    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c5fe5daa-4a4f-45c8-9aaf-e83382f78471-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8r27k\" (UID: \"c5fe5daa-4a4f-45c8-9aaf-e83382f78471\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8r27k"
Sep 10 19:24:35.756419 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:35.756261    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c5fe5daa-4a4f-45c8-9aaf-e83382f78471-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8r27k\" (UID: \"c5fe5daa-4a4f-45c8-9aaf-e83382f78471\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8r27k"
Sep 10 19:24:35.756419 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:35.756299    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c5fe5daa-4a4f-45c8-9aaf-e83382f78471-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8r27k\" (UID: \"c5fe5daa-4a4f-45c8-9aaf-e83382f78471\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8r27k"
Sep 10 19:24:35.756419 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:35.756327    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5fe5daa-4a4f-45c8-9aaf-e83382f78471-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8r27k\" (UID: \"c5fe5daa-4a4f-45c8-9aaf-e83382f78471\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8r27k"
Sep 10 19:24:35.756419 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:35.756385    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c5fe5daa-4a4f-45c8-9aaf-e83382f78471-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8r27k\" (UID: \"c5fe5daa-4a4f-45c8-9aaf-e83382f78471\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8r27k"
Sep 10 19:24:35.756419 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:35.756401    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rbjn\" (UniqueName: \"kubernetes.io/projected/c5fe5daa-4a4f-45c8-9aaf-e83382f78471-kube-api-access-8rbjn\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8r27k\" (UID: \"c5fe5daa-4a4f-45c8-9aaf-e83382f78471\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8r27k"
Sep 10 19:24:35.756812 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:35.756465    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c5fe5daa-4a4f-45c8-9aaf-e83382f78471-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8r27k\" (UID: \"c5fe5daa-4a4f-45c8-9aaf-e83382f78471\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8r27k"
Sep 10 19:24:35.756812 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:35.756514    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5fe5daa-4a4f-45c8-9aaf-e83382f78471-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8r27k\" (UID: \"c5fe5daa-4a4f-45c8-9aaf-e83382f78471\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8r27k"
Sep 10 19:24:35.756812 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:35.756671    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c5fe5daa-4a4f-45c8-9aaf-e83382f78471-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8r27k\" (UID: \"c5fe5daa-4a4f-45c8-9aaf-e83382f78471\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8r27k"
Sep 10 19:24:35.756812 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:35.756722    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c5fe5daa-4a4f-45c8-9aaf-e83382f78471-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8r27k\" (UID: \"c5fe5daa-4a4f-45c8-9aaf-e83382f78471\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8r27k"
Sep 10 19:24:35.756812 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:35.756761    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5fe5daa-4a4f-45c8-9aaf-e83382f78471-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8r27k\" (UID: \"c5fe5daa-4a4f-45c8-9aaf-e83382f78471\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8r27k"
Sep 10 19:24:35.757397 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:35.757374    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c5fe5daa-4a4f-45c8-9aaf-e83382f78471-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8r27k\" (UID: \"c5fe5daa-4a4f-45c8-9aaf-e83382f78471\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8r27k"
Sep 10 19:24:35.757477 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:35.757458    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5fe5daa-4a4f-45c8-9aaf-e83382f78471-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8r27k\" (UID: \"c5fe5daa-4a4f-45c8-9aaf-e83382f78471\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8r27k"
Sep 10 19:24:35.758708 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:35.758689    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c5fe5daa-4a4f-45c8-9aaf-e83382f78471-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8r27k\" (UID: \"c5fe5daa-4a4f-45c8-9aaf-e83382f78471\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8r27k"
Sep 10 19:24:35.758909 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:35.758889    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c5fe5daa-4a4f-45c8-9aaf-e83382f78471-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8r27k\" (UID: \"c5fe5daa-4a4f-45c8-9aaf-e83382f78471\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8r27k"
Sep 10 19:24:35.768222 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:35.768194    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c5fe5daa-4a4f-45c8-9aaf-e83382f78471-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8r27k\" (UID: \"c5fe5daa-4a4f-45c8-9aaf-e83382f78471\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8r27k"
Sep 10 19:24:35.768341 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:35.768255    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rbjn\" (UniqueName: \"kubernetes.io/projected/c5fe5daa-4a4f-45c8-9aaf-e83382f78471-kube-api-access-8rbjn\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8r27k\" (UID: \"c5fe5daa-4a4f-45c8-9aaf-e83382f78471\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8r27k"
Sep 10 19:24:35.783120 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:35.783085    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:24:35.854375 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:35.854290    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8r27k"
Sep 10 19:24:36.129599 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:36.129506    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8r27k"]
Sep 10 19:24:36.132657 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:24:36.132602    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5fe5daa_4a4f_45c8_9aaf_e83382f78471.slice/crio-0f7eeb991b0a2c579a8b6c5262cc85f2f1b565018b5df2432a74d7deec3a7d48 WatchSource:0}: Error finding container 0f7eeb991b0a2c579a8b6c5262cc85f2f1b565018b5df2432a74d7deec3a7d48: Status 404 returned error can't find the container with id 0f7eeb991b0a2c579a8b6c5262cc85f2f1b565018b5df2432a74d7deec3a7d48
Sep 10 19:24:36.134753 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:36.134720    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:24:36.134831 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:36.134785    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:24:36.134831 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:36.134817    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:24:36.282613 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:36.282573    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8r27k" event={"ID":"c5fe5daa-4a4f-45c8-9aaf-e83382f78471","Type":"ContainerStarted","Data":"7f1f9705519cb7f6791821059f76adaa75d5dec368bda5689478f5a7aeaf4bff"}
Sep 10 19:24:36.282790 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:36.282665    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8r27k" event={"ID":"c5fe5daa-4a4f-45c8-9aaf-e83382f78471","Type":"ContainerStarted","Data":"0f7eeb991b0a2c579a8b6c5262cc85f2f1b565018b5df2432a74d7deec3a7d48"}
Sep 10 19:24:36.303978 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:36.303925    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8r27k" podStartSLOduration=1.303907855 podStartE2EDuration="1.303907855s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:24:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:24:36.30167958 +0000 UTC m=+531.902586822" watchObservedRunningTime="2026-09-10 19:24:36.303907855 +0000 UTC m=+531.904815095"
Sep 10 19:24:36.855154 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:36.855117    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8r27k"
Sep 10 19:24:37.859683 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:37.859650    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8r27k"
Sep 10 19:24:38.292089 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:38.292055    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8r27k"
Sep 10 19:24:38.293104 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:38.293083    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8r27k"
Sep 10 19:24:51.267554 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:51.267505    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:24:51.562307 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:51.562223    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:24:51.669199 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:51.669160    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:24:51.771288 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:51.771226    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:24:52.465054 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:52.465007    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:24:52.564664 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:24:52.564604    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:25:08.842766 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:25:08.842725    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:25:08.945441 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:25:08.945404    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:25:09.316494 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:25:09.316456    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:25:21.370483 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:25:21.370444    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:25:21.649039 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:25:21.648956    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:25:28.639319 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:25:28.639274    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:25:28.678843 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:25:28.678810    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-56887f8cd7-6zmmd"]
Sep 10 19:25:28.682437 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:25:28.682420    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56887f8cd7-6zmmd"
Sep 10 19:25:28.685109 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:25:28.685084    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 10 19:25:28.685351 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:25:28.685333    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 10 19:25:28.685404 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:25:28.685364    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 10 19:25:28.685926 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:25:28.685907    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-sn4np\""
Sep 10 19:25:28.694320 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:25:28.694294    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-56887f8cd7-6zmmd"]
Sep 10 19:25:28.717386 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:25:28.717339    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:25:28.772752 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:25:28.772715    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:25:28.818049 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:25:28.818011    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47ac938c-077c-46c6-896e-c48668f54cf2-metrics-certs\") pod \"maas-controller-56887f8cd7-6zmmd\" (UID: \"47ac938c-077c-46c6-896e-c48668f54cf2\") " pod="opendatahub/maas-controller-56887f8cd7-6zmmd"
Sep 10 19:25:28.818219 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:25:28.818065    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/47ac938c-077c-46c6-896e-c48668f54cf2-webhook-cert\") pod \"maas-controller-56887f8cd7-6zmmd\" (UID: \"47ac938c-077c-46c6-896e-c48668f54cf2\") " pod="opendatahub/maas-controller-56887f8cd7-6zmmd"
Sep 10 19:25:28.818219 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:25:28.818088    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggppn\" (UniqueName: \"kubernetes.io/projected/47ac938c-077c-46c6-896e-c48668f54cf2-kube-api-access-ggppn\") pod \"maas-controller-56887f8cd7-6zmmd\" (UID: \"47ac938c-077c-46c6-896e-c48668f54cf2\") " pod="opendatahub/maas-controller-56887f8cd7-6zmmd"
Sep 10 19:25:28.863824 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:25:28.863794    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:25:28.919567 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:25:28.919469    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/47ac938c-077c-46c6-896e-c48668f54cf2-webhook-cert\") pod \"maas-controller-56887f8cd7-6zmmd\" (UID: \"47ac938c-077c-46c6-896e-c48668f54cf2\") " pod="opendatahub/maas-controller-56887f8cd7-6zmmd"
Sep 10 19:25:28.919567 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:25:28.919511    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggppn\" (UniqueName: \"kubernetes.io/projected/47ac938c-077c-46c6-896e-c48668f54cf2-kube-api-access-ggppn\") pod \"maas-controller-56887f8cd7-6zmmd\" (UID: \"47ac938c-077c-46c6-896e-c48668f54cf2\") " pod="opendatahub/maas-controller-56887f8cd7-6zmmd"
Sep 10 19:25:28.919815 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:25:28.919595    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47ac938c-077c-46c6-896e-c48668f54cf2-metrics-certs\") pod \"maas-controller-56887f8cd7-6zmmd\" (UID: \"47ac938c-077c-46c6-896e-c48668f54cf2\") " pod="opendatahub/maas-controller-56887f8cd7-6zmmd"
Sep 10 19:25:28.921870 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:25:28.921839    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47ac938c-077c-46c6-896e-c48668f54cf2-metrics-certs\") pod \"maas-controller-56887f8cd7-6zmmd\" (UID: \"47ac938c-077c-46c6-896e-c48668f54cf2\") " pod="opendatahub/maas-controller-56887f8cd7-6zmmd"
Sep 10 19:25:28.921991 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:25:28.921884    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/47ac938c-077c-46c6-896e-c48668f54cf2-webhook-cert\") pod \"maas-controller-56887f8cd7-6zmmd\" (UID: \"47ac938c-077c-46c6-896e-c48668f54cf2\") " pod="opendatahub/maas-controller-56887f8cd7-6zmmd"
Sep 10 19:25:28.928497 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:25:28.928467    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:25:28.930731 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:25:28.930708    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggppn\" (UniqueName: \"kubernetes.io/projected/47ac938c-077c-46c6-896e-c48668f54cf2-kube-api-access-ggppn\") pod \"maas-controller-56887f8cd7-6zmmd\" (UID: \"47ac938c-077c-46c6-896e-c48668f54cf2\") " pod="opendatahub/maas-controller-56887f8cd7-6zmmd"
Sep 10 19:25:28.993885 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:25:28.993847    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56887f8cd7-6zmmd"
Sep 10 19:25:29.265521 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:25:29.265487    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-56887f8cd7-6zmmd"]
Sep 10 19:25:29.268763 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:25:29.268735    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47ac938c_077c_46c6_896e_c48668f54cf2.slice/crio-44934df45d24241dd9f102210661e56d57c63f1e20a7d2a3aebd4c370c6bbf1b WatchSource:0}: Error finding container 44934df45d24241dd9f102210661e56d57c63f1e20a7d2a3aebd4c370c6bbf1b: Status 404 returned error can't find the container with id 44934df45d24241dd9f102210661e56d57c63f1e20a7d2a3aebd4c370c6bbf1b
Sep 10 19:25:29.532580 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:25:29.532492    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56887f8cd7-6zmmd" event={"ID":"47ac938c-077c-46c6-896e-c48668f54cf2","Type":"ContainerStarted","Data":"44934df45d24241dd9f102210661e56d57c63f1e20a7d2a3aebd4c370c6bbf1b"}
Sep 10 19:25:31.834378 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:25:31.834334    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:25:31.934932 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:25:31.934840    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:25:32.550953 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:25:32.550922    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56887f8cd7-6zmmd" event={"ID":"47ac938c-077c-46c6-896e-c48668f54cf2","Type":"ContainerStarted","Data":"8302400f50ceb538efe862132357fbc1193e22c29faf972545ef9f82a45866d4"}
Sep 10 19:25:32.551133 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:25:32.551038    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-56887f8cd7-6zmmd"
Sep 10 19:25:32.570152 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:25:32.570098    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-56887f8cd7-6zmmd" podStartSLOduration=2.225342841 podStartE2EDuration="4.570071762s" totalImagesPullingTime="2.344728921s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:25:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:25:32.570009001 +0000 UTC m=+588.170916245" watchObservedRunningTime="2026-09-10 19:25:32.570071762 +0000 UTC m=+588.170979002"
Sep 10 19:25:43.159439 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:25:43.159403    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-56887f8cd7-6zmmd"]
Sep 10 19:25:43.159975 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:25:43.159600    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-56887f8cd7-6zmmd" podUID="47ac938c-077c-46c6-896e-c48668f54cf2" containerName="manager" containerID="cri-o://8302400f50ceb538efe862132357fbc1193e22c29faf972545ef9f82a45866d4" gracePeriod=30
Sep 10 19:25:43.161538 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:25:43.161517    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-56887f8cd7-6zmmd"
Sep 10 19:25:43.407537 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:25:43.407511    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56887f8cd7-6zmmd"
Sep 10 19:25:43.460051 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:25:43.459963    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/47ac938c-077c-46c6-896e-c48668f54cf2-webhook-cert\") pod \"47ac938c-077c-46c6-896e-c48668f54cf2\" (UID: \"47ac938c-077c-46c6-896e-c48668f54cf2\") "
Sep 10 19:25:43.460051 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:25:43.460001    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ggppn\" (UniqueName: \"kubernetes.io/projected/47ac938c-077c-46c6-896e-c48668f54cf2-kube-api-access-ggppn\") pod \"47ac938c-077c-46c6-896e-c48668f54cf2\" (UID: \"47ac938c-077c-46c6-896e-c48668f54cf2\") "
Sep 10 19:25:43.460051 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:25:43.460045    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47ac938c-077c-46c6-896e-c48668f54cf2-metrics-certs\") pod \"47ac938c-077c-46c6-896e-c48668f54cf2\" (UID: \"47ac938c-077c-46c6-896e-c48668f54cf2\") "
Sep 10 19:25:43.462296 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:25:43.462263    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/47ac938c-077c-46c6-896e-c48668f54cf2-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "47ac938c-077c-46c6-896e-c48668f54cf2" (UID: "47ac938c-077c-46c6-896e-c48668f54cf2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:25:43.462409 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:25:43.462299    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/47ac938c-077c-46c6-896e-c48668f54cf2-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "47ac938c-077c-46c6-896e-c48668f54cf2" (UID: "47ac938c-077c-46c6-896e-c48668f54cf2"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:25:43.462409 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:25:43.462321    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47ac938c-077c-46c6-896e-c48668f54cf2-kube-api-access-ggppn" (OuterVolumeSpecName: "kube-api-access-ggppn") pod "47ac938c-077c-46c6-896e-c48668f54cf2" (UID: "47ac938c-077c-46c6-896e-c48668f54cf2"). InnerVolumeSpecName "kube-api-access-ggppn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:25:43.560983 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:25:43.560949    2564 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47ac938c-077c-46c6-896e-c48668f54cf2-metrics-certs\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:25:43.560983 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:25:43.560975    2564 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/47ac938c-077c-46c6-896e-c48668f54cf2-webhook-cert\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:25:43.560983 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:25:43.560984    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ggppn\" (UniqueName: \"kubernetes.io/projected/47ac938c-077c-46c6-896e-c48668f54cf2-kube-api-access-ggppn\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:25:43.605116 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:25:43.605081    2564 generic.go:356] "Generic (PLEG): container finished" podID="47ac938c-077c-46c6-896e-c48668f54cf2" containerID="8302400f50ceb538efe862132357fbc1193e22c29faf972545ef9f82a45866d4" exitCode=0
Sep 10 19:25:43.605279 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:25:43.605129    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56887f8cd7-6zmmd"
Sep 10 19:25:43.605279 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:25:43.605165    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56887f8cd7-6zmmd" event={"ID":"47ac938c-077c-46c6-896e-c48668f54cf2","Type":"ContainerDied","Data":"8302400f50ceb538efe862132357fbc1193e22c29faf972545ef9f82a45866d4"}
Sep 10 19:25:43.605279 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:25:43.605218    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56887f8cd7-6zmmd" event={"ID":"47ac938c-077c-46c6-896e-c48668f54cf2","Type":"ContainerDied","Data":"44934df45d24241dd9f102210661e56d57c63f1e20a7d2a3aebd4c370c6bbf1b"}
Sep 10 19:25:43.605279 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:25:43.605241    2564 scope.go:117] "RemoveContainer" containerID="8302400f50ceb538efe862132357fbc1193e22c29faf972545ef9f82a45866d4"
Sep 10 19:25:43.613892 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:25:43.613868    2564 scope.go:117] "RemoveContainer" containerID="8302400f50ceb538efe862132357fbc1193e22c29faf972545ef9f82a45866d4"
Sep 10 19:25:43.614169 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:25:43.614147    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8302400f50ceb538efe862132357fbc1193e22c29faf972545ef9f82a45866d4\": container with ID starting with 8302400f50ceb538efe862132357fbc1193e22c29faf972545ef9f82a45866d4 not found: ID does not exist" containerID="8302400f50ceb538efe862132357fbc1193e22c29faf972545ef9f82a45866d4"
Sep 10 19:25:43.614236 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:25:43.614183    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8302400f50ceb538efe862132357fbc1193e22c29faf972545ef9f82a45866d4"} err="failed to get container status \"8302400f50ceb538efe862132357fbc1193e22c29faf972545ef9f82a45866d4\": rpc error: code = NotFound desc = could not find container \"8302400f50ceb538efe862132357fbc1193e22c29faf972545ef9f82a45866d4\": container with ID starting with 8302400f50ceb538efe862132357fbc1193e22c29faf972545ef9f82a45866d4 not found: ID does not exist"
Sep 10 19:25:43.629937 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:25:43.629902    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-56887f8cd7-6zmmd"]
Sep 10 19:25:43.636015 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:25:43.635987    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-56887f8cd7-6zmmd"]
Sep 10 19:25:44.950118 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:25:44.950082    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="47ac938c-077c-46c6-896e-c48668f54cf2" path="/var/lib/kubelet/pods/47ac938c-077c-46c6-896e-c48668f54cf2/volumes"
Sep 10 19:26:03.379349 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:03.379242    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:26:03.474773 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:03.474733    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:26:04.841739 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:04.841701    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:26:05.666367 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:05.666329    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-ll4jr"]
Sep 10 19:26:05.666785 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:05.666761    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47ac938c-077c-46c6-896e-c48668f54cf2" containerName="manager"
Sep 10 19:26:05.666785 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:05.666782    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="47ac938c-077c-46c6-896e-c48668f54cf2" containerName="manager"
Sep 10 19:26:05.667011 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:05.666850    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="47ac938c-077c-46c6-896e-c48668f54cf2" containerName="manager"
Sep 10 19:26:05.682573 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:05.682518    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-ll4jr"
Sep 10 19:26:05.685165 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:05.685136    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 10 19:26:05.685297 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:05.685234    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-ll4jr"]
Sep 10 19:26:05.685664 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:05.685616    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-nnfkc\""
Sep 10 19:26:05.721474 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:05.721443    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-plw9s"]
Sep 10 19:26:05.724763 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:05.724743    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-plw9s"
Sep 10 19:26:05.736146 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:05.736120    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-plw9s"]
Sep 10 19:26:05.760028 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:05.759996    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f80fdb52-770e-4a98-904f-556d9a3d9b28-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-ll4jr\" (UID: \"f80fdb52-770e-4a98-904f-556d9a3d9b28\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-ll4jr"
Sep 10 19:26:05.760195 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:05.760057    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79v94\" (UniqueName: \"kubernetes.io/projected/f80fdb52-770e-4a98-904f-556d9a3d9b28-kube-api-access-79v94\") pod \"payload-processing-5b54b7fbb7-ll4jr\" (UID: \"f80fdb52-770e-4a98-904f-556d9a3d9b28\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-ll4jr"
Sep 10 19:26:05.861332 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:05.861300    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6428c980-c9b3-46f4-b29d-d11aa11fca18-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-plw9s\" (UID: \"6428c980-c9b3-46f4-b29d-d11aa11fca18\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-plw9s"
Sep 10 19:26:05.861332 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:05.861341    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f80fdb52-770e-4a98-904f-556d9a3d9b28-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-ll4jr\" (UID: \"f80fdb52-770e-4a98-904f-556d9a3d9b28\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-ll4jr"
Sep 10 19:26:05.861928 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:05.861406    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7spzj\" (UniqueName: \"kubernetes.io/projected/6428c980-c9b3-46f4-b29d-d11aa11fca18-kube-api-access-7spzj\") pod \"payload-pre-processing-59fbc47bf8-plw9s\" (UID: \"6428c980-c9b3-46f4-b29d-d11aa11fca18\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-plw9s"
Sep 10 19:26:05.861928 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:05.861426    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79v94\" (UniqueName: \"kubernetes.io/projected/f80fdb52-770e-4a98-904f-556d9a3d9b28-kube-api-access-79v94\") pod \"payload-processing-5b54b7fbb7-ll4jr\" (UID: \"f80fdb52-770e-4a98-904f-556d9a3d9b28\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-ll4jr"
Sep 10 19:26:05.862231 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:05.862206    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f80fdb52-770e-4a98-904f-556d9a3d9b28-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-ll4jr\" (UID: \"f80fdb52-770e-4a98-904f-556d9a3d9b28\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-ll4jr"
Sep 10 19:26:05.878039 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:05.878003    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-79v94\" (UniqueName: \"kubernetes.io/projected/f80fdb52-770e-4a98-904f-556d9a3d9b28-kube-api-access-79v94\") pod \"payload-processing-5b54b7fbb7-ll4jr\" (UID: \"f80fdb52-770e-4a98-904f-556d9a3d9b28\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-ll4jr"
Sep 10 19:26:05.962418 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:05.962318    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6428c980-c9b3-46f4-b29d-d11aa11fca18-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-plw9s\" (UID: \"6428c980-c9b3-46f4-b29d-d11aa11fca18\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-plw9s"
Sep 10 19:26:05.962418 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:05.962382    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7spzj\" (UniqueName: \"kubernetes.io/projected/6428c980-c9b3-46f4-b29d-d11aa11fca18-kube-api-access-7spzj\") pod \"payload-pre-processing-59fbc47bf8-plw9s\" (UID: \"6428c980-c9b3-46f4-b29d-d11aa11fca18\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-plw9s"
Sep 10 19:26:05.963127 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:05.963105    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6428c980-c9b3-46f4-b29d-d11aa11fca18-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-plw9s\" (UID: \"6428c980-c9b3-46f4-b29d-d11aa11fca18\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-plw9s"
Sep 10 19:26:05.972247 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:05.972209    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7spzj\" (UniqueName: \"kubernetes.io/projected/6428c980-c9b3-46f4-b29d-d11aa11fca18-kube-api-access-7spzj\") pod \"payload-pre-processing-59fbc47bf8-plw9s\" (UID: \"6428c980-c9b3-46f4-b29d-d11aa11fca18\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-plw9s"
Sep 10 19:26:05.995069 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:05.995034    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-ll4jr"
Sep 10 19:26:06.003509 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:06.003461    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:26:06.036194 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:06.036163    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-plw9s"
Sep 10 19:26:06.272652 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:06.270362    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-ll4jr"]
Sep 10 19:26:06.289646 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:26:06.288224    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf80fdb52_770e_4a98_904f_556d9a3d9b28.slice/crio-648571bc60aaeb32f6a247b8a381ad6ae79585372c63807ebbd023b4121fb060 WatchSource:0}: Error finding container 648571bc60aaeb32f6a247b8a381ad6ae79585372c63807ebbd023b4121fb060: Status 404 returned error can't find the container with id 648571bc60aaeb32f6a247b8a381ad6ae79585372c63807ebbd023b4121fb060
Sep 10 19:26:06.320767 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:06.320734    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-plw9s"]
Sep 10 19:26:06.324145 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:26:06.324112    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6428c980_c9b3_46f4_b29d_d11aa11fca18.slice/crio-94531f230bb33720e8c9518be3e6e8c2402c1851e38364f2a034027950b2add8 WatchSource:0}: Error finding container 94531f230bb33720e8c9518be3e6e8c2402c1851e38364f2a034027950b2add8: Status 404 returned error can't find the container with id 94531f230bb33720e8c9518be3e6e8c2402c1851e38364f2a034027950b2add8
Sep 10 19:26:06.713304 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:06.713206    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-plw9s" event={"ID":"6428c980-c9b3-46f4-b29d-d11aa11fca18","Type":"ContainerStarted","Data":"94531f230bb33720e8c9518be3e6e8c2402c1851e38364f2a034027950b2add8"}
Sep 10 19:26:06.714400 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:06.714379    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-ll4jr" event={"ID":"f80fdb52-770e-4a98-904f-556d9a3d9b28","Type":"ContainerStarted","Data":"648571bc60aaeb32f6a247b8a381ad6ae79585372c63807ebbd023b4121fb060"}
Sep 10 19:26:08.729571 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:08.729524    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-plw9s" event={"ID":"6428c980-c9b3-46f4-b29d-d11aa11fca18","Type":"ContainerStarted","Data":"aef7a2e625289bd4ed2d16c5df7cfb00d68e5412349c4657987271890cfe67bb"}
Sep 10 19:26:08.730035 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:08.729638    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-plw9s"
Sep 10 19:26:08.731424 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:08.731402    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-ll4jr" event={"ID":"f80fdb52-770e-4a98-904f-556d9a3d9b28","Type":"ContainerStarted","Data":"e8d82a0236faed70a45691cdcaab9bf724ca562baa4c20bdba93a5bf496a135d"}
Sep 10 19:26:08.731654 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:08.731637    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-ll4jr"
Sep 10 19:26:08.748573 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:08.748525    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-plw9s" podStartSLOduration=1.4852641979999999 podStartE2EDuration="3.748510163s" totalImagesPullingTime="2.263245965s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:26:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:26:08.746101231 +0000 UTC m=+624.347008472" watchObservedRunningTime="2026-09-10 19:26:08.748510163 +0000 UTC m=+624.349417404"
Sep 10 19:26:10.878976 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:10.878912    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:26:14.761363 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:14.761331    2564 generic.go:356] "Generic (PLEG): container finished" podID="c5fe5daa-4a4f-45c8-9aaf-e83382f78471" containerID="7f1f9705519cb7f6791821059f76adaa75d5dec368bda5689478f5a7aeaf4bff" exitCode=0
Sep 10 19:26:14.761796 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:14.761409    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8r27k" event={"ID":"c5fe5daa-4a4f-45c8-9aaf-e83382f78471","Type":"ContainerDied","Data":"7f1f9705519cb7f6791821059f76adaa75d5dec368bda5689478f5a7aeaf4bff"}
Sep 10 19:26:14.761909 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:14.761894    2564 scope.go:117] "RemoveContainer" containerID="7f1f9705519cb7f6791821059f76adaa75d5dec368bda5689478f5a7aeaf4bff"
Sep 10 19:26:14.762932 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:14.762898    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:26:14.762996 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:14.762985    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:26:14.763074 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:14.763021    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:26:14.788042 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:14.787992    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-ll4jr" podStartSLOduration=7.478444147 podStartE2EDuration="9.787977751s" totalImagesPullingTime="2.309533604s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:26:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:26:08.766842134 +0000 UTC m=+624.367749375" watchObservedRunningTime="2026-09-10 19:26:14.787977751 +0000 UTC m=+630.388884990"
Sep 10 19:26:15.290272 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:15.290233    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:26:15.389414 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:15.389345    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:26:15.792675 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:15.792606    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8r27k" event={"ID":"c5fe5daa-4a4f-45c8-9aaf-e83382f78471","Type":"ContainerStarted","Data":"b2569e48ba905362ba57eca283ae848a8ce84152d0db47583b27079569e7da4b"}
Sep 10 19:26:15.855236 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:15.855200    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8r27k"
Sep 10 19:26:15.855489 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:15.855465    2564 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-8r27k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Sep 10 19:26:15.855534 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:15.855518    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8r27k" podUID="c5fe5daa-4a4f-45c8-9aaf-e83382f78471" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Sep 10 19:26:16.855391 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:16.855354    2564 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-8r27k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Sep 10 19:26:16.855809 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:16.855410    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8r27k" podUID="c5fe5daa-4a4f-45c8-9aaf-e83382f78471" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Sep 10 19:26:17.855119 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:17.855079    2564 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-8r27k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Sep 10 19:26:17.855296 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:17.855143    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8r27k" podUID="c5fe5daa-4a4f-45c8-9aaf-e83382f78471" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Sep 10 19:26:18.855260 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:18.855222    2564 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-8r27k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Sep 10 19:26:18.855704 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:18.855277    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8r27k" podUID="c5fe5daa-4a4f-45c8-9aaf-e83382f78471" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Sep 10 19:26:19.739449 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:19.739414    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-ll4jr"
Sep 10 19:26:19.739651 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:19.739492    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-plw9s"
Sep 10 19:26:19.855612 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:19.855570    2564 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-8r27k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Sep 10 19:26:19.856007 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:19.855651    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8r27k" podUID="c5fe5daa-4a4f-45c8-9aaf-e83382f78471" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Sep 10 19:26:20.858577 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:20.858549    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8r27k"
Sep 10 19:26:20.859065 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:20.858952    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8r27k"
Sep 10 19:26:20.859581 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:20.859561    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8r27k"
Sep 10 19:26:28.179568 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:28.179528    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:26:42.343976 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:42.343934    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:26:43.724180 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:43.724133    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-nc2pk"]
Sep 10 19:26:43.728675 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:43.728652    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nc2pk"
Sep 10 19:26:43.732596 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:43.732549    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 10 19:26:43.732735 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:43.732549    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-28lpp\""
Sep 10 19:26:43.732735 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:43.732556    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 10 19:26:43.732735 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:43.732583    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 10 19:26:43.738218 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:43.738197    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-nc2pk"]
Sep 10 19:26:43.827580 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:43.827550    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4x4h\" (UniqueName: \"kubernetes.io/projected/8612f429-9013-4747-a3a5-b680fedc3021-kube-api-access-s4x4h\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nc2pk\" (UID: \"8612f429-9013-4747-a3a5-b680fedc3021\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nc2pk"
Sep 10 19:26:43.827794 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:43.827607    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8612f429-9013-4747-a3a5-b680fedc3021-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nc2pk\" (UID: \"8612f429-9013-4747-a3a5-b680fedc3021\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nc2pk"
Sep 10 19:26:43.827794 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:43.827743    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8612f429-9013-4747-a3a5-b680fedc3021-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nc2pk\" (UID: \"8612f429-9013-4747-a3a5-b680fedc3021\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nc2pk"
Sep 10 19:26:43.827925 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:43.827816    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8612f429-9013-4747-a3a5-b680fedc3021-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nc2pk\" (UID: \"8612f429-9013-4747-a3a5-b680fedc3021\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nc2pk"
Sep 10 19:26:43.827925 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:43.827849    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8612f429-9013-4747-a3a5-b680fedc3021-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nc2pk\" (UID: \"8612f429-9013-4747-a3a5-b680fedc3021\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nc2pk"
Sep 10 19:26:43.827925 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:43.827881    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8612f429-9013-4747-a3a5-b680fedc3021-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nc2pk\" (UID: \"8612f429-9013-4747-a3a5-b680fedc3021\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nc2pk"
Sep 10 19:26:43.828784 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:43.828750    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:26:43.929179 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:43.929147    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8612f429-9013-4747-a3a5-b680fedc3021-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nc2pk\" (UID: \"8612f429-9013-4747-a3a5-b680fedc3021\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nc2pk"
Sep 10 19:26:43.929363 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:43.929194    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8612f429-9013-4747-a3a5-b680fedc3021-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nc2pk\" (UID: \"8612f429-9013-4747-a3a5-b680fedc3021\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nc2pk"
Sep 10 19:26:43.929363 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:43.929213    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8612f429-9013-4747-a3a5-b680fedc3021-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nc2pk\" (UID: \"8612f429-9013-4747-a3a5-b680fedc3021\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nc2pk"
Sep 10 19:26:43.929363 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:43.929238    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8612f429-9013-4747-a3a5-b680fedc3021-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nc2pk\" (UID: \"8612f429-9013-4747-a3a5-b680fedc3021\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nc2pk"
Sep 10 19:26:43.929363 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:43.929293    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4x4h\" (UniqueName: \"kubernetes.io/projected/8612f429-9013-4747-a3a5-b680fedc3021-kube-api-access-s4x4h\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nc2pk\" (UID: \"8612f429-9013-4747-a3a5-b680fedc3021\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nc2pk"
Sep 10 19:26:43.929363 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:43.929354    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8612f429-9013-4747-a3a5-b680fedc3021-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nc2pk\" (UID: \"8612f429-9013-4747-a3a5-b680fedc3021\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nc2pk"
Sep 10 19:26:43.929698 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:43.929674    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8612f429-9013-4747-a3a5-b680fedc3021-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nc2pk\" (UID: \"8612f429-9013-4747-a3a5-b680fedc3021\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nc2pk"
Sep 10 19:26:43.929775 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:43.929674    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8612f429-9013-4747-a3a5-b680fedc3021-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nc2pk\" (UID: \"8612f429-9013-4747-a3a5-b680fedc3021\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nc2pk"
Sep 10 19:26:43.929775 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:43.929734    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8612f429-9013-4747-a3a5-b680fedc3021-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nc2pk\" (UID: \"8612f429-9013-4747-a3a5-b680fedc3021\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nc2pk"
Sep 10 19:26:43.931548 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:43.931523    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8612f429-9013-4747-a3a5-b680fedc3021-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nc2pk\" (UID: \"8612f429-9013-4747-a3a5-b680fedc3021\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nc2pk"
Sep 10 19:26:43.932035 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:43.932013    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8612f429-9013-4747-a3a5-b680fedc3021-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nc2pk\" (UID: \"8612f429-9013-4747-a3a5-b680fedc3021\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nc2pk"
Sep 10 19:26:43.941494 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:43.941466    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4x4h\" (UniqueName: \"kubernetes.io/projected/8612f429-9013-4747-a3a5-b680fedc3021-kube-api-access-s4x4h\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nc2pk\" (UID: \"8612f429-9013-4747-a3a5-b680fedc3021\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nc2pk"
Sep 10 19:26:44.039802 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:44.039766    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nc2pk"
Sep 10 19:26:44.327896 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:44.327868    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-nc2pk"]
Sep 10 19:26:44.329876 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:26:44.329847    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8612f429_9013_4747_a3a5_b680fedc3021.slice/crio-a4f18abbbb0fcdeb30ea8c84fd690b34345ab5023c098863c5275b9bd4df40bd WatchSource:0}: Error finding container a4f18abbbb0fcdeb30ea8c84fd690b34345ab5023c098863c5275b9bd4df40bd: Status 404 returned error can't find the container with id a4f18abbbb0fcdeb30ea8c84fd690b34345ab5023c098863c5275b9bd4df40bd
Sep 10 19:26:44.331203 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:44.331185    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 19:26:44.942117 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:44.942078    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nc2pk" event={"ID":"8612f429-9013-4747-a3a5-b680fedc3021","Type":"ContainerStarted","Data":"a4f18abbbb0fcdeb30ea8c84fd690b34345ab5023c098863c5275b9bd4df40bd"}
Sep 10 19:26:45.948279 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:45.948243    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nc2pk" event={"ID":"8612f429-9013-4747-a3a5-b680fedc3021","Type":"ContainerStarted","Data":"7f5048e9f6493005e33ad1ffd2c8f27470d4795e9a0cdaeb897c215b6ef79295"}
Sep 10 19:26:45.976430 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:45.976377    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nc2pk" podStartSLOduration=1.900421812 podStartE2EDuration="2.976359337s" totalImagesPullingTime="1.075937525s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:26:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:26:45.975156198 +0000 UTC m=+661.576063438" watchObservedRunningTime="2026-09-10 19:26:45.976359337 +0000 UTC m=+661.577266577"
Sep 10 19:26:51.835906 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:51.835866    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:26:54.040578 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:54.040541    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nc2pk"
Sep 10 19:26:54.040998 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:54.040615    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nc2pk"
Sep 10 19:26:54.053446 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:54.053419    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nc2pk"
Sep 10 19:26:55.002876 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:26:55.002849    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nc2pk"
Sep 10 19:27:01.826434 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:27:01.826400    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-t7fwh"]
Sep 10 19:27:01.830270 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:27:01.830249    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t7fwh"
Sep 10 19:27:01.833295 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:27:01.833266    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Sep 10 19:27:01.847100 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:27:01.847060    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-t7fwh"]
Sep 10 19:27:01.851852 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:27:01.851804    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:01.895500 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:27:01.895461    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sk7qg\" (UniqueName: \"kubernetes.io/projected/f937b290-1298-4e30-b917-0b8e83243444-kube-api-access-sk7qg\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t7fwh\" (UID: \"f937b290-1298-4e30-b917-0b8e83243444\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t7fwh"
Sep 10 19:27:01.895727 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:27:01.895515    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f937b290-1298-4e30-b917-0b8e83243444-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t7fwh\" (UID: \"f937b290-1298-4e30-b917-0b8e83243444\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t7fwh"
Sep 10 19:27:01.895727 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:27:01.895538    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f937b290-1298-4e30-b917-0b8e83243444-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t7fwh\" (UID: \"f937b290-1298-4e30-b917-0b8e83243444\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t7fwh"
Sep 10 19:27:01.895727 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:27:01.895603    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f937b290-1298-4e30-b917-0b8e83243444-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t7fwh\" (UID: \"f937b290-1298-4e30-b917-0b8e83243444\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t7fwh"
Sep 10 19:27:01.895727 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:27:01.895679    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f937b290-1298-4e30-b917-0b8e83243444-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t7fwh\" (UID: \"f937b290-1298-4e30-b917-0b8e83243444\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t7fwh"
Sep 10 19:27:01.895727 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:27:01.895726    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f937b290-1298-4e30-b917-0b8e83243444-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t7fwh\" (UID: \"f937b290-1298-4e30-b917-0b8e83243444\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t7fwh"
Sep 10 19:27:01.996160 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:27:01.996111    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f937b290-1298-4e30-b917-0b8e83243444-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t7fwh\" (UID: \"f937b290-1298-4e30-b917-0b8e83243444\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t7fwh"
Sep 10 19:27:01.996356 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:27:01.996168    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f937b290-1298-4e30-b917-0b8e83243444-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t7fwh\" (UID: \"f937b290-1298-4e30-b917-0b8e83243444\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t7fwh"
Sep 10 19:27:01.996356 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:27:01.996232    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f937b290-1298-4e30-b917-0b8e83243444-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t7fwh\" (UID: \"f937b290-1298-4e30-b917-0b8e83243444\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t7fwh"
Sep 10 19:27:01.996356 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:27:01.996267    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sk7qg\" (UniqueName: \"kubernetes.io/projected/f937b290-1298-4e30-b917-0b8e83243444-kube-api-access-sk7qg\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t7fwh\" (UID: \"f937b290-1298-4e30-b917-0b8e83243444\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t7fwh"
Sep 10 19:27:01.996356 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:27:01.996305    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f937b290-1298-4e30-b917-0b8e83243444-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t7fwh\" (UID: \"f937b290-1298-4e30-b917-0b8e83243444\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t7fwh"
Sep 10 19:27:01.996582 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:27:01.996353    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f937b290-1298-4e30-b917-0b8e83243444-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t7fwh\" (UID: \"f937b290-1298-4e30-b917-0b8e83243444\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t7fwh"
Sep 10 19:27:01.996679 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:27:01.996606    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f937b290-1298-4e30-b917-0b8e83243444-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t7fwh\" (UID: \"f937b290-1298-4e30-b917-0b8e83243444\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t7fwh"
Sep 10 19:27:01.996679 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:27:01.996667    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f937b290-1298-4e30-b917-0b8e83243444-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t7fwh\" (UID: \"f937b290-1298-4e30-b917-0b8e83243444\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t7fwh"
Sep 10 19:27:01.996832 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:27:01.996812    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f937b290-1298-4e30-b917-0b8e83243444-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t7fwh\" (UID: \"f937b290-1298-4e30-b917-0b8e83243444\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t7fwh"
Sep 10 19:27:01.998709 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:27:01.998685    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f937b290-1298-4e30-b917-0b8e83243444-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t7fwh\" (UID: \"f937b290-1298-4e30-b917-0b8e83243444\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t7fwh"
Sep 10 19:27:01.998979 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:27:01.998960    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f937b290-1298-4e30-b917-0b8e83243444-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t7fwh\" (UID: \"f937b290-1298-4e30-b917-0b8e83243444\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t7fwh"
Sep 10 19:27:02.007074 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:27:02.007052    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sk7qg\" (UniqueName: \"kubernetes.io/projected/f937b290-1298-4e30-b917-0b8e83243444-kube-api-access-sk7qg\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t7fwh\" (UID: \"f937b290-1298-4e30-b917-0b8e83243444\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t7fwh"
Sep 10 19:27:02.140849 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:27:02.140750    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t7fwh"
Sep 10 19:27:02.421582 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:27:02.421557    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-t7fwh"]
Sep 10 19:27:02.423582 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:27:02.423555    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf937b290_1298_4e30_b917_0b8e83243444.slice/crio-ac92ac5d20fb28f8a56093bce09f6866e26181a1f308cf3ba1fb1d081ed354f9 WatchSource:0}: Error finding container ac92ac5d20fb28f8a56093bce09f6866e26181a1f308cf3ba1fb1d081ed354f9: Status 404 returned error can't find the container with id ac92ac5d20fb28f8a56093bce09f6866e26181a1f308cf3ba1fb1d081ed354f9
Sep 10 19:27:03.031417 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:27:03.031376    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t7fwh" event={"ID":"f937b290-1298-4e30-b917-0b8e83243444","Type":"ContainerStarted","Data":"3630d3e7a043d936d12a9a7d1ca81512c264e80b0fb35350ae6afc2deb0ddfbc"}
Sep 10 19:27:03.031853 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:27:03.031438    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t7fwh" event={"ID":"f937b290-1298-4e30-b917-0b8e83243444","Type":"ContainerStarted","Data":"ac92ac5d20fb28f8a56093bce09f6866e26181a1f308cf3ba1fb1d081ed354f9"}
Sep 10 19:27:03.054724 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:27:03.054670    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t7fwh" podStartSLOduration=1.863161627 podStartE2EDuration="2.054652144s" totalImagesPullingTime="191.490517ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:27:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:27:03.053283791 +0000 UTC m=+678.654191033" watchObservedRunningTime="2026-09-10 19:27:03.054652144 +0000 UTC m=+678.655559388"
Sep 10 19:27:03.115750 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:27:03.115717    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:11.567534 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:27:11.567495    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:12.140942 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:27:12.140900    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t7fwh"
Sep 10 19:27:12.141142 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:27:12.140994    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t7fwh"
Sep 10 19:27:12.153910 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:27:12.153882    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t7fwh"
Sep 10 19:27:13.099149 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:27:13.099112    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t7fwh"
Sep 10 19:27:19.647289 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:27:19.647253    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:37.061742 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:27:37.061695    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:01.460758 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:01.460716    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:01.556648 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:01.556586    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:02.644222 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:02.644189    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:02.739957 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:02.739924    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:04.148154 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:04.148122    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:18.425483 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:18.425375    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:18.519487 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:18.519457    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:18.711035 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:18.710954    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b"]
Sep 10 19:29:18.715249 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:18.715218    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b"
Sep 10 19:29:18.717489 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:18.717468    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-173596-openshift-default-dockercfg-l7xpz\""
Sep 10 19:29:18.727363 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:18.727334    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b"]
Sep 10 19:29:18.738312 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:18.738277    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:18.764729 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:18.764694    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f43262c-1566-404b-8e4b-fbe09a10913f-workload-socket\") pod \"e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b\" (UID: \"5f43262c-1566-404b-8e4b-fbe09a10913f\") " pod="openshift-ingress/e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b"
Sep 10 19:29:18.764908 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:18.764735    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f43262c-1566-404b-8e4b-fbe09a10913f-workload-certs\") pod \"e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b\" (UID: \"5f43262c-1566-404b-8e4b-fbe09a10913f\") " pod="openshift-ingress/e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b"
Sep 10 19:29:18.764908 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:18.764762    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f43262c-1566-404b-8e4b-fbe09a10913f-istiod-ca-cert\") pod \"e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b\" (UID: \"5f43262c-1566-404b-8e4b-fbe09a10913f\") " pod="openshift-ingress/e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b"
Sep 10 19:29:18.765029 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:18.764903    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f43262c-1566-404b-8e4b-fbe09a10913f-credential-socket\") pod \"e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b\" (UID: \"5f43262c-1566-404b-8e4b-fbe09a10913f\") " pod="openshift-ingress/e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b"
Sep 10 19:29:18.765029 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:18.764951    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f43262c-1566-404b-8e4b-fbe09a10913f-istio-envoy\") pod \"e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b\" (UID: \"5f43262c-1566-404b-8e4b-fbe09a10913f\") " pod="openshift-ingress/e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b"
Sep 10 19:29:18.765133 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:18.765053    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f43262c-1566-404b-8e4b-fbe09a10913f-istio-data\") pod \"e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b\" (UID: \"5f43262c-1566-404b-8e4b-fbe09a10913f\") " pod="openshift-ingress/e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b"
Sep 10 19:29:18.765133 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:18.765086    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f43262c-1566-404b-8e4b-fbe09a10913f-istio-token\") pod \"e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b\" (UID: \"5f43262c-1566-404b-8e4b-fbe09a10913f\") " pod="openshift-ingress/e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b"
Sep 10 19:29:18.765133 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:18.765123    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f43262c-1566-404b-8e4b-fbe09a10913f-istio-podinfo\") pod \"e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b\" (UID: \"5f43262c-1566-404b-8e4b-fbe09a10913f\") " pod="openshift-ingress/e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b"
Sep 10 19:29:18.765299 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:18.765165    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5t4nl\" (UniqueName: \"kubernetes.io/projected/5f43262c-1566-404b-8e4b-fbe09a10913f-kube-api-access-5t4nl\") pod \"e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b\" (UID: \"5f43262c-1566-404b-8e4b-fbe09a10913f\") " pod="openshift-ingress/e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b"
Sep 10 19:29:18.866110 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:18.866064    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f43262c-1566-404b-8e4b-fbe09a10913f-istio-data\") pod \"e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b\" (UID: \"5f43262c-1566-404b-8e4b-fbe09a10913f\") " pod="openshift-ingress/e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b"
Sep 10 19:29:18.866110 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:18.866112    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f43262c-1566-404b-8e4b-fbe09a10913f-istio-token\") pod \"e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b\" (UID: \"5f43262c-1566-404b-8e4b-fbe09a10913f\") " pod="openshift-ingress/e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b"
Sep 10 19:29:18.866368 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:18.866142    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f43262c-1566-404b-8e4b-fbe09a10913f-istio-podinfo\") pod \"e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b\" (UID: \"5f43262c-1566-404b-8e4b-fbe09a10913f\") " pod="openshift-ingress/e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b"
Sep 10 19:29:18.866368 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:18.866172    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5t4nl\" (UniqueName: \"kubernetes.io/projected/5f43262c-1566-404b-8e4b-fbe09a10913f-kube-api-access-5t4nl\") pod \"e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b\" (UID: \"5f43262c-1566-404b-8e4b-fbe09a10913f\") " pod="openshift-ingress/e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b"
Sep 10 19:29:18.866368 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:18.866247    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f43262c-1566-404b-8e4b-fbe09a10913f-workload-socket\") pod \"e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b\" (UID: \"5f43262c-1566-404b-8e4b-fbe09a10913f\") " pod="openshift-ingress/e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b"
Sep 10 19:29:18.866368 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:18.866268    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f43262c-1566-404b-8e4b-fbe09a10913f-workload-certs\") pod \"e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b\" (UID: \"5f43262c-1566-404b-8e4b-fbe09a10913f\") " pod="openshift-ingress/e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b"
Sep 10 19:29:18.866368 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:18.866289    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f43262c-1566-404b-8e4b-fbe09a10913f-istiod-ca-cert\") pod \"e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b\" (UID: \"5f43262c-1566-404b-8e4b-fbe09a10913f\") " pod="openshift-ingress/e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b"
Sep 10 19:29:18.866368 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:18.866356    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f43262c-1566-404b-8e4b-fbe09a10913f-credential-socket\") pod \"e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b\" (UID: \"5f43262c-1566-404b-8e4b-fbe09a10913f\") " pod="openshift-ingress/e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b"
Sep 10 19:29:18.866694 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:18.866385    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f43262c-1566-404b-8e4b-fbe09a10913f-istio-envoy\") pod \"e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b\" (UID: \"5f43262c-1566-404b-8e4b-fbe09a10913f\") " pod="openshift-ingress/e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b"
Sep 10 19:29:18.866945 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:18.866889    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f43262c-1566-404b-8e4b-fbe09a10913f-credential-socket\") pod \"e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b\" (UID: \"5f43262c-1566-404b-8e4b-fbe09a10913f\") " pod="openshift-ingress/e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b"
Sep 10 19:29:18.867085 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:18.866967    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f43262c-1566-404b-8e4b-fbe09a10913f-workload-socket\") pod \"e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b\" (UID: \"5f43262c-1566-404b-8e4b-fbe09a10913f\") " pod="openshift-ingress/e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b"
Sep 10 19:29:18.867085 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:18.867055    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f43262c-1566-404b-8e4b-fbe09a10913f-workload-certs\") pod \"e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b\" (UID: \"5f43262c-1566-404b-8e4b-fbe09a10913f\") " pod="openshift-ingress/e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b"
Sep 10 19:29:18.867268 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:18.867167    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f43262c-1566-404b-8e4b-fbe09a10913f-istiod-ca-cert\") pod \"e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b\" (UID: \"5f43262c-1566-404b-8e4b-fbe09a10913f\") " pod="openshift-ingress/e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b"
Sep 10 19:29:18.867268 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:18.867166    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f43262c-1566-404b-8e4b-fbe09a10913f-istio-data\") pod \"e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b\" (UID: \"5f43262c-1566-404b-8e4b-fbe09a10913f\") " pod="openshift-ingress/e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b"
Sep 10 19:29:18.868767 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:18.868739    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f43262c-1566-404b-8e4b-fbe09a10913f-istio-envoy\") pod \"e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b\" (UID: \"5f43262c-1566-404b-8e4b-fbe09a10913f\") " pod="openshift-ingress/e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b"
Sep 10 19:29:18.869017 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:18.868987    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f43262c-1566-404b-8e4b-fbe09a10913f-istio-podinfo\") pod \"e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b\" (UID: \"5f43262c-1566-404b-8e4b-fbe09a10913f\") " pod="openshift-ingress/e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b"
Sep 10 19:29:18.874380 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:18.874351    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f43262c-1566-404b-8e4b-fbe09a10913f-istio-token\") pod \"e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b\" (UID: \"5f43262c-1566-404b-8e4b-fbe09a10913f\") " pod="openshift-ingress/e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b"
Sep 10 19:29:18.874747 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:18.874728    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5t4nl\" (UniqueName: \"kubernetes.io/projected/5f43262c-1566-404b-8e4b-fbe09a10913f-kube-api-access-5t4nl\") pod \"e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b\" (UID: \"5f43262c-1566-404b-8e4b-fbe09a10913f\") " pod="openshift-ingress/e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b"
Sep 10 19:29:19.026149 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:19.026115    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b"
Sep 10 19:29:19.033606 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:19.033570    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:19.307121 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:19.307089    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b"]
Sep 10 19:29:19.308177 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:29:19.308152    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f43262c_1566_404b_8e4b_fbe09a10913f.slice/crio-5a9d2303e5d3ce9a7abf7a60872a7a641115b466ee3227bb300fd23f7c1798f5 WatchSource:0}: Error finding container 5a9d2303e5d3ce9a7abf7a60872a7a641115b466ee3227bb300fd23f7c1798f5: Status 404 returned error can't find the container with id 5a9d2303e5d3ce9a7abf7a60872a7a641115b466ee3227bb300fd23f7c1798f5
Sep 10 19:29:19.310551 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:19.310517    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:29:19.310666 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:19.310585    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:29:19.310666 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:19.310636    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:29:19.711135 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:19.711046    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b" event={"ID":"5f43262c-1566-404b-8e4b-fbe09a10913f","Type":"ContainerStarted","Data":"a6054b6d9512bee4d937398714ecd847ddad16130ef81391277a7dde2b5862b2"}
Sep 10 19:29:19.711135 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:19.711092    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b" event={"ID":"5f43262c-1566-404b-8e4b-fbe09a10913f","Type":"ContainerStarted","Data":"5a9d2303e5d3ce9a7abf7a60872a7a641115b466ee3227bb300fd23f7c1798f5"}
Sep 10 19:29:19.732558 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:19.732505    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b" podStartSLOduration=1.732489718 podStartE2EDuration="1.732489718s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:29:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:29:19.730924263 +0000 UTC m=+815.331831504" watchObservedRunningTime="2026-09-10 19:29:19.732489718 +0000 UTC m=+815.333396957"
Sep 10 19:29:19.845665 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:19.845602    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:19.941667 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:19.941609    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:20.026987 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:20.026942    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b"
Sep 10 19:29:20.032142 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:20.032115    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b"
Sep 10 19:29:20.334840 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:20.334732    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:20.554473 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:20.554440    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:20.617631 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:20.617523    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:20.716243 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:20.716218    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b"
Sep 10 19:29:20.717268 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:20.717247    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b"
Sep 10 19:29:21.787376 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:21.787340    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:21.881249 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:21.881213    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:22.998089 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:22.998057    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:23.077760 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:23.077714    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:23.363559 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:23.363446    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:23.517205 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:23.517170    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:23.614976 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:23.614877    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:23.663312 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:23.663274    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-173596-586b787-5hxkf"]
Sep 10 19:29:23.667827 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:23.667798    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-173596-586b787-5hxkf"
Sep 10 19:29:23.671175 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:23.671149    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-173596-dockercfg-85hnf\""
Sep 10 19:29:23.671666 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:23.671452    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-173596\""
Sep 10 19:29:23.679085 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:23.678800    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-173596-586b787-5hxkf"]
Sep 10 19:29:23.719931 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:23.719893    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/77fd5b21-2532-4eae-b847-64079cce6907-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-173596-586b787-5hxkf\" (UID: \"77fd5b21-2532-4eae-b847-64079cce6907\") " pod="openshift-ingress/payload-processing-e2e-inf-a-173596-586b787-5hxkf"
Sep 10 19:29:23.720129 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:23.720003    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wr77l\" (UniqueName: \"kubernetes.io/projected/77fd5b21-2532-4eae-b847-64079cce6907-kube-api-access-wr77l\") pod \"payload-processing-e2e-inf-a-173596-586b787-5hxkf\" (UID: \"77fd5b21-2532-4eae-b847-64079cce6907\") " pod="openshift-ingress/payload-processing-e2e-inf-a-173596-586b787-5hxkf"
Sep 10 19:29:23.821169 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:23.821121    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wr77l\" (UniqueName: \"kubernetes.io/projected/77fd5b21-2532-4eae-b847-64079cce6907-kube-api-access-wr77l\") pod \"payload-processing-e2e-inf-a-173596-586b787-5hxkf\" (UID: \"77fd5b21-2532-4eae-b847-64079cce6907\") " pod="openshift-ingress/payload-processing-e2e-inf-a-173596-586b787-5hxkf"
Sep 10 19:29:23.821374 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:23.821297    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/77fd5b21-2532-4eae-b847-64079cce6907-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-173596-586b787-5hxkf\" (UID: \"77fd5b21-2532-4eae-b847-64079cce6907\") " pod="openshift-ingress/payload-processing-e2e-inf-a-173596-586b787-5hxkf"
Sep 10 19:29:23.822275 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:23.822250    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/77fd5b21-2532-4eae-b847-64079cce6907-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-173596-586b787-5hxkf\" (UID: \"77fd5b21-2532-4eae-b847-64079cce6907\") " pod="openshift-ingress/payload-processing-e2e-inf-a-173596-586b787-5hxkf"
Sep 10 19:29:23.830747 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:23.830723    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wr77l\" (UniqueName: \"kubernetes.io/projected/77fd5b21-2532-4eae-b847-64079cce6907-kube-api-access-wr77l\") pod \"payload-processing-e2e-inf-a-173596-586b787-5hxkf\" (UID: \"77fd5b21-2532-4eae-b847-64079cce6907\") " pod="openshift-ingress/payload-processing-e2e-inf-a-173596-586b787-5hxkf"
Sep 10 19:29:23.982801 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:23.982763    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-173596-586b787-5hxkf"
Sep 10 19:29:24.258901 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:24.258872    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-173596-586b787-5hxkf"]
Sep 10 19:29:24.261215 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:29:24.261178    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77fd5b21_2532_4eae_b847_64079cce6907.slice/crio-6deb4595ec0268cbcfc4d5d13e75d0c24573dd6ec5a71c5bfd540d4a1b9b74ea WatchSource:0}: Error finding container 6deb4595ec0268cbcfc4d5d13e75d0c24573dd6ec5a71c5bfd540d4a1b9b74ea: Status 404 returned error can't find the container with id 6deb4595ec0268cbcfc4d5d13e75d0c24573dd6ec5a71c5bfd540d4a1b9b74ea
Sep 10 19:29:24.742329 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:24.742236    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-173596-586b787-5hxkf" event={"ID":"77fd5b21-2532-4eae-b847-64079cce6907","Type":"ContainerStarted","Data":"083022a27dc279cdfcb60c5c49a291dea71d867bddd1880b190631abbfd380b7"}
Sep 10 19:29:24.742329 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:24.742288    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-173596-586b787-5hxkf" event={"ID":"77fd5b21-2532-4eae-b847-64079cce6907","Type":"ContainerStarted","Data":"6deb4595ec0268cbcfc4d5d13e75d0c24573dd6ec5a71c5bfd540d4a1b9b74ea"}
Sep 10 19:29:24.742538 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:24.742377    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-173596-586b787-5hxkf"
Sep 10 19:29:24.761491 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:24.761427    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-173596-586b787-5hxkf" podStartSLOduration=1.761378456 podStartE2EDuration="1.761378456s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:29:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:29:24.760521424 +0000 UTC m=+820.361428665" watchObservedRunningTime="2026-09-10 19:29:24.761378456 +0000 UTC m=+820.362285697"
Sep 10 19:29:26.220580 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:26.220543    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:26.495681 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:26.495575    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b"]
Sep 10 19:29:26.499932 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:26.499908    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b"
Sep 10 19:29:26.502326 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:26.502300    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-autores-5352fd-openshift-default-dockercfg-7jj6z\""
Sep 10 19:29:26.511262 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:26.511233    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b"]
Sep 10 19:29:26.526846 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:26.526811    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:26.646275 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:26.646246    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a02f4a9-bd86-4658-aa0a-a82758731f76-istio-podinfo\") pod \"e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b\" (UID: \"1a02f4a9-bd86-4658-aa0a-a82758731f76\") " pod="openshift-ingress/e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b"
Sep 10 19:29:26.646275 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:26.646278    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a02f4a9-bd86-4658-aa0a-a82758731f76-credential-socket\") pod \"e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b\" (UID: \"1a02f4a9-bd86-4658-aa0a-a82758731f76\") " pod="openshift-ingress/e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b"
Sep 10 19:29:26.646528 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:26.646297    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a02f4a9-bd86-4658-aa0a-a82758731f76-istio-envoy\") pod \"e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b\" (UID: \"1a02f4a9-bd86-4658-aa0a-a82758731f76\") " pod="openshift-ingress/e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b"
Sep 10 19:29:26.646528 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:26.646429    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a02f4a9-bd86-4658-aa0a-a82758731f76-istiod-ca-cert\") pod \"e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b\" (UID: \"1a02f4a9-bd86-4658-aa0a-a82758731f76\") " pod="openshift-ingress/e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b"
Sep 10 19:29:26.646528 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:26.646447    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a02f4a9-bd86-4658-aa0a-a82758731f76-istio-data\") pod \"e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b\" (UID: \"1a02f4a9-bd86-4658-aa0a-a82758731f76\") " pod="openshift-ingress/e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b"
Sep 10 19:29:26.646717 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:26.646522    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a02f4a9-bd86-4658-aa0a-a82758731f76-istio-token\") pod \"e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b\" (UID: \"1a02f4a9-bd86-4658-aa0a-a82758731f76\") " pod="openshift-ingress/e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b"
Sep 10 19:29:26.646717 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:26.646643    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a02f4a9-bd86-4658-aa0a-a82758731f76-workload-certs\") pod \"e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b\" (UID: \"1a02f4a9-bd86-4658-aa0a-a82758731f76\") " pod="openshift-ingress/e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b"
Sep 10 19:29:26.646717 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:26.646694    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bf4xk\" (UniqueName: \"kubernetes.io/projected/1a02f4a9-bd86-4658-aa0a-a82758731f76-kube-api-access-bf4xk\") pod \"e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b\" (UID: \"1a02f4a9-bd86-4658-aa0a-a82758731f76\") " pod="openshift-ingress/e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b"
Sep 10 19:29:26.646848 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:26.646723    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a02f4a9-bd86-4658-aa0a-a82758731f76-workload-socket\") pod \"e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b\" (UID: \"1a02f4a9-bd86-4658-aa0a-a82758731f76\") " pod="openshift-ingress/e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b"
Sep 10 19:29:26.748100 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:26.748073    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a02f4a9-bd86-4658-aa0a-a82758731f76-workload-certs\") pod \"e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b\" (UID: \"1a02f4a9-bd86-4658-aa0a-a82758731f76\") " pod="openshift-ingress/e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b"
Sep 10 19:29:26.748273 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:26.748117    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bf4xk\" (UniqueName: \"kubernetes.io/projected/1a02f4a9-bd86-4658-aa0a-a82758731f76-kube-api-access-bf4xk\") pod \"e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b\" (UID: \"1a02f4a9-bd86-4658-aa0a-a82758731f76\") " pod="openshift-ingress/e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b"
Sep 10 19:29:26.748273 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:26.748138    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a02f4a9-bd86-4658-aa0a-a82758731f76-workload-socket\") pod \"e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b\" (UID: \"1a02f4a9-bd86-4658-aa0a-a82758731f76\") " pod="openshift-ingress/e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b"
Sep 10 19:29:26.748273 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:26.748163    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a02f4a9-bd86-4658-aa0a-a82758731f76-istio-podinfo\") pod \"e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b\" (UID: \"1a02f4a9-bd86-4658-aa0a-a82758731f76\") " pod="openshift-ingress/e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b"
Sep 10 19:29:26.748273 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:26.748179    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a02f4a9-bd86-4658-aa0a-a82758731f76-credential-socket\") pod \"e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b\" (UID: \"1a02f4a9-bd86-4658-aa0a-a82758731f76\") " pod="openshift-ingress/e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b"
Sep 10 19:29:26.748273 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:26.748200    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a02f4a9-bd86-4658-aa0a-a82758731f76-istio-envoy\") pod \"e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b\" (UID: \"1a02f4a9-bd86-4658-aa0a-a82758731f76\") " pod="openshift-ingress/e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b"
Sep 10 19:29:26.748273 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:26.748273    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a02f4a9-bd86-4658-aa0a-a82758731f76-istiod-ca-cert\") pod \"e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b\" (UID: \"1a02f4a9-bd86-4658-aa0a-a82758731f76\") " pod="openshift-ingress/e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b"
Sep 10 19:29:26.748582 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:26.748560    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a02f4a9-bd86-4658-aa0a-a82758731f76-workload-certs\") pod \"e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b\" (UID: \"1a02f4a9-bd86-4658-aa0a-a82758731f76\") " pod="openshift-ingress/e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b"
Sep 10 19:29:26.748677 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:26.748618    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a02f4a9-bd86-4658-aa0a-a82758731f76-istio-data\") pod \"e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b\" (UID: \"1a02f4a9-bd86-4658-aa0a-a82758731f76\") " pod="openshift-ingress/e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b"
Sep 10 19:29:26.748741 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:26.748683    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a02f4a9-bd86-4658-aa0a-a82758731f76-workload-socket\") pod \"e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b\" (UID: \"1a02f4a9-bd86-4658-aa0a-a82758731f76\") " pod="openshift-ingress/e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b"
Sep 10 19:29:26.748741 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:26.748684    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a02f4a9-bd86-4658-aa0a-a82758731f76-credential-socket\") pod \"e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b\" (UID: \"1a02f4a9-bd86-4658-aa0a-a82758731f76\") " pod="openshift-ingress/e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b"
Sep 10 19:29:26.748741 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:26.748727    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a02f4a9-bd86-4658-aa0a-a82758731f76-istio-token\") pod \"e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b\" (UID: \"1a02f4a9-bd86-4658-aa0a-a82758731f76\") " pod="openshift-ingress/e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b"
Sep 10 19:29:26.748968 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:26.748948    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a02f4a9-bd86-4658-aa0a-a82758731f76-istio-data\") pod \"e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b\" (UID: \"1a02f4a9-bd86-4658-aa0a-a82758731f76\") " pod="openshift-ingress/e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b"
Sep 10 19:29:26.749051 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:26.749026    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a02f4a9-bd86-4658-aa0a-a82758731f76-istiod-ca-cert\") pod \"e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b\" (UID: \"1a02f4a9-bd86-4658-aa0a-a82758731f76\") " pod="openshift-ingress/e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b"
Sep 10 19:29:26.750794 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:26.750768    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a02f4a9-bd86-4658-aa0a-a82758731f76-istio-envoy\") pod \"e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b\" (UID: \"1a02f4a9-bd86-4658-aa0a-a82758731f76\") " pod="openshift-ingress/e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b"
Sep 10 19:29:26.750897 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:26.750859    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a02f4a9-bd86-4658-aa0a-a82758731f76-istio-podinfo\") pod \"e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b\" (UID: \"1a02f4a9-bd86-4658-aa0a-a82758731f76\") " pod="openshift-ingress/e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b"
Sep 10 19:29:26.763805 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:26.763374    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a02f4a9-bd86-4658-aa0a-a82758731f76-istio-token\") pod \"e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b\" (UID: \"1a02f4a9-bd86-4658-aa0a-a82758731f76\") " pod="openshift-ingress/e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b"
Sep 10 19:29:26.763984 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:26.763959    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bf4xk\" (UniqueName: \"kubernetes.io/projected/1a02f4a9-bd86-4658-aa0a-a82758731f76-kube-api-access-bf4xk\") pod \"e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b\" (UID: \"1a02f4a9-bd86-4658-aa0a-a82758731f76\") " pod="openshift-ingress/e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b"
Sep 10 19:29:26.779303 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:26.779250    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:26.812196 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:26.812157    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b"
Sep 10 19:29:26.881719 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:26.881661    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:26.981389 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:26.981344    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:27.084800 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:27.084713    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:27.098916 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:27.098893    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b"]
Sep 10 19:29:27.100786 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:29:27.100758    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a02f4a9_bd86_4658_aa0a_a82758731f76.slice/crio-ae7306c5d82c336f0a2ae4391f62f200679aa3655abc4506c900692dacf49c40 WatchSource:0}: Error finding container ae7306c5d82c336f0a2ae4391f62f200679aa3655abc4506c900692dacf49c40: Status 404 returned error can't find the container with id ae7306c5d82c336f0a2ae4391f62f200679aa3655abc4506c900692dacf49c40
Sep 10 19:29:27.103132 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:27.103099    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:29:27.103211 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:27.103170    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:29:27.103211 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:27.103197    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:29:27.289306 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:27.289268    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns"]
Sep 10 19:29:27.293698 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:27.293674    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns"
Sep 10 19:29:27.296078 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:27.296056    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-e7fa2c81-openshift-default-dockercfg-6kdsl\""
Sep 10 19:29:27.303687 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:27.303660    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns"]
Sep 10 19:29:27.320739 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:27.320705    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:27.353933 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:27.353851    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/050f7885-379b-4256-b25f-4e8f437dc8b9-workload-socket\") pod \"e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns\" (UID: \"050f7885-379b-4256-b25f-4e8f437dc8b9\") " pod="openshift-ingress/e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns"
Sep 10 19:29:27.353933 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:27.353908    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/050f7885-379b-4256-b25f-4e8f437dc8b9-workload-certs\") pod \"e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns\" (UID: \"050f7885-379b-4256-b25f-4e8f437dc8b9\") " pod="openshift-ingress/e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns"
Sep 10 19:29:27.354106 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:27.354014    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/050f7885-379b-4256-b25f-4e8f437dc8b9-istio-token\") pod \"e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns\" (UID: \"050f7885-379b-4256-b25f-4e8f437dc8b9\") " pod="openshift-ingress/e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns"
Sep 10 19:29:27.354106 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:27.354053    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/050f7885-379b-4256-b25f-4e8f437dc8b9-istio-podinfo\") pod \"e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns\" (UID: \"050f7885-379b-4256-b25f-4e8f437dc8b9\") " pod="openshift-ingress/e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns"
Sep 10 19:29:27.354181 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:27.354110    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/050f7885-379b-4256-b25f-4e8f437dc8b9-istio-data\") pod \"e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns\" (UID: \"050f7885-379b-4256-b25f-4e8f437dc8b9\") " pod="openshift-ingress/e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns"
Sep 10 19:29:27.354181 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:27.354139    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/050f7885-379b-4256-b25f-4e8f437dc8b9-credential-socket\") pod \"e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns\" (UID: \"050f7885-379b-4256-b25f-4e8f437dc8b9\") " pod="openshift-ingress/e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns"
Sep 10 19:29:27.354250 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:27.354217    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/050f7885-379b-4256-b25f-4e8f437dc8b9-istiod-ca-cert\") pod \"e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns\" (UID: \"050f7885-379b-4256-b25f-4e8f437dc8b9\") " pod="openshift-ingress/e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns"
Sep 10 19:29:27.354289 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:27.354252    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/050f7885-379b-4256-b25f-4e8f437dc8b9-istio-envoy\") pod \"e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns\" (UID: \"050f7885-379b-4256-b25f-4e8f437dc8b9\") " pod="openshift-ingress/e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns"
Sep 10 19:29:27.354336 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:27.354290    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggjwr\" (UniqueName: \"kubernetes.io/projected/050f7885-379b-4256-b25f-4e8f437dc8b9-kube-api-access-ggjwr\") pod \"e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns\" (UID: \"050f7885-379b-4256-b25f-4e8f437dc8b9\") " pod="openshift-ingress/e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns"
Sep 10 19:29:27.454947 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:27.454915    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/050f7885-379b-4256-b25f-4e8f437dc8b9-workload-socket\") pod \"e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns\" (UID: \"050f7885-379b-4256-b25f-4e8f437dc8b9\") " pod="openshift-ingress/e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns"
Sep 10 19:29:27.455145 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:27.454978    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/050f7885-379b-4256-b25f-4e8f437dc8b9-workload-certs\") pod \"e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns\" (UID: \"050f7885-379b-4256-b25f-4e8f437dc8b9\") " pod="openshift-ingress/e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns"
Sep 10 19:29:27.455145 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:27.455008    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/050f7885-379b-4256-b25f-4e8f437dc8b9-istio-token\") pod \"e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns\" (UID: \"050f7885-379b-4256-b25f-4e8f437dc8b9\") " pod="openshift-ingress/e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns"
Sep 10 19:29:27.455145 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:27.455027    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/050f7885-379b-4256-b25f-4e8f437dc8b9-istio-podinfo\") pod \"e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns\" (UID: \"050f7885-379b-4256-b25f-4e8f437dc8b9\") " pod="openshift-ingress/e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns"
Sep 10 19:29:27.455145 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:27.455056    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/050f7885-379b-4256-b25f-4e8f437dc8b9-istio-data\") pod \"e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns\" (UID: \"050f7885-379b-4256-b25f-4e8f437dc8b9\") " pod="openshift-ingress/e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns"
Sep 10 19:29:27.455372 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:27.455209    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/050f7885-379b-4256-b25f-4e8f437dc8b9-credential-socket\") pod \"e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns\" (UID: \"050f7885-379b-4256-b25f-4e8f437dc8b9\") " pod="openshift-ingress/e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns"
Sep 10 19:29:27.455372 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:27.455297    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/050f7885-379b-4256-b25f-4e8f437dc8b9-istiod-ca-cert\") pod \"e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns\" (UID: \"050f7885-379b-4256-b25f-4e8f437dc8b9\") " pod="openshift-ingress/e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns"
Sep 10 19:29:27.455372 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:27.455330    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/050f7885-379b-4256-b25f-4e8f437dc8b9-istio-envoy\") pod \"e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns\" (UID: \"050f7885-379b-4256-b25f-4e8f437dc8b9\") " pod="openshift-ingress/e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns"
Sep 10 19:29:27.455535 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:27.455385    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/050f7885-379b-4256-b25f-4e8f437dc8b9-workload-certs\") pod \"e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns\" (UID: \"050f7885-379b-4256-b25f-4e8f437dc8b9\") " pod="openshift-ingress/e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns"
Sep 10 19:29:27.455535 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:27.455393    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggjwr\" (UniqueName: \"kubernetes.io/projected/050f7885-379b-4256-b25f-4e8f437dc8b9-kube-api-access-ggjwr\") pod \"e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns\" (UID: \"050f7885-379b-4256-b25f-4e8f437dc8b9\") " pod="openshift-ingress/e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns"
Sep 10 19:29:27.455535 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:27.455481    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/050f7885-379b-4256-b25f-4e8f437dc8b9-istio-data\") pod \"e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns\" (UID: \"050f7885-379b-4256-b25f-4e8f437dc8b9\") " pod="openshift-ingress/e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns"
Sep 10 19:29:27.455719 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:27.455605    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/050f7885-379b-4256-b25f-4e8f437dc8b9-workload-socket\") pod \"e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns\" (UID: \"050f7885-379b-4256-b25f-4e8f437dc8b9\") " pod="openshift-ingress/e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns"
Sep 10 19:29:27.455719 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:27.455668    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/050f7885-379b-4256-b25f-4e8f437dc8b9-credential-socket\") pod \"e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns\" (UID: \"050f7885-379b-4256-b25f-4e8f437dc8b9\") " pod="openshift-ingress/e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns"
Sep 10 19:29:27.456003 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:27.455978    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/050f7885-379b-4256-b25f-4e8f437dc8b9-istiod-ca-cert\") pod \"e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns\" (UID: \"050f7885-379b-4256-b25f-4e8f437dc8b9\") " pod="openshift-ingress/e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns"
Sep 10 19:29:27.457518 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:27.457498    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/050f7885-379b-4256-b25f-4e8f437dc8b9-istio-envoy\") pod \"e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns\" (UID: \"050f7885-379b-4256-b25f-4e8f437dc8b9\") " pod="openshift-ingress/e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns"
Sep 10 19:29:27.457752 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:27.457732    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/050f7885-379b-4256-b25f-4e8f437dc8b9-istio-podinfo\") pod \"e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns\" (UID: \"050f7885-379b-4256-b25f-4e8f437dc8b9\") " pod="openshift-ingress/e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns"
Sep 10 19:29:27.464704 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:27.464674    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggjwr\" (UniqueName: \"kubernetes.io/projected/050f7885-379b-4256-b25f-4e8f437dc8b9-kube-api-access-ggjwr\") pod \"e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns\" (UID: \"050f7885-379b-4256-b25f-4e8f437dc8b9\") " pod="openshift-ingress/e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns"
Sep 10 19:29:27.464837 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:27.464707    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/050f7885-379b-4256-b25f-4e8f437dc8b9-istio-token\") pod \"e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns\" (UID: \"050f7885-379b-4256-b25f-4e8f437dc8b9\") " pod="openshift-ingress/e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns"
Sep 10 19:29:27.464937 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:27.464838    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:27.608844 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:27.608762    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns"
Sep 10 19:29:27.763809 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:27.763755    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b" event={"ID":"1a02f4a9-bd86-4658-aa0a-a82758731f76","Type":"ContainerStarted","Data":"0719669fed48400ec1ffee0ad0988611190291cfd6751063730d5e6835b052a3"}
Sep 10 19:29:27.763999 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:27.763824    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b" event={"ID":"1a02f4a9-bd86-4658-aa0a-a82758731f76","Type":"ContainerStarted","Data":"ae7306c5d82c336f0a2ae4391f62f200679aa3655abc4506c900692dacf49c40"}
Sep 10 19:29:27.783195 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:27.783136    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b" podStartSLOduration=1.783119946 podStartE2EDuration="1.783119946s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:29:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:29:27.781398119 +0000 UTC m=+823.382305359" watchObservedRunningTime="2026-09-10 19:29:27.783119946 +0000 UTC m=+823.384027255"
Sep 10 19:29:27.813038 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:27.812991    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b"
Sep 10 19:29:27.886477 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:27.886447    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns"]
Sep 10 19:29:27.887499 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:29:27.887472    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod050f7885_379b_4256_b25f_4e8f437dc8b9.slice/crio-b591a370bcdb81c6e58565bf39a5af254e872d083767f53da645c91eccf287a8 WatchSource:0}: Error finding container b591a370bcdb81c6e58565bf39a5af254e872d083767f53da645c91eccf287a8: Status 404 returned error can't find the container with id b591a370bcdb81c6e58565bf39a5af254e872d083767f53da645c91eccf287a8
Sep 10 19:29:27.890205 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:27.890154    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:29:27.890342 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:27.890250    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:29:27.890449 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:27.890373    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:29:28.770776 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:28.770740    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns" event={"ID":"050f7885-379b-4256-b25f-4e8f437dc8b9","Type":"ContainerStarted","Data":"687f1dbc8dc4e59fdd35fc10fbf68541b05d56e93b354369310f5d4d63279cad"}
Sep 10 19:29:28.771191 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:28.770795    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns" event={"ID":"050f7885-379b-4256-b25f-4e8f437dc8b9","Type":"ContainerStarted","Data":"b591a370bcdb81c6e58565bf39a5af254e872d083767f53da645c91eccf287a8"}
Sep 10 19:29:28.791761 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:28.791707    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns" podStartSLOduration=1.791689431 podStartE2EDuration="1.791689431s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:29:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:29:28.790659644 +0000 UTC m=+824.391566879" watchObservedRunningTime="2026-09-10 19:29:28.791689431 +0000 UTC m=+824.392596721"
Sep 10 19:29:28.818388 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:28.818360    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b"
Sep 10 19:29:29.609800 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:29.609762    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns"
Sep 10 19:29:29.614985 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:29.614960    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns"
Sep 10 19:29:29.663475 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:29.663441    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:29.758984 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:29.758940    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:29.776765 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:29.776723    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns"
Sep 10 19:29:29.777193 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:29.776779    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b"
Sep 10 19:29:29.777792 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:29.777765    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns"
Sep 10 19:29:29.777932 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:29.777914    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b"
Sep 10 19:29:30.214553 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:30.214510    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:30.301296 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:30.301257    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:31.007952 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:31.003745    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:31.089640 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:31.089581    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:31.370203 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:31.370081    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:31.540067 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:31.540034    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:31.638256 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:31.638145    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:31.957603 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:31.957506    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:32.058590 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:32.058550    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:32.200173 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:32.200126    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:32.541555 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:32.541525    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:32.646015 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:32.645981    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:32.653766 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:32.653737    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e7fa2c81-849ccf7f78-4zgm7"]
Sep 10 19:29:32.657492 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:32.657475    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-e7fa2c81-849ccf7f78-4zgm7"
Sep 10 19:29:32.659612 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:32.659591    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-e7fa2c81\""
Sep 10 19:29:32.659758 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:32.659591    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-e7fa2c81-dockercfg-kxzft\""
Sep 10 19:29:32.681450 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:32.681413    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e7fa2c81-849ccf7f78-4zgm7"]
Sep 10 19:29:32.682084 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:32.682064    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e7fa2c81-69d97c9fd9-w7ngm"]
Sep 10 19:29:32.701366 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:32.698845    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e7fa2c81-69d97c9fd9-w7ngm"]
Sep 10 19:29:32.701366 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:32.698965    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-e7fa2c81-69d97c9fd9-w7ngm"
Sep 10 19:29:32.823018 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:32.822934    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/905033d9-9b29-4ed6-96f6-c1909587af70-plugins-config-volume\") pod \"payload-processing-e2e-ait-e7fa2c81-69d97c9fd9-w7ngm\" (UID: \"905033d9-9b29-4ed6-96f6-c1909587af70\") " pod="openshift-ingress/payload-processing-e2e-ait-e7fa2c81-69d97c9fd9-w7ngm"
Sep 10 19:29:32.823018 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:32.822997    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9n9l\" (UniqueName: \"kubernetes.io/projected/b8d1b28c-28fd-43b4-9193-95032ec530a7-kube-api-access-b9n9l\") pod \"payload-pre-processing-e2e-ait-e7fa2c81-849ccf7f78-4zgm7\" (UID: \"b8d1b28c-28fd-43b4-9193-95032ec530a7\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e7fa2c81-849ccf7f78-4zgm7"
Sep 10 19:29:32.823200 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:32.823038    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8d1b28c-28fd-43b4-9193-95032ec530a7-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-e7fa2c81-849ccf7f78-4zgm7\" (UID: \"b8d1b28c-28fd-43b4-9193-95032ec530a7\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e7fa2c81-849ccf7f78-4zgm7"
Sep 10 19:29:32.823200 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:32.823129    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsr8w\" (UniqueName: \"kubernetes.io/projected/905033d9-9b29-4ed6-96f6-c1909587af70-kube-api-access-hsr8w\") pod \"payload-processing-e2e-ait-e7fa2c81-69d97c9fd9-w7ngm\" (UID: \"905033d9-9b29-4ed6-96f6-c1909587af70\") " pod="openshift-ingress/payload-processing-e2e-ait-e7fa2c81-69d97c9fd9-w7ngm"
Sep 10 19:29:32.924527 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:32.924482    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsr8w\" (UniqueName: \"kubernetes.io/projected/905033d9-9b29-4ed6-96f6-c1909587af70-kube-api-access-hsr8w\") pod \"payload-processing-e2e-ait-e7fa2c81-69d97c9fd9-w7ngm\" (UID: \"905033d9-9b29-4ed6-96f6-c1909587af70\") " pod="openshift-ingress/payload-processing-e2e-ait-e7fa2c81-69d97c9fd9-w7ngm"
Sep 10 19:29:32.924741 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:32.924546    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/905033d9-9b29-4ed6-96f6-c1909587af70-plugins-config-volume\") pod \"payload-processing-e2e-ait-e7fa2c81-69d97c9fd9-w7ngm\" (UID: \"905033d9-9b29-4ed6-96f6-c1909587af70\") " pod="openshift-ingress/payload-processing-e2e-ait-e7fa2c81-69d97c9fd9-w7ngm"
Sep 10 19:29:32.924741 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:32.924608    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9n9l\" (UniqueName: \"kubernetes.io/projected/b8d1b28c-28fd-43b4-9193-95032ec530a7-kube-api-access-b9n9l\") pod \"payload-pre-processing-e2e-ait-e7fa2c81-849ccf7f78-4zgm7\" (UID: \"b8d1b28c-28fd-43b4-9193-95032ec530a7\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e7fa2c81-849ccf7f78-4zgm7"
Sep 10 19:29:32.924741 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:32.924694    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8d1b28c-28fd-43b4-9193-95032ec530a7-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-e7fa2c81-849ccf7f78-4zgm7\" (UID: \"b8d1b28c-28fd-43b4-9193-95032ec530a7\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e7fa2c81-849ccf7f78-4zgm7"
Sep 10 19:29:32.925291 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:32.925269    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/905033d9-9b29-4ed6-96f6-c1909587af70-plugins-config-volume\") pod \"payload-processing-e2e-ait-e7fa2c81-69d97c9fd9-w7ngm\" (UID: \"905033d9-9b29-4ed6-96f6-c1909587af70\") " pod="openshift-ingress/payload-processing-e2e-ait-e7fa2c81-69d97c9fd9-w7ngm"
Sep 10 19:29:32.925347 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:32.925335    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8d1b28c-28fd-43b4-9193-95032ec530a7-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-e7fa2c81-849ccf7f78-4zgm7\" (UID: \"b8d1b28c-28fd-43b4-9193-95032ec530a7\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e7fa2c81-849ccf7f78-4zgm7"
Sep 10 19:29:32.934135 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:32.934109    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsr8w\" (UniqueName: \"kubernetes.io/projected/905033d9-9b29-4ed6-96f6-c1909587af70-kube-api-access-hsr8w\") pod \"payload-processing-e2e-ait-e7fa2c81-69d97c9fd9-w7ngm\" (UID: \"905033d9-9b29-4ed6-96f6-c1909587af70\") " pod="openshift-ingress/payload-processing-e2e-ait-e7fa2c81-69d97c9fd9-w7ngm"
Sep 10 19:29:32.934242 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:32.934220    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9n9l\" (UniqueName: \"kubernetes.io/projected/b8d1b28c-28fd-43b4-9193-95032ec530a7-kube-api-access-b9n9l\") pod \"payload-pre-processing-e2e-ait-e7fa2c81-849ccf7f78-4zgm7\" (UID: \"b8d1b28c-28fd-43b4-9193-95032ec530a7\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e7fa2c81-849ccf7f78-4zgm7"
Sep 10 19:29:33.001813 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:33.001775    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-e7fa2c81-849ccf7f78-4zgm7"
Sep 10 19:29:33.010919 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:33.010894    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-e7fa2c81-69d97c9fd9-w7ngm"
Sep 10 19:29:33.113544 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:33.113508    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:33.297116 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:33.297087    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e7fa2c81-849ccf7f78-4zgm7"]
Sep 10 19:29:33.299335 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:29:33.299314    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8d1b28c_28fd_43b4_9193_95032ec530a7.slice/crio-a591d71d69cb45f0872f24437670b3fcb8e14ec91e722ed9f7890d452104b53f WatchSource:0}: Error finding container a591d71d69cb45f0872f24437670b3fcb8e14ec91e722ed9f7890d452104b53f: Status 404 returned error can't find the container with id a591d71d69cb45f0872f24437670b3fcb8e14ec91e722ed9f7890d452104b53f
Sep 10 19:29:33.324273 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:33.324244    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e7fa2c81-69d97c9fd9-w7ngm"]
Sep 10 19:29:33.328798 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:29:33.328770    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod905033d9_9b29_4ed6_96f6_c1909587af70.slice/crio-a4afd83b08254ce303c1677e38368e7f90520730af5f56a1cc58389a7695e080 WatchSource:0}: Error finding container a4afd83b08254ce303c1677e38368e7f90520730af5f56a1cc58389a7695e080: Status 404 returned error can't find the container with id a4afd83b08254ce303c1677e38368e7f90520730af5f56a1cc58389a7695e080
Sep 10 19:29:33.819306 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:33.819274    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-e7fa2c81-69d97c9fd9-w7ngm" event={"ID":"905033d9-9b29-4ed6-96f6-c1909587af70","Type":"ContainerStarted","Data":"fc3fcb9ce67f8dfa7621a8b3b72ab88e61cf82752863253fa892c1c388c72549"}
Sep 10 19:29:33.819549 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:33.819534    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-e7fa2c81-69d97c9fd9-w7ngm" event={"ID":"905033d9-9b29-4ed6-96f6-c1909587af70","Type":"ContainerStarted","Data":"a4afd83b08254ce303c1677e38368e7f90520730af5f56a1cc58389a7695e080"}
Sep 10 19:29:33.819809 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:33.819775    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-e7fa2c81-69d97c9fd9-w7ngm"
Sep 10 19:29:33.821166 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:33.821143    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-e7fa2c81-849ccf7f78-4zgm7" event={"ID":"b8d1b28c-28fd-43b4-9193-95032ec530a7","Type":"ContainerStarted","Data":"ca9a35e17da73f91d6bbd69cc3c9edc4451b2b4e3783140c0f02cf4c68a0b12a"}
Sep 10 19:29:33.821276 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:33.821181    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-e7fa2c81-849ccf7f78-4zgm7" event={"ID":"b8d1b28c-28fd-43b4-9193-95032ec530a7","Type":"ContainerStarted","Data":"a591d71d69cb45f0872f24437670b3fcb8e14ec91e722ed9f7890d452104b53f"}
Sep 10 19:29:33.821276 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:33.821201    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-e7fa2c81-849ccf7f78-4zgm7"
Sep 10 19:29:33.837580 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:33.837407    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-e7fa2c81-69d97c9fd9-w7ngm" podStartSLOduration=1.837387119 podStartE2EDuration="1.837387119s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:29:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:29:33.836550367 +0000 UTC m=+829.437457607" watchObservedRunningTime="2026-09-10 19:29:33.837387119 +0000 UTC m=+829.438294362"
Sep 10 19:29:33.853410 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:33.853364    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-e7fa2c81-849ccf7f78-4zgm7" podStartSLOduration=1.853349538 podStartE2EDuration="1.853349538s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:29:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:29:33.852858591 +0000 UTC m=+829.453765832" watchObservedRunningTime="2026-09-10 19:29:33.853349538 +0000 UTC m=+829.454256777"
Sep 10 19:29:33.907377 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:33.907323    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:35.751018 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:35.750993    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-173596-586b787-5hxkf"
Sep 10 19:29:37.385553 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:37.385516    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:37.537795 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:37.537758    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:37.635453 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:37.635413    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:37.734848 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:37.734764    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:39.853286 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:39.853255    2564 generic.go:356] "Generic (PLEG): container finished" podID="5f43262c-1566-404b-8e4b-fbe09a10913f" containerID="a6054b6d9512bee4d937398714ecd847ddad16130ef81391277a7dde2b5862b2" exitCode=0
Sep 10 19:29:39.853745 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:39.853332    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b" event={"ID":"5f43262c-1566-404b-8e4b-fbe09a10913f","Type":"ContainerDied","Data":"a6054b6d9512bee4d937398714ecd847ddad16130ef81391277a7dde2b5862b2"}
Sep 10 19:29:39.853793 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:39.853756    2564 scope.go:117] "RemoveContainer" containerID="a6054b6d9512bee4d937398714ecd847ddad16130ef81391277a7dde2b5862b2"
Sep 10 19:29:39.854870 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:39.854840    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:29:39.854948 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:39.854907    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:29:39.854948 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:39.854940    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:29:40.440656 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:40.440599    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:40.862287 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:40.862238    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b" event={"ID":"5f43262c-1566-404b-8e4b-fbe09a10913f","Type":"ContainerStarted","Data":"2060e830b128eefa2acd7a82d4eef525b24a1d7fb8d99c17bed1dd1799bc1061"}
Sep 10 19:29:41.027225 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:41.027185    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b"
Sep 10 19:29:41.027426 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:41.027305    2564 patch_prober.go:28] interesting pod/e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Sep 10 19:29:41.027426 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:41.027351    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b" podUID="5f43262c-1566-404b-8e4b-fbe09a10913f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Sep 10 19:29:42.027374 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:42.027325    2564 patch_prober.go:28] interesting pod/e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Sep 10 19:29:42.027875 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:42.027406    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b" podUID="5f43262c-1566-404b-8e4b-fbe09a10913f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Sep 10 19:29:42.867342 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:42.867298    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:43.026754 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:43.026702    2564 patch_prober.go:28] interesting pod/e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Sep 10 19:29:43.027056 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:43.027018    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b" podUID="5f43262c-1566-404b-8e4b-fbe09a10913f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Sep 10 19:29:43.937603 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:43.937390    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:44.026777 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:44.026671    2564 patch_prober.go:28] interesting pod/e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Sep 10 19:29:44.026777 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:44.026736    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b" podUID="5f43262c-1566-404b-8e4b-fbe09a10913f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Sep 10 19:29:44.829849 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:44.829806    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-e7fa2c81-849ccf7f78-4zgm7"
Sep 10 19:29:45.038801 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:45.038764    2564 patch_prober.go:28] interesting pod/e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Sep 10 19:29:45.039383 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:45.039349    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b" podUID="5f43262c-1566-404b-8e4b-fbe09a10913f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Sep 10 19:29:45.835047 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:45.834961    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-e7fa2c81-69d97c9fd9-w7ngm"
Sep 10 19:29:45.841891 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:45.841856    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:45.940214 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:45.940179    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:46.027024 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:46.026988    2564 patch_prober.go:28] interesting pod/e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Sep 10 19:29:46.027192 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:46.027058    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b" podUID="5f43262c-1566-404b-8e4b-fbe09a10913f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Sep 10 19:29:46.076690 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:46.076653    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:46.764408 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:46.764320    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:47.031595 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:47.031509    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b"
Sep 10 19:29:47.031919 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:47.031887    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b"
Sep 10 19:29:47.032900 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:47.032877    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b"
Sep 10 19:29:47.119739 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:47.119687    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:47.416528 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:47.416387    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:47.684224 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:47.684121    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:48.942044 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:48.942003    2564 generic.go:356] "Generic (PLEG): container finished" podID="050f7885-379b-4256-b25f-4e8f437dc8b9" containerID="687f1dbc8dc4e59fdd35fc10fbf68541b05d56e93b354369310f5d4d63279cad" exitCode=0
Sep 10 19:29:48.942512 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:48.942067    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns" event={"ID":"050f7885-379b-4256-b25f-4e8f437dc8b9","Type":"ContainerDied","Data":"687f1dbc8dc4e59fdd35fc10fbf68541b05d56e93b354369310f5d4d63279cad"}
Sep 10 19:29:48.955319 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:48.955278    2564 scope.go:117] "RemoveContainer" containerID="687f1dbc8dc4e59fdd35fc10fbf68541b05d56e93b354369310f5d4d63279cad"
Sep 10 19:29:48.957837 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:48.957790    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:29:48.957962 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:48.957895    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:29:48.957962 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:48.957950    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:29:49.949688 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:49.949654    2564 generic.go:356] "Generic (PLEG): container finished" podID="1a02f4a9-bd86-4658-aa0a-a82758731f76" containerID="0719669fed48400ec1ffee0ad0988611190291cfd6751063730d5e6835b052a3" exitCode=0
Sep 10 19:29:49.950412 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:49.949721    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b" event={"ID":"1a02f4a9-bd86-4658-aa0a-a82758731f76","Type":"ContainerDied","Data":"0719669fed48400ec1ffee0ad0988611190291cfd6751063730d5e6835b052a3"}
Sep 10 19:29:49.950412 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:49.950209    2564 scope.go:117] "RemoveContainer" containerID="0719669fed48400ec1ffee0ad0988611190291cfd6751063730d5e6835b052a3"
Sep 10 19:29:49.951434 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:49.951392    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:29:49.951554 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:49.951482    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:29:49.951554 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:49.951516    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:29:49.952021 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:49.951998    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns" event={"ID":"050f7885-379b-4256-b25f-4e8f437dc8b9","Type":"ContainerStarted","Data":"655db90c1d1c67d141aa28f9bb275e632fb61a9c0d0cc778aea1711bb0d92afd"}
Sep 10 19:29:50.609998 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:50.609965    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns"
Sep 10 19:29:50.609998 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:50.609997    2564 patch_prober.go:28] interesting pod/e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Sep 10 19:29:50.610217 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:50.610046    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns" podUID="050f7885-379b-4256-b25f-4e8f437dc8b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 10 19:29:50.959024 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:50.958935    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b" event={"ID":"1a02f4a9-bd86-4658-aa0a-a82758731f76","Type":"ContainerStarted","Data":"b8d0baf5c584f74e1aa02444d6ba7138b85ee3396c1d248b1fe5bbed846f5e94"}
Sep 10 19:29:51.609779 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:51.609731    2564 patch_prober.go:28] interesting pod/e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Sep 10 19:29:51.609992 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:51.609824    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns" podUID="050f7885-379b-4256-b25f-4e8f437dc8b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 10 19:29:51.813153 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:51.813106    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b"
Sep 10 19:29:51.813370 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:51.813348    2564 patch_prober.go:28] interesting pod/e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Sep 10 19:29:51.813443 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:51.813406    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b" podUID="1a02f4a9-bd86-4658-aa0a-a82758731f76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Sep 10 19:29:52.059801 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:52.059768    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:52.609732 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:52.609685    2564 patch_prober.go:28] interesting pod/e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Sep 10 19:29:52.610023 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:52.609993    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns" podUID="050f7885-379b-4256-b25f-4e8f437dc8b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 10 19:29:52.813163 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:52.813116    2564 patch_prober.go:28] interesting pod/e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Sep 10 19:29:52.813367 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:52.813194    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b" podUID="1a02f4a9-bd86-4658-aa0a-a82758731f76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Sep 10 19:29:52.982296 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:52.982196    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:53.045467 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:53.045429    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:53.125636 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:53.125560    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:53.221075 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:53.221046    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:53.372956 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:53.372900    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:53.472501 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:53.472447    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:53.609664 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:53.609596    2564 patch_prober.go:28] interesting pod/e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Sep 10 19:29:53.609872 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:53.609691    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns" podUID="050f7885-379b-4256-b25f-4e8f437dc8b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 10 19:29:53.812987 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:53.812942    2564 patch_prober.go:28] interesting pod/e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Sep 10 19:29:53.813200 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:53.813013    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b" podUID="1a02f4a9-bd86-4658-aa0a-a82758731f76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Sep 10 19:29:54.274054 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:54.274010    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:54.610165 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:54.610061    2564 patch_prober.go:28] interesting pod/e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Sep 10 19:29:54.610443 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:54.610409    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns" podUID="050f7885-379b-4256-b25f-4e8f437dc8b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 10 19:29:54.652964 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:54.652855    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:54.812671 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:54.812609    2564 patch_prober.go:28] interesting pod/e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Sep 10 19:29:54.812962 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:54.812929    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b" podUID="1a02f4a9-bd86-4658-aa0a-a82758731f76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Sep 10 19:29:55.251404 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:55.251358    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:55.614098 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:55.614018    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns"
Sep 10 19:29:55.614456 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:55.614305    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns"
Sep 10 19:29:55.615134 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:55.615113    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns"
Sep 10 19:29:55.813375 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:55.813336    2564 patch_prober.go:28] interesting pod/e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Sep 10 19:29:55.813570 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:55.813398    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b" podUID="1a02f4a9-bd86-4658-aa0a-a82758731f76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Sep 10 19:29:55.999731 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:55.999686    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:56.057083 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:56.057049    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:56.156856 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:56.156819    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:56.495212 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:56.495117    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:56.614984 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:56.614947    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-e7e4c3-7c7ff4c845-rtghl"]
Sep 10 19:29:56.618887 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:56.618869    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-e7e4c3-7c7ff4c845-rtghl"
Sep 10 19:29:56.621099 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:56.621077    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-e7e4c3\""
Sep 10 19:29:56.621211 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:56.621140    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-e7e4c3-dockercfg-vwmm5\""
Sep 10 19:29:56.629054 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:56.629031    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-e7e4c3-7c7ff4c845-rtghl"]
Sep 10 19:29:56.704355 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:56.704316    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b338b98-3519-46ae-9fb9-9b594ab5cd86-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-e7e4c3-7c7ff4c845-rtghl\" (UID: \"3b338b98-3519-46ae-9fb9-9b594ab5cd86\") " pod="openshift-ingress/payload-processing-e2e-inf-b-e7e4c3-7c7ff4c845-rtghl"
Sep 10 19:29:56.704583 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:56.704434    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75m7v\" (UniqueName: \"kubernetes.io/projected/3b338b98-3519-46ae-9fb9-9b594ab5cd86-kube-api-access-75m7v\") pod \"payload-processing-e2e-inf-b-e7e4c3-7c7ff4c845-rtghl\" (UID: \"3b338b98-3519-46ae-9fb9-9b594ab5cd86\") " pod="openshift-ingress/payload-processing-e2e-inf-b-e7e4c3-7c7ff4c845-rtghl"
Sep 10 19:29:56.713824 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:56.713788    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:56.805506 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:56.805467    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75m7v\" (UniqueName: \"kubernetes.io/projected/3b338b98-3519-46ae-9fb9-9b594ab5cd86-kube-api-access-75m7v\") pod \"payload-processing-e2e-inf-b-e7e4c3-7c7ff4c845-rtghl\" (UID: \"3b338b98-3519-46ae-9fb9-9b594ab5cd86\") " pod="openshift-ingress/payload-processing-e2e-inf-b-e7e4c3-7c7ff4c845-rtghl"
Sep 10 19:29:56.805718 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:56.805575    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b338b98-3519-46ae-9fb9-9b594ab5cd86-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-e7e4c3-7c7ff4c845-rtghl\" (UID: \"3b338b98-3519-46ae-9fb9-9b594ab5cd86\") " pod="openshift-ingress/payload-processing-e2e-inf-b-e7e4c3-7c7ff4c845-rtghl"
Sep 10 19:29:56.806284 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:56.806261    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b338b98-3519-46ae-9fb9-9b594ab5cd86-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-e7e4c3-7c7ff4c845-rtghl\" (UID: \"3b338b98-3519-46ae-9fb9-9b594ab5cd86\") " pod="openshift-ingress/payload-processing-e2e-inf-b-e7e4c3-7c7ff4c845-rtghl"
Sep 10 19:29:56.815224 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:56.815186    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-75m7v\" (UniqueName: \"kubernetes.io/projected/3b338b98-3519-46ae-9fb9-9b594ab5cd86-kube-api-access-75m7v\") pod \"payload-processing-e2e-inf-b-e7e4c3-7c7ff4c845-rtghl\" (UID: \"3b338b98-3519-46ae-9fb9-9b594ab5cd86\") " pod="openshift-ingress/payload-processing-e2e-inf-b-e7e4c3-7c7ff4c845-rtghl"
Sep 10 19:29:56.923949 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:56.923909    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:56.930399 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:56.930374    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-e7e4c3-7c7ff4c845-rtghl"
Sep 10 19:29:57.060933 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:57.059686    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b"
Sep 10 19:29:57.060933 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:57.060765    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b"
Sep 10 19:29:57.062099 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:57.062048    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b"
Sep 10 19:29:57.224964 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:57.224937    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-e7e4c3-7c7ff4c845-rtghl"]
Sep 10 19:29:57.226814 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:29:57.226781    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b338b98_3519_46ae_9fb9_9b594ab5cd86.slice/crio-4e4453eb33dbd13137239402dd594d0eac68a49ddea22b5518ba78df4a3946b3 WatchSource:0}: Error finding container 4e4453eb33dbd13137239402dd594d0eac68a49ddea22b5518ba78df4a3946b3: Status 404 returned error can't find the container with id 4e4453eb33dbd13137239402dd594d0eac68a49ddea22b5518ba78df4a3946b3
Sep 10 19:29:57.870205 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:57.870161    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:58.029954 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:58.029908    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-e7e4c3-7c7ff4c845-rtghl" event={"ID":"3b338b98-3519-46ae-9fb9-9b594ab5cd86","Type":"ContainerStarted","Data":"4a1f104b04548cfc551b7915dfdd6574d258c20376515550c332070d3f422ba5"}
Sep 10 19:29:58.030129 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:58.030002    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-e7e4c3-7c7ff4c845-rtghl" event={"ID":"3b338b98-3519-46ae-9fb9-9b594ab5cd86","Type":"ContainerStarted","Data":"4e4453eb33dbd13137239402dd594d0eac68a49ddea22b5518ba78df4a3946b3"}
Sep 10 19:29:58.030129 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:58.030038    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-e7e4c3-7c7ff4c845-rtghl"
Sep 10 19:29:58.048788 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:29:58.048734    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-e7e4c3-7c7ff4c845-rtghl" podStartSLOduration=2.048718547 podStartE2EDuration="2.048718547s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:29:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:29:58.046502926 +0000 UTC m=+853.647410165" watchObservedRunningTime="2026-09-10 19:29:58.048718547 +0000 UTC m=+853.649625786"
Sep 10 19:30:00.051844 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:00.051814    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:00.139777 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:00.139739    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29817810-svwf6"]
Sep 10 19:30:00.140016 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:00.139981    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:00.143685 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:00.143665    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817810-svwf6"
Sep 10 19:30:00.146065 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:00.146043    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tjmqn\""
Sep 10 19:30:00.147283 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:00.147262    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-5352fd-29817810-2866b"]
Sep 10 19:30:00.151453 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:00.151427    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-e7fa2c81-29817810-g8srk"]
Sep 10 19:30:00.152143 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:00.152118    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-5352fd-29817810-2866b"
Sep 10 19:30:00.155537 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:00.155517    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29817810-svwf6"]
Sep 10 19:30:00.155714 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:00.155613    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-e7fa2c81-29817810-g8srk"
Sep 10 19:30:00.159416 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:00.159391    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817810-svwf6"
Sep 10 19:30:00.160858 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:00.160827    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-e7fa2c81-29817810-g8srk"]
Sep 10 19:30:00.170160 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:00.170126    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-5352fd-29817810-2866b"]
Sep 10 19:30:00.174689 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:00.174268    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-5352fd-29817810-2866b"
Sep 10 19:30:00.230417 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:00.230201    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-e7fa2c81-29817810-g8srk"
Sep 10 19:30:00.478708 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:00.478676    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-5352fd-29817810-2866b"]
Sep 10 19:30:00.480212 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:30:00.480182    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd06d6e1_9456_427b_951b_93aeec4025d4.slice/crio-4a244f75e660a97ff9d9e92c3b34774c3b9b8cf2bb2c6ebe1bfef7e6dbab0df3 WatchSource:0}: Error finding container 4a244f75e660a97ff9d9e92c3b34774c3b9b8cf2bb2c6ebe1bfef7e6dbab0df3: Status 404 returned error can't find the container with id 4a244f75e660a97ff9d9e92c3b34774c3b9b8cf2bb2c6ebe1bfef7e6dbab0df3
Sep 10 19:30:00.496681 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:00.496658    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29817810-svwf6"]
Sep 10 19:30:00.498168 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:30:00.498140    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50e21465_bf07_4a6c_89f0_b3214b0b62f9.slice/crio-6935ca804017367de3e228db05063d490bb6f0f320a6188ca03ecc6166aceb42 WatchSource:0}: Error finding container 6935ca804017367de3e228db05063d490bb6f0f320a6188ca03ecc6166aceb42: Status 404 returned error can't find the container with id 6935ca804017367de3e228db05063d490bb6f0f320a6188ca03ecc6166aceb42
Sep 10 19:30:00.541085 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:00.541035    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-e7fa2c81-29817810-g8srk"]
Sep 10 19:30:00.543805 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:30:00.543771    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod301c8b30_9c95_43af_bd65_574e8dbb1d91.slice/crio-b8849f9d179203c1daf65ccf3cc6ff69f1f27355b9e1b24ce7514c733ad1ad58 WatchSource:0}: Error finding container b8849f9d179203c1daf65ccf3cc6ff69f1f27355b9e1b24ce7514c733ad1ad58: Status 404 returned error can't find the container with id b8849f9d179203c1daf65ccf3cc6ff69f1f27355b9e1b24ce7514c733ad1ad58
Sep 10 19:30:01.047872 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:01.047838    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-e7fa2c81-29817810-g8srk" event={"ID":"301c8b30-9c95-43af-bd65-574e8dbb1d91","Type":"ContainerStarted","Data":"b8849f9d179203c1daf65ccf3cc6ff69f1f27355b9e1b24ce7514c733ad1ad58"}
Sep 10 19:30:01.049085 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:01.049053    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-5352fd-29817810-2866b" event={"ID":"fd06d6e1-9456-427b-951b-93aeec4025d4","Type":"ContainerStarted","Data":"4a244f75e660a97ff9d9e92c3b34774c3b9b8cf2bb2c6ebe1bfef7e6dbab0df3"}
Sep 10 19:30:01.050255 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:01.050231    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817810-svwf6" event={"ID":"50e21465-bf07-4a6c-89f0-b3214b0b62f9","Type":"ContainerStarted","Data":"6935ca804017367de3e228db05063d490bb6f0f320a6188ca03ecc6166aceb42"}
Sep 10 19:30:01.135375 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:01.135332    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:04.074948 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:04.074909    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817810-svwf6" event={"ID":"50e21465-bf07-4a6c-89f0-b3214b0b62f9","Type":"ContainerStarted","Data":"1e3a9e1aab855411302714f94af77ce8bac0d0b84328327f568187dc19702194"}
Sep 10 19:30:04.076415 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:04.076390    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-e7fa2c81-29817810-g8srk" event={"ID":"301c8b30-9c95-43af-bd65-574e8dbb1d91","Type":"ContainerStarted","Data":"df8bd430fdd5a8eea7dc9046407c8450016f1dedb02624ec6439545d988683ed"}
Sep 10 19:30:04.077867 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:04.077847    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-5352fd-29817810-2866b" event={"ID":"fd06d6e1-9456-427b-951b-93aeec4025d4","Type":"ContainerStarted","Data":"bbac273b662039a55e874dc17dd0b9a44b88a61bf06ae7f5b5b17dfeb9bc9d0f"}
Sep 10 19:30:04.097483 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:04.097427    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817810-svwf6" podStartSLOduration=1.39352947 podStartE2EDuration="4.097412207s" totalImagesPullingTime="2.703882737s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:30:04.096206341 +0000 UTC m=+859.697113606" watchObservedRunningTime="2026-09-10 19:30:04.097412207 +0000 UTC m=+859.698319446"
Sep 10 19:30:04.112246 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:04.112194    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-5352fd-29817810-2866b" podStartSLOduration=1.454107879 podStartE2EDuration="4.112175324s" totalImagesPullingTime="2.658067445s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:30:04.110652628 +0000 UTC m=+859.711559864" watchObservedRunningTime="2026-09-10 19:30:04.112175324 +0000 UTC m=+859.713082564"
Sep 10 19:30:05.086034 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:05.085991    2564 generic.go:356] "Generic (PLEG): container finished" podID="301c8b30-9c95-43af-bd65-574e8dbb1d91" containerID="df8bd430fdd5a8eea7dc9046407c8450016f1dedb02624ec6439545d988683ed" exitCode=0
Sep 10 19:30:05.086526 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:05.086066    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-e7fa2c81-29817810-g8srk" event={"ID":"301c8b30-9c95-43af-bd65-574e8dbb1d91","Type":"ContainerDied","Data":"df8bd430fdd5a8eea7dc9046407c8450016f1dedb02624ec6439545d988683ed"}
Sep 10 19:30:05.087859 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:05.087831    2564 generic.go:356] "Generic (PLEG): container finished" podID="fd06d6e1-9456-427b-951b-93aeec4025d4" containerID="bbac273b662039a55e874dc17dd0b9a44b88a61bf06ae7f5b5b17dfeb9bc9d0f" exitCode=0
Sep 10 19:30:05.087989 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:05.087939    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-5352fd-29817810-2866b" event={"ID":"fd06d6e1-9456-427b-951b-93aeec4025d4","Type":"ContainerDied","Data":"bbac273b662039a55e874dc17dd0b9a44b88a61bf06ae7f5b5b17dfeb9bc9d0f"}
Sep 10 19:30:05.089549 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:05.089511    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:05.089698 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:05.089670    2564 generic.go:356] "Generic (PLEG): container finished" podID="50e21465-bf07-4a6c-89f0-b3214b0b62f9" containerID="1e3a9e1aab855411302714f94af77ce8bac0d0b84328327f568187dc19702194" exitCode=0
Sep 10 19:30:05.089811 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:05.089794    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817810-svwf6" event={"ID":"50e21465-bf07-4a6c-89f0-b3214b0b62f9","Type":"ContainerDied","Data":"1e3a9e1aab855411302714f94af77ce8bac0d0b84328327f568187dc19702194"}
Sep 10 19:30:05.193405 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:05.193367    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:06.256980 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:06.256948    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-e7fa2c81-29817810-g8srk"
Sep 10 19:30:06.300732 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:06.300701    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-5352fd-29817810-2866b"
Sep 10 19:30:06.303830 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:06.303809    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817810-svwf6"
Sep 10 19:30:07.101377 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:07.101346    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-e7fa2c81-29817810-g8srk"
Sep 10 19:30:07.101572 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:07.101345    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-e7fa2c81-29817810-g8srk" event={"ID":"301c8b30-9c95-43af-bd65-574e8dbb1d91","Type":"ContainerDied","Data":"b8849f9d179203c1daf65ccf3cc6ff69f1f27355b9e1b24ce7514c733ad1ad58"}
Sep 10 19:30:07.101572 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:07.101487    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b8849f9d179203c1daf65ccf3cc6ff69f1f27355b9e1b24ce7514c733ad1ad58"
Sep 10 19:30:07.102753 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:07.102736    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-5352fd-29817810-2866b"
Sep 10 19:30:07.102877 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:07.102785    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-5352fd-29817810-2866b" event={"ID":"fd06d6e1-9456-427b-951b-93aeec4025d4","Type":"ContainerDied","Data":"4a244f75e660a97ff9d9e92c3b34774c3b9b8cf2bb2c6ebe1bfef7e6dbab0df3"}
Sep 10 19:30:07.102877 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:07.102842    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4a244f75e660a97ff9d9e92c3b34774c3b9b8cf2bb2c6ebe1bfef7e6dbab0df3"
Sep 10 19:30:07.104209 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:07.104191    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817810-svwf6" event={"ID":"50e21465-bf07-4a6c-89f0-b3214b0b62f9","Type":"ContainerDied","Data":"6935ca804017367de3e228db05063d490bb6f0f320a6188ca03ecc6166aceb42"}
Sep 10 19:30:07.104282 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:07.104225    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6935ca804017367de3e228db05063d490bb6f0f320a6188ca03ecc6166aceb42"
Sep 10 19:30:07.104282 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:07.104197    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817810-svwf6"
Sep 10 19:30:09.038200 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:09.038166    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-e7e4c3-7c7ff4c845-rtghl"
Sep 10 19:30:11.625058 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:11.625018    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:12.256089 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:12.256032    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:12.265838 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:12.265806    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-e7fa2c81-29817810-g8srk"]
Sep 10 19:30:12.267990 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:12.267968    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-e7fa2c81-29817810-g8srk"]
Sep 10 19:30:12.296130 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:12.296096    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e7fa2c81-69d97c9fd9-w7ngm"]
Sep 10 19:30:12.296356 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:12.296319    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-e7fa2c81-69d97c9fd9-w7ngm" podUID="905033d9-9b29-4ed6-96f6-c1909587af70" containerName="payload-processing" containerID="cri-o://fc3fcb9ce67f8dfa7621a8b3b72ab88e61cf82752863253fa892c1c388c72549" gracePeriod=30
Sep 10 19:30:12.300723 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:12.300659    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e7fa2c81-849ccf7f78-4zgm7"]
Sep 10 19:30:12.300996 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:12.300963    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-e7fa2c81-849ccf7f78-4zgm7" podUID="b8d1b28c-28fd-43b4-9193-95032ec530a7" containerName="payload-pre-processing" containerID="cri-o://ca9a35e17da73f91d6bbd69cc3c9edc4451b2b4e3783140c0f02cf4c68a0b12a" gracePeriod=30
Sep 10 19:30:12.344679 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:12.344644    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:12.411038 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:12.410990    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:12.476211 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:12.476181    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:12.531830 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:12.531779    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:12.596588 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:12.596492    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-e7fa2c81-69d97c9fd9-w7ngm"
Sep 10 19:30:12.602694 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:12.602666    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-e7fa2c81-849ccf7f78-4zgm7"
Sep 10 19:30:12.628279 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:12.628229    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:12.641384 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:12.637335    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns"]
Sep 10 19:30:12.641384 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:12.637596    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns" podUID="050f7885-379b-4256-b25f-4e8f437dc8b9" containerName="istio-proxy" containerID="cri-o://655db90c1d1c67d141aa28f9bb275e632fb61a9c0d0cc778aea1711bb0d92afd" gracePeriod=30
Sep 10 19:30:12.680132 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:12.680099    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hsr8w\" (UniqueName: \"kubernetes.io/projected/905033d9-9b29-4ed6-96f6-c1909587af70-kube-api-access-hsr8w\") pod \"905033d9-9b29-4ed6-96f6-c1909587af70\" (UID: \"905033d9-9b29-4ed6-96f6-c1909587af70\") "
Sep 10 19:30:12.680318 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:12.680234    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/905033d9-9b29-4ed6-96f6-c1909587af70-plugins-config-volume\") pod \"905033d9-9b29-4ed6-96f6-c1909587af70\" (UID: \"905033d9-9b29-4ed6-96f6-c1909587af70\") "
Sep 10 19:30:12.680576 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:12.680551    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/905033d9-9b29-4ed6-96f6-c1909587af70-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "905033d9-9b29-4ed6-96f6-c1909587af70" (UID: "905033d9-9b29-4ed6-96f6-c1909587af70"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:30:12.682447 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:12.682414    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/905033d9-9b29-4ed6-96f6-c1909587af70-kube-api-access-hsr8w" (OuterVolumeSpecName: "kube-api-access-hsr8w") pod "905033d9-9b29-4ed6-96f6-c1909587af70" (UID: "905033d9-9b29-4ed6-96f6-c1909587af70"). InnerVolumeSpecName "kube-api-access-hsr8w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:30:12.780914 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:12.780828    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b9n9l\" (UniqueName: \"kubernetes.io/projected/b8d1b28c-28fd-43b4-9193-95032ec530a7-kube-api-access-b9n9l\") pod \"b8d1b28c-28fd-43b4-9193-95032ec530a7\" (UID: \"b8d1b28c-28fd-43b4-9193-95032ec530a7\") "
Sep 10 19:30:12.781063 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:12.780943    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8d1b28c-28fd-43b4-9193-95032ec530a7-plugins-config-volume\") pod \"b8d1b28c-28fd-43b4-9193-95032ec530a7\" (UID: \"b8d1b28c-28fd-43b4-9193-95032ec530a7\") "
Sep 10 19:30:12.781149 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:12.781137    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/905033d9-9b29-4ed6-96f6-c1909587af70-plugins-config-volume\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:30:12.781190 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:12.781152    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hsr8w\" (UniqueName: \"kubernetes.io/projected/905033d9-9b29-4ed6-96f6-c1909587af70-kube-api-access-hsr8w\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:30:12.781267 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:12.781242    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8d1b28c-28fd-43b4-9193-95032ec530a7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b8d1b28c-28fd-43b4-9193-95032ec530a7" (UID: "b8d1b28c-28fd-43b4-9193-95032ec530a7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:30:12.782916 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:12.782896    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8d1b28c-28fd-43b4-9193-95032ec530a7-kube-api-access-b9n9l" (OuterVolumeSpecName: "kube-api-access-b9n9l") pod "b8d1b28c-28fd-43b4-9193-95032ec530a7" (UID: "b8d1b28c-28fd-43b4-9193-95032ec530a7"). InnerVolumeSpecName "kube-api-access-b9n9l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:30:12.881993 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:12.881956    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8d1b28c-28fd-43b4-9193-95032ec530a7-plugins-config-volume\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:30:12.881993 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:12.881988    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b9n9l\" (UniqueName: \"kubernetes.io/projected/b8d1b28c-28fd-43b4-9193-95032ec530a7-kube-api-access-b9n9l\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:30:12.950375 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:12.950343    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="301c8b30-9c95-43af-bd65-574e8dbb1d91" path="/var/lib/kubelet/pods/301c8b30-9c95-43af-bd65-574e8dbb1d91/volumes"
Sep 10 19:30:13.118319 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.118223    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:13.139933 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.139898    2564 generic.go:356] "Generic (PLEG): container finished" podID="b8d1b28c-28fd-43b4-9193-95032ec530a7" containerID="ca9a35e17da73f91d6bbd69cc3c9edc4451b2b4e3783140c0f02cf4c68a0b12a" exitCode=0
Sep 10 19:30:13.140131 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.140022    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-e7fa2c81-849ccf7f78-4zgm7" event={"ID":"b8d1b28c-28fd-43b4-9193-95032ec530a7","Type":"ContainerDied","Data":"ca9a35e17da73f91d6bbd69cc3c9edc4451b2b4e3783140c0f02cf4c68a0b12a"}
Sep 10 19:30:13.140131 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.140039    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-e7fa2c81-849ccf7f78-4zgm7"
Sep 10 19:30:13.140131 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.140112    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-e7fa2c81-849ccf7f78-4zgm7" event={"ID":"b8d1b28c-28fd-43b4-9193-95032ec530a7","Type":"ContainerDied","Data":"a591d71d69cb45f0872f24437670b3fcb8e14ec91e722ed9f7890d452104b53f"}
Sep 10 19:30:13.140309 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.140141    2564 scope.go:117] "RemoveContainer" containerID="ca9a35e17da73f91d6bbd69cc3c9edc4451b2b4e3783140c0f02cf4c68a0b12a"
Sep 10 19:30:13.141804 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.141777    2564 generic.go:356] "Generic (PLEG): container finished" podID="905033d9-9b29-4ed6-96f6-c1909587af70" containerID="fc3fcb9ce67f8dfa7621a8b3b72ab88e61cf82752863253fa892c1c388c72549" exitCode=0
Sep 10 19:30:13.141915 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.141829    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-e7fa2c81-69d97c9fd9-w7ngm"
Sep 10 19:30:13.141915 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.141833    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-e7fa2c81-69d97c9fd9-w7ngm" event={"ID":"905033d9-9b29-4ed6-96f6-c1909587af70","Type":"ContainerDied","Data":"fc3fcb9ce67f8dfa7621a8b3b72ab88e61cf82752863253fa892c1c388c72549"}
Sep 10 19:30:13.141915 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.141885    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-e7fa2c81-69d97c9fd9-w7ngm" event={"ID":"905033d9-9b29-4ed6-96f6-c1909587af70","Type":"ContainerDied","Data":"a4afd83b08254ce303c1677e38368e7f90520730af5f56a1cc58389a7695e080"}
Sep 10 19:30:13.150215 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.150190    2564 scope.go:117] "RemoveContainer" containerID="ca9a35e17da73f91d6bbd69cc3c9edc4451b2b4e3783140c0f02cf4c68a0b12a"
Sep 10 19:30:13.150510 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:30:13.150490    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca9a35e17da73f91d6bbd69cc3c9edc4451b2b4e3783140c0f02cf4c68a0b12a\": container with ID starting with ca9a35e17da73f91d6bbd69cc3c9edc4451b2b4e3783140c0f02cf4c68a0b12a not found: ID does not exist" containerID="ca9a35e17da73f91d6bbd69cc3c9edc4451b2b4e3783140c0f02cf4c68a0b12a"
Sep 10 19:30:13.150592 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.150519    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca9a35e17da73f91d6bbd69cc3c9edc4451b2b4e3783140c0f02cf4c68a0b12a"} err="failed to get container status \"ca9a35e17da73f91d6bbd69cc3c9edc4451b2b4e3783140c0f02cf4c68a0b12a\": rpc error: code = NotFound desc = could not find container \"ca9a35e17da73f91d6bbd69cc3c9edc4451b2b4e3783140c0f02cf4c68a0b12a\": container with ID starting with ca9a35e17da73f91d6bbd69cc3c9edc4451b2b4e3783140c0f02cf4c68a0b12a not found: ID does not exist"
Sep 10 19:30:13.150592 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.150536    2564 scope.go:117] "RemoveContainer" containerID="fc3fcb9ce67f8dfa7621a8b3b72ab88e61cf82752863253fa892c1c388c72549"
Sep 10 19:30:13.159109 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.159088    2564 scope.go:117] "RemoveContainer" containerID="fc3fcb9ce67f8dfa7621a8b3b72ab88e61cf82752863253fa892c1c388c72549"
Sep 10 19:30:13.159405 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:30:13.159382    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc3fcb9ce67f8dfa7621a8b3b72ab88e61cf82752863253fa892c1c388c72549\": container with ID starting with fc3fcb9ce67f8dfa7621a8b3b72ab88e61cf82752863253fa892c1c388c72549 not found: ID does not exist" containerID="fc3fcb9ce67f8dfa7621a8b3b72ab88e61cf82752863253fa892c1c388c72549"
Sep 10 19:30:13.159476 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.159412    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc3fcb9ce67f8dfa7621a8b3b72ab88e61cf82752863253fa892c1c388c72549"} err="failed to get container status \"fc3fcb9ce67f8dfa7621a8b3b72ab88e61cf82752863253fa892c1c388c72549\": rpc error: code = NotFound desc = could not find container \"fc3fcb9ce67f8dfa7621a8b3b72ab88e61cf82752863253fa892c1c388c72549\": container with ID starting with fc3fcb9ce67f8dfa7621a8b3b72ab88e61cf82752863253fa892c1c388c72549 not found: ID does not exist"
Sep 10 19:30:13.166207 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.166181    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e7fa2c81-69d97c9fd9-w7ngm"]
Sep 10 19:30:13.168643 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.168595    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e7fa2c81-69d97c9fd9-w7ngm"]
Sep 10 19:30:13.178931 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.178899    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e7fa2c81-849ccf7f78-4zgm7"]
Sep 10 19:30:13.183538 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.183511    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e7fa2c81-849ccf7f78-4zgm7"]
Sep 10 19:30:13.186396 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.186367    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:13.319300 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.319263    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:13.598064 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.598029    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf"]
Sep 10 19:30:13.598748 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.598726    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd06d6e1-9456-427b-951b-93aeec4025d4" containerName="cleanup"
Sep 10 19:30:13.598748 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.598747    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd06d6e1-9456-427b-951b-93aeec4025d4" containerName="cleanup"
Sep 10 19:30:13.598748 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.598758    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="301c8b30-9c95-43af-bd65-574e8dbb1d91" containerName="cleanup"
Sep 10 19:30:13.598748 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.598765    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="301c8b30-9c95-43af-bd65-574e8dbb1d91" containerName="cleanup"
Sep 10 19:30:13.599069 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.598796    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50e21465-bf07-4a6c-89f0-b3214b0b62f9" containerName="cleanup"
Sep 10 19:30:13.599069 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.598804    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="50e21465-bf07-4a6c-89f0-b3214b0b62f9" containerName="cleanup"
Sep 10 19:30:13.599069 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.598819    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="905033d9-9b29-4ed6-96f6-c1909587af70" containerName="payload-processing"
Sep 10 19:30:13.599069 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.598827    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="905033d9-9b29-4ed6-96f6-c1909587af70" containerName="payload-processing"
Sep 10 19:30:13.599069 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.598871    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8d1b28c-28fd-43b4-9193-95032ec530a7" containerName="payload-pre-processing"
Sep 10 19:30:13.599069 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.598880    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8d1b28c-28fd-43b4-9193-95032ec530a7" containerName="payload-pre-processing"
Sep 10 19:30:13.599069 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.598966    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="301c8b30-9c95-43af-bd65-574e8dbb1d91" containerName="cleanup"
Sep 10 19:30:13.599069 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.598979    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd06d6e1-9456-427b-951b-93aeec4025d4" containerName="cleanup"
Sep 10 19:30:13.599069 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.599033    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8d1b28c-28fd-43b4-9193-95032ec530a7" containerName="payload-pre-processing"
Sep 10 19:30:13.599069 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.599044    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="50e21465-bf07-4a6c-89f0-b3214b0b62f9" containerName="cleanup"
Sep 10 19:30:13.599069 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.599051    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="905033d9-9b29-4ed6-96f6-c1909587af70" containerName="payload-processing"
Sep 10 19:30:13.603762 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.603740    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf"
Sep 10 19:30:13.605934 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.605916    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-ac5183cd-gw-openshift-default-dockercfg-tsb8l\""
Sep 10 19:30:13.613733 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.613699    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf"]
Sep 10 19:30:13.628989 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.628958    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:13.690604 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.690026    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-llk8q\" (UniqueName: \"kubernetes.io/projected/22c0b2d9-7b39-43d7-8133-8982866c8130-kube-api-access-llk8q\") pod \"e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf\" (UID: \"22c0b2d9-7b39-43d7-8133-8982866c8130\") " pod="openshift-ingress/e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf"
Sep 10 19:30:13.690604 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.690084    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22c0b2d9-7b39-43d7-8133-8982866c8130-workload-socket\") pod \"e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf\" (UID: \"22c0b2d9-7b39-43d7-8133-8982866c8130\") " pod="openshift-ingress/e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf"
Sep 10 19:30:13.690604 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.690128    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22c0b2d9-7b39-43d7-8133-8982866c8130-istio-podinfo\") pod \"e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf\" (UID: \"22c0b2d9-7b39-43d7-8133-8982866c8130\") " pod="openshift-ingress/e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf"
Sep 10 19:30:13.690604 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.690166    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22c0b2d9-7b39-43d7-8133-8982866c8130-istio-token\") pod \"e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf\" (UID: \"22c0b2d9-7b39-43d7-8133-8982866c8130\") " pod="openshift-ingress/e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf"
Sep 10 19:30:13.690604 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.690201    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22c0b2d9-7b39-43d7-8133-8982866c8130-istio-data\") pod \"e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf\" (UID: \"22c0b2d9-7b39-43d7-8133-8982866c8130\") " pod="openshift-ingress/e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf"
Sep 10 19:30:13.690604 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.690263    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22c0b2d9-7b39-43d7-8133-8982866c8130-istiod-ca-cert\") pod \"e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf\" (UID: \"22c0b2d9-7b39-43d7-8133-8982866c8130\") " pod="openshift-ingress/e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf"
Sep 10 19:30:13.690604 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.690418    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22c0b2d9-7b39-43d7-8133-8982866c8130-istio-envoy\") pod \"e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf\" (UID: \"22c0b2d9-7b39-43d7-8133-8982866c8130\") " pod="openshift-ingress/e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf"
Sep 10 19:30:13.690604 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.690515    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22c0b2d9-7b39-43d7-8133-8982866c8130-workload-certs\") pod \"e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf\" (UID: \"22c0b2d9-7b39-43d7-8133-8982866c8130\") " pod="openshift-ingress/e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf"
Sep 10 19:30:13.690604 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.690612    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22c0b2d9-7b39-43d7-8133-8982866c8130-credential-socket\") pod \"e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf\" (UID: \"22c0b2d9-7b39-43d7-8133-8982866c8130\") " pod="openshift-ingress/e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf"
Sep 10 19:30:13.792136 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.792094    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22c0b2d9-7b39-43d7-8133-8982866c8130-istio-envoy\") pod \"e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf\" (UID: \"22c0b2d9-7b39-43d7-8133-8982866c8130\") " pod="openshift-ingress/e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf"
Sep 10 19:30:13.792136 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.792134    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22c0b2d9-7b39-43d7-8133-8982866c8130-workload-certs\") pod \"e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf\" (UID: \"22c0b2d9-7b39-43d7-8133-8982866c8130\") " pod="openshift-ingress/e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf"
Sep 10 19:30:13.792362 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.792159    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22c0b2d9-7b39-43d7-8133-8982866c8130-credential-socket\") pod \"e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf\" (UID: \"22c0b2d9-7b39-43d7-8133-8982866c8130\") " pod="openshift-ingress/e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf"
Sep 10 19:30:13.792362 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.792196    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llk8q\" (UniqueName: \"kubernetes.io/projected/22c0b2d9-7b39-43d7-8133-8982866c8130-kube-api-access-llk8q\") pod \"e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf\" (UID: \"22c0b2d9-7b39-43d7-8133-8982866c8130\") " pod="openshift-ingress/e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf"
Sep 10 19:30:13.792362 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.792217    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22c0b2d9-7b39-43d7-8133-8982866c8130-workload-socket\") pod \"e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf\" (UID: \"22c0b2d9-7b39-43d7-8133-8982866c8130\") " pod="openshift-ingress/e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf"
Sep 10 19:30:13.792362 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.792243    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22c0b2d9-7b39-43d7-8133-8982866c8130-istio-podinfo\") pod \"e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf\" (UID: \"22c0b2d9-7b39-43d7-8133-8982866c8130\") " pod="openshift-ingress/e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf"
Sep 10 19:30:13.792362 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.792281    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22c0b2d9-7b39-43d7-8133-8982866c8130-istio-token\") pod \"e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf\" (UID: \"22c0b2d9-7b39-43d7-8133-8982866c8130\") " pod="openshift-ingress/e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf"
Sep 10 19:30:13.792672 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.792458    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22c0b2d9-7b39-43d7-8133-8982866c8130-istio-data\") pod \"e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf\" (UID: \"22c0b2d9-7b39-43d7-8133-8982866c8130\") " pod="openshift-ingress/e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf"
Sep 10 19:30:13.792672 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.792604    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22c0b2d9-7b39-43d7-8133-8982866c8130-workload-socket\") pod \"e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf\" (UID: \"22c0b2d9-7b39-43d7-8133-8982866c8130\") " pod="openshift-ingress/e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf"
Sep 10 19:30:13.792803 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.792669    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22c0b2d9-7b39-43d7-8133-8982866c8130-credential-socket\") pod \"e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf\" (UID: \"22c0b2d9-7b39-43d7-8133-8982866c8130\") " pod="openshift-ingress/e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf"
Sep 10 19:30:13.792803 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.792686    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22c0b2d9-7b39-43d7-8133-8982866c8130-istiod-ca-cert\") pod \"e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf\" (UID: \"22c0b2d9-7b39-43d7-8133-8982866c8130\") " pod="openshift-ingress/e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf"
Sep 10 19:30:13.792803 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.792676    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22c0b2d9-7b39-43d7-8133-8982866c8130-workload-certs\") pod \"e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf\" (UID: \"22c0b2d9-7b39-43d7-8133-8982866c8130\") " pod="openshift-ingress/e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf"
Sep 10 19:30:13.792957 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.792809    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22c0b2d9-7b39-43d7-8133-8982866c8130-istio-data\") pod \"e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf\" (UID: \"22c0b2d9-7b39-43d7-8133-8982866c8130\") " pod="openshift-ingress/e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf"
Sep 10 19:30:13.793160 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.793139    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22c0b2d9-7b39-43d7-8133-8982866c8130-istiod-ca-cert\") pod \"e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf\" (UID: \"22c0b2d9-7b39-43d7-8133-8982866c8130\") " pod="openshift-ingress/e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf"
Sep 10 19:30:13.794390 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.794367    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22c0b2d9-7b39-43d7-8133-8982866c8130-istio-envoy\") pod \"e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf\" (UID: \"22c0b2d9-7b39-43d7-8133-8982866c8130\") " pod="openshift-ingress/e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf"
Sep 10 19:30:13.794873 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.794852    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22c0b2d9-7b39-43d7-8133-8982866c8130-istio-podinfo\") pod \"e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf\" (UID: \"22c0b2d9-7b39-43d7-8133-8982866c8130\") " pod="openshift-ingress/e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf"
Sep 10 19:30:13.801229 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.801202    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-llk8q\" (UniqueName: \"kubernetes.io/projected/22c0b2d9-7b39-43d7-8133-8982866c8130-kube-api-access-llk8q\") pod \"e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf\" (UID: \"22c0b2d9-7b39-43d7-8133-8982866c8130\") " pod="openshift-ingress/e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf"
Sep 10 19:30:13.801527 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.801504    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22c0b2d9-7b39-43d7-8133-8982866c8130-istio-token\") pod \"e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf\" (UID: \"22c0b2d9-7b39-43d7-8133-8982866c8130\") " pod="openshift-ingress/e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf"
Sep 10 19:30:13.915706 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:13.915610    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf"
Sep 10 19:30:14.014777 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:14.014731    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:14.192926 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:14.192900    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf"]
Sep 10 19:30:14.195113 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:30:14.195089    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22c0b2d9_7b39_43d7_8133_8982866c8130.slice/crio-a46949ce505d9fe91a0b03a5a0f93e44932032032769c2b3d1ff9a00b65388ff WatchSource:0}: Error finding container a46949ce505d9fe91a0b03a5a0f93e44932032032769c2b3d1ff9a00b65388ff: Status 404 returned error can't find the container with id a46949ce505d9fe91a0b03a5a0f93e44932032032769c2b3d1ff9a00b65388ff
Sep 10 19:30:14.197253 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:14.197219    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:30:14.197331 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:14.197286    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:30:14.197331 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:14.197313    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:30:14.949891 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:14.949859    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="905033d9-9b29-4ed6-96f6-c1909587af70" path="/var/lib/kubelet/pods/905033d9-9b29-4ed6-96f6-c1909587af70/volumes"
Sep 10 19:30:14.950270 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:14.950243    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8d1b28c-28fd-43b4-9193-95032ec530a7" path="/var/lib/kubelet/pods/b8d1b28c-28fd-43b4-9193-95032ec530a7/volumes"
Sep 10 19:30:15.163381 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:15.163338    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf" event={"ID":"22c0b2d9-7b39-43d7-8133-8982866c8130","Type":"ContainerStarted","Data":"f7df2c433a07436c98322458e4cbaf458a33f30cd3813284ad8545a52b6d8498"}
Sep 10 19:30:15.163551 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:15.163409    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf" event={"ID":"22c0b2d9-7b39-43d7-8133-8982866c8130","Type":"ContainerStarted","Data":"a46949ce505d9fe91a0b03a5a0f93e44932032032769c2b3d1ff9a00b65388ff"}
Sep 10 19:30:15.185436 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:15.185386    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf" podStartSLOduration=2.185368637 podStartE2EDuration="2.185368637s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:30:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:30:15.183543985 +0000 UTC m=+870.784451225" watchObservedRunningTime="2026-09-10 19:30:15.185368637 +0000 UTC m=+870.786275878"
Sep 10 19:30:15.915891 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:15.915852    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf"
Sep 10 19:30:15.920949 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:15.920926    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf"
Sep 10 19:30:16.170606 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:16.170515    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf"
Sep 10 19:30:16.171669 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:16.171648    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf"
Sep 10 19:30:16.623821 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:16.623788    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:16.721990 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:16.721953    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:16.871736 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:16.871700    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:16.926843 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:16.926735    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:17.099027 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:17.098986    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:17.474602 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:17.474572    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:17.569939 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:17.569905    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:17.589904 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:17.589870    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-ac5183cd-69cd54b878-r2x2b"]
Sep 10 19:30:17.594593 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:17.594574    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-ac5183cd-69cd54b878-r2x2b"
Sep 10 19:30:17.597006 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:17.596976    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-ac5183cd\""
Sep 10 19:30:17.597145 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:17.597127    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-ac5183cd-dockercfg-9bnsj\""
Sep 10 19:30:17.604578 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:17.604547    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-ac5183cd-69cd54b878-r2x2b"]
Sep 10 19:30:17.733915 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:17.733833    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1f3a04a-7d93-4331-a612-8c580ec5060c-plugins-config-volume\") pod \"payload-processing-e2e-migrate-ac5183cd-69cd54b878-r2x2b\" (UID: \"c1f3a04a-7d93-4331-a612-8c580ec5060c\") " pod="openshift-ingress/payload-processing-e2e-migrate-ac5183cd-69cd54b878-r2x2b"
Sep 10 19:30:17.734071 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:17.733931    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-586dk\" (UniqueName: \"kubernetes.io/projected/c1f3a04a-7d93-4331-a612-8c580ec5060c-kube-api-access-586dk\") pod \"payload-processing-e2e-migrate-ac5183cd-69cd54b878-r2x2b\" (UID: \"c1f3a04a-7d93-4331-a612-8c580ec5060c\") " pod="openshift-ingress/payload-processing-e2e-migrate-ac5183cd-69cd54b878-r2x2b"
Sep 10 19:30:17.835897 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:17.835861    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1f3a04a-7d93-4331-a612-8c580ec5060c-plugins-config-volume\") pod \"payload-processing-e2e-migrate-ac5183cd-69cd54b878-r2x2b\" (UID: \"c1f3a04a-7d93-4331-a612-8c580ec5060c\") " pod="openshift-ingress/payload-processing-e2e-migrate-ac5183cd-69cd54b878-r2x2b"
Sep 10 19:30:17.836675 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:17.836611    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-586dk\" (UniqueName: \"kubernetes.io/projected/c1f3a04a-7d93-4331-a612-8c580ec5060c-kube-api-access-586dk\") pod \"payload-processing-e2e-migrate-ac5183cd-69cd54b878-r2x2b\" (UID: \"c1f3a04a-7d93-4331-a612-8c580ec5060c\") " pod="openshift-ingress/payload-processing-e2e-migrate-ac5183cd-69cd54b878-r2x2b"
Sep 10 19:30:17.838536 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:17.837940    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1f3a04a-7d93-4331-a612-8c580ec5060c-plugins-config-volume\") pod \"payload-processing-e2e-migrate-ac5183cd-69cd54b878-r2x2b\" (UID: \"c1f3a04a-7d93-4331-a612-8c580ec5060c\") " pod="openshift-ingress/payload-processing-e2e-migrate-ac5183cd-69cd54b878-r2x2b"
Sep 10 19:30:17.849948 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:17.849920    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-586dk\" (UniqueName: \"kubernetes.io/projected/c1f3a04a-7d93-4331-a612-8c580ec5060c-kube-api-access-586dk\") pod \"payload-processing-e2e-migrate-ac5183cd-69cd54b878-r2x2b\" (UID: \"c1f3a04a-7d93-4331-a612-8c580ec5060c\") " pod="openshift-ingress/payload-processing-e2e-migrate-ac5183cd-69cd54b878-r2x2b"
Sep 10 19:30:17.895843 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:17.895818    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns"
Sep 10 19:30:17.908833 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:17.908795    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-ac5183cd-69cd54b878-r2x2b"
Sep 10 19:30:17.940771 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:17.937987    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/050f7885-379b-4256-b25f-4e8f437dc8b9-istio-token\") pod \"050f7885-379b-4256-b25f-4e8f437dc8b9\" (UID: \"050f7885-379b-4256-b25f-4e8f437dc8b9\") "
Sep 10 19:30:17.940771 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:17.938040    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/050f7885-379b-4256-b25f-4e8f437dc8b9-istio-envoy\") pod \"050f7885-379b-4256-b25f-4e8f437dc8b9\" (UID: \"050f7885-379b-4256-b25f-4e8f437dc8b9\") "
Sep 10 19:30:17.940771 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:17.938075    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ggjwr\" (UniqueName: \"kubernetes.io/projected/050f7885-379b-4256-b25f-4e8f437dc8b9-kube-api-access-ggjwr\") pod \"050f7885-379b-4256-b25f-4e8f437dc8b9\" (UID: \"050f7885-379b-4256-b25f-4e8f437dc8b9\") "
Sep 10 19:30:17.940771 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:17.938110    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/050f7885-379b-4256-b25f-4e8f437dc8b9-istio-podinfo\") pod \"050f7885-379b-4256-b25f-4e8f437dc8b9\" (UID: \"050f7885-379b-4256-b25f-4e8f437dc8b9\") "
Sep 10 19:30:17.940771 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:17.938158    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/050f7885-379b-4256-b25f-4e8f437dc8b9-credential-socket\") pod \"050f7885-379b-4256-b25f-4e8f437dc8b9\" (UID: \"050f7885-379b-4256-b25f-4e8f437dc8b9\") "
Sep 10 19:30:17.940771 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:17.938201    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/050f7885-379b-4256-b25f-4e8f437dc8b9-istiod-ca-cert\") pod \"050f7885-379b-4256-b25f-4e8f437dc8b9\" (UID: \"050f7885-379b-4256-b25f-4e8f437dc8b9\") "
Sep 10 19:30:17.940771 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:17.938270    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/050f7885-379b-4256-b25f-4e8f437dc8b9-istio-data\") pod \"050f7885-379b-4256-b25f-4e8f437dc8b9\" (UID: \"050f7885-379b-4256-b25f-4e8f437dc8b9\") "
Sep 10 19:30:17.940771 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:17.938298    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/050f7885-379b-4256-b25f-4e8f437dc8b9-workload-socket\") pod \"050f7885-379b-4256-b25f-4e8f437dc8b9\" (UID: \"050f7885-379b-4256-b25f-4e8f437dc8b9\") "
Sep 10 19:30:17.940771 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:17.938326    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/050f7885-379b-4256-b25f-4e8f437dc8b9-workload-certs\") pod \"050f7885-379b-4256-b25f-4e8f437dc8b9\" (UID: \"050f7885-379b-4256-b25f-4e8f437dc8b9\") "
Sep 10 19:30:17.940771 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:17.939665    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/050f7885-379b-4256-b25f-4e8f437dc8b9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "050f7885-379b-4256-b25f-4e8f437dc8b9" (UID: "050f7885-379b-4256-b25f-4e8f437dc8b9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:30:17.940771 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:17.939907    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/050f7885-379b-4256-b25f-4e8f437dc8b9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "050f7885-379b-4256-b25f-4e8f437dc8b9" (UID: "050f7885-379b-4256-b25f-4e8f437dc8b9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:30:17.940771 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:17.940067    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/050f7885-379b-4256-b25f-4e8f437dc8b9-istio-data" (OuterVolumeSpecName: "istio-data") pod "050f7885-379b-4256-b25f-4e8f437dc8b9" (UID: "050f7885-379b-4256-b25f-4e8f437dc8b9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:30:17.940771 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:17.940226    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/050f7885-379b-4256-b25f-4e8f437dc8b9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "050f7885-379b-4256-b25f-4e8f437dc8b9" (UID: "050f7885-379b-4256-b25f-4e8f437dc8b9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:30:17.940771 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:17.940496    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/050f7885-379b-4256-b25f-4e8f437dc8b9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "050f7885-379b-4256-b25f-4e8f437dc8b9" (UID: "050f7885-379b-4256-b25f-4e8f437dc8b9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:30:17.947778 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:17.947492    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/050f7885-379b-4256-b25f-4e8f437dc8b9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "050f7885-379b-4256-b25f-4e8f437dc8b9" (UID: "050f7885-379b-4256-b25f-4e8f437dc8b9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:30:17.949077 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:17.949046    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/050f7885-379b-4256-b25f-4e8f437dc8b9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "050f7885-379b-4256-b25f-4e8f437dc8b9" (UID: "050f7885-379b-4256-b25f-4e8f437dc8b9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 19:30:17.950490 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:17.950443    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/050f7885-379b-4256-b25f-4e8f437dc8b9-kube-api-access-ggjwr" (OuterVolumeSpecName: "kube-api-access-ggjwr") pod "050f7885-379b-4256-b25f-4e8f437dc8b9" (UID: "050f7885-379b-4256-b25f-4e8f437dc8b9"). InnerVolumeSpecName "kube-api-access-ggjwr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:30:17.950672 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:17.950606    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/050f7885-379b-4256-b25f-4e8f437dc8b9-istio-token" (OuterVolumeSpecName: "istio-token") pod "050f7885-379b-4256-b25f-4e8f437dc8b9" (UID: "050f7885-379b-4256-b25f-4e8f437dc8b9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:30:17.960110 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:17.959985    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:18.039950 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:18.039800    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/050f7885-379b-4256-b25f-4e8f437dc8b9-istio-token\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:30:18.039950 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:18.039833    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/050f7885-379b-4256-b25f-4e8f437dc8b9-istio-envoy\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:30:18.039950 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:18.039848    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ggjwr\" (UniqueName: \"kubernetes.io/projected/050f7885-379b-4256-b25f-4e8f437dc8b9-kube-api-access-ggjwr\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:30:18.039950 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:18.039861    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/050f7885-379b-4256-b25f-4e8f437dc8b9-istio-podinfo\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:30:18.039950 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:18.039874    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/050f7885-379b-4256-b25f-4e8f437dc8b9-credential-socket\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:30:18.039950 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:18.039888    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/050f7885-379b-4256-b25f-4e8f437dc8b9-istiod-ca-cert\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:30:18.039950 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:18.039901    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/050f7885-379b-4256-b25f-4e8f437dc8b9-istio-data\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:30:18.039950 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:18.039912    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/050f7885-379b-4256-b25f-4e8f437dc8b9-workload-socket\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:30:18.039950 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:18.039926    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/050f7885-379b-4256-b25f-4e8f437dc8b9-workload-certs\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:30:18.184493 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:18.184460    2564 generic.go:356] "Generic (PLEG): container finished" podID="050f7885-379b-4256-b25f-4e8f437dc8b9" containerID="655db90c1d1c67d141aa28f9bb275e632fb61a9c0d0cc778aea1711bb0d92afd" exitCode=0
Sep 10 19:30:18.184646 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:18.184523    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns"
Sep 10 19:30:18.184646 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:18.184542    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns" event={"ID":"050f7885-379b-4256-b25f-4e8f437dc8b9","Type":"ContainerDied","Data":"655db90c1d1c67d141aa28f9bb275e632fb61a9c0d0cc778aea1711bb0d92afd"}
Sep 10 19:30:18.184646 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:18.184601    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns" event={"ID":"050f7885-379b-4256-b25f-4e8f437dc8b9","Type":"ContainerDied","Data":"b591a370bcdb81c6e58565bf39a5af254e872d083767f53da645c91eccf287a8"}
Sep 10 19:30:18.184646 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:18.184644    2564 scope.go:117] "RemoveContainer" containerID="655db90c1d1c67d141aa28f9bb275e632fb61a9c0d0cc778aea1711bb0d92afd"
Sep 10 19:30:18.193586 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:18.193569    2564 scope.go:117] "RemoveContainer" containerID="687f1dbc8dc4e59fdd35fc10fbf68541b05d56e93b354369310f5d4d63279cad"
Sep 10 19:30:18.198992 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:18.198850    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-ac5183cd-69cd54b878-r2x2b"]
Sep 10 19:30:18.201110 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:30:18.201083    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1f3a04a_7d93_4331_a612_8c580ec5060c.slice/crio-7abb53f21358c0a662669bede0caf630c02198c9a2323ea7f6d349940023c0e3 WatchSource:0}: Error finding container 7abb53f21358c0a662669bede0caf630c02198c9a2323ea7f6d349940023c0e3: Status 404 returned error can't find the container with id 7abb53f21358c0a662669bede0caf630c02198c9a2323ea7f6d349940023c0e3
Sep 10 19:30:18.204084 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:18.204065    2564 scope.go:117] "RemoveContainer" containerID="655db90c1d1c67d141aa28f9bb275e632fb61a9c0d0cc778aea1711bb0d92afd"
Sep 10 19:30:18.204343 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:30:18.204318    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"655db90c1d1c67d141aa28f9bb275e632fb61a9c0d0cc778aea1711bb0d92afd\": container with ID starting with 655db90c1d1c67d141aa28f9bb275e632fb61a9c0d0cc778aea1711bb0d92afd not found: ID does not exist" containerID="655db90c1d1c67d141aa28f9bb275e632fb61a9c0d0cc778aea1711bb0d92afd"
Sep 10 19:30:18.204441 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:18.204351    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"655db90c1d1c67d141aa28f9bb275e632fb61a9c0d0cc778aea1711bb0d92afd"} err="failed to get container status \"655db90c1d1c67d141aa28f9bb275e632fb61a9c0d0cc778aea1711bb0d92afd\": rpc error: code = NotFound desc = could not find container \"655db90c1d1c67d141aa28f9bb275e632fb61a9c0d0cc778aea1711bb0d92afd\": container with ID starting with 655db90c1d1c67d141aa28f9bb275e632fb61a9c0d0cc778aea1711bb0d92afd not found: ID does not exist"
Sep 10 19:30:18.204441 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:18.204369    2564 scope.go:117] "RemoveContainer" containerID="687f1dbc8dc4e59fdd35fc10fbf68541b05d56e93b354369310f5d4d63279cad"
Sep 10 19:30:18.204664 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:30:18.204614    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"687f1dbc8dc4e59fdd35fc10fbf68541b05d56e93b354369310f5d4d63279cad\": container with ID starting with 687f1dbc8dc4e59fdd35fc10fbf68541b05d56e93b354369310f5d4d63279cad not found: ID does not exist" containerID="687f1dbc8dc4e59fdd35fc10fbf68541b05d56e93b354369310f5d4d63279cad"
Sep 10 19:30:18.204714 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:18.204673    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"687f1dbc8dc4e59fdd35fc10fbf68541b05d56e93b354369310f5d4d63279cad"} err="failed to get container status \"687f1dbc8dc4e59fdd35fc10fbf68541b05d56e93b354369310f5d4d63279cad\": rpc error: code = NotFound desc = could not find container \"687f1dbc8dc4e59fdd35fc10fbf68541b05d56e93b354369310f5d4d63279cad\": container with ID starting with 687f1dbc8dc4e59fdd35fc10fbf68541b05d56e93b354369310f5d4d63279cad not found: ID does not exist"
Sep 10 19:30:18.208141 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:18.208119    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns"]
Sep 10 19:30:18.212551 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:18.212527    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-e7fa2c81-openshift-default-75dcc8d8cd-xpkns"]
Sep 10 19:30:18.846642 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:18.846601    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:18.950552 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:18.950476    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="050f7885-379b-4256-b25f-4e8f437dc8b9" path="/var/lib/kubelet/pods/050f7885-379b-4256-b25f-4e8f437dc8b9/volumes"
Sep 10 19:30:19.191955 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:19.191849    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-ac5183cd-69cd54b878-r2x2b" event={"ID":"c1f3a04a-7d93-4331-a612-8c580ec5060c","Type":"ContainerStarted","Data":"20bae07ad951f796112b90fd830419c4e1eedc1a259af70e5ef9bcdc89271f06"}
Sep 10 19:30:19.191955 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:19.191910    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-ac5183cd-69cd54b878-r2x2b" event={"ID":"c1f3a04a-7d93-4331-a612-8c580ec5060c","Type":"ContainerStarted","Data":"7abb53f21358c0a662669bede0caf630c02198c9a2323ea7f6d349940023c0e3"}
Sep 10 19:30:19.191955 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:19.191938    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-ac5183cd-69cd54b878-r2x2b"
Sep 10 19:30:19.209952 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:19.209841    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-ac5183cd-69cd54b878-r2x2b" podStartSLOduration=2.209822114 podStartE2EDuration="2.209822114s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:30:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:30:19.208318937 +0000 UTC m=+874.809226179" watchObservedRunningTime="2026-09-10 19:30:19.209822114 +0000 UTC m=+874.810729408"
Sep 10 19:30:19.604816 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:19.604777    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:22.378314 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:22.378270    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:23.243805 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:23.243764    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:30.201116 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:30.201080    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-ac5183cd-69cd54b878-r2x2b"
Sep 10 19:30:37.299583 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:37.299497    2564 generic.go:356] "Generic (PLEG): container finished" podID="22c0b2d9-7b39-43d7-8133-8982866c8130" containerID="f7df2c433a07436c98322458e4cbaf458a33f30cd3813284ad8545a52b6d8498" exitCode=0
Sep 10 19:30:37.299583 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:37.299569    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf" event={"ID":"22c0b2d9-7b39-43d7-8133-8982866c8130","Type":"ContainerDied","Data":"f7df2c433a07436c98322458e4cbaf458a33f30cd3813284ad8545a52b6d8498"}
Sep 10 19:30:37.300046 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:37.299969    2564 scope.go:117] "RemoveContainer" containerID="f7df2c433a07436c98322458e4cbaf458a33f30cd3813284ad8545a52b6d8498"
Sep 10 19:30:37.301072 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:37.301043    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:30:37.301131 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:37.301110    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:30:37.301168 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:37.301140    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:30:38.025080 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:38.025037    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:38.309901 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:38.309808    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf" event={"ID":"22c0b2d9-7b39-43d7-8133-8982866c8130","Type":"ContainerStarted","Data":"a954197f5ffcce6ddc7b9c93f380dac018c07ea067893b000c5b8253da44f22f"}
Sep 10 19:30:38.916227 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:38.916193    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf"
Sep 10 19:30:38.916472 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:38.916443    2564 patch_prober.go:28] interesting pod/e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Sep 10 19:30:38.916602 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:38.916499    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf" podUID="22c0b2d9-7b39-43d7-8133-8982866c8130" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Sep 10 19:30:39.916820 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:39.916782    2564 patch_prober.go:28] interesting pod/e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Sep 10 19:30:39.917205 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:39.916844    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf" podUID="22c0b2d9-7b39-43d7-8133-8982866c8130" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Sep 10 19:30:40.916143 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:40.916104    2564 patch_prober.go:28] interesting pod/e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Sep 10 19:30:40.916313 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:40.916159    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf" podUID="22c0b2d9-7b39-43d7-8133-8982866c8130" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Sep 10 19:30:41.916880 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:41.916836    2564 patch_prober.go:28] interesting pod/e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Sep 10 19:30:41.917346 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:41.916900    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf" podUID="22c0b2d9-7b39-43d7-8133-8982866c8130" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Sep 10 19:30:42.916782 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:42.916738    2564 patch_prober.go:28] interesting pod/e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Sep 10 19:30:42.916959 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:42.916800    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf" podUID="22c0b2d9-7b39-43d7-8133-8982866c8130" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Sep 10 19:30:43.916709 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:43.916677    2564 patch_prober.go:28] interesting pod/e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Sep 10 19:30:43.916887 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:43.916738    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf" podUID="22c0b2d9-7b39-43d7-8133-8982866c8130" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Sep 10 19:30:44.920843 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:44.920814    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf"
Sep 10 19:30:44.921206 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:44.921080    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf"
Sep 10 19:30:44.921955 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:44.921934    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf"
Sep 10 19:30:47.528283 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:47.528246    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:47.568468 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:47.568435    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-ac5183cd-69cd54b878-r2x2b"]
Sep 10 19:30:47.568735 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:47.568706    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-ac5183cd-69cd54b878-r2x2b" podUID="c1f3a04a-7d93-4331-a612-8c580ec5060c" containerName="payload-processing" containerID="cri-o://20bae07ad951f796112b90fd830419c4e1eedc1a259af70e5ef9bcdc89271f06" gracePeriod=30
Sep 10 19:30:47.630199 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:47.630145    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:47.726178 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:47.726147    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:47.824158 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:47.824129    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:47.831903 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:47.831863    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-ac5183cd-69cd54b878-r2x2b"
Sep 10 19:30:47.860320 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:47.860098    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-586dk\" (UniqueName: \"kubernetes.io/projected/c1f3a04a-7d93-4331-a612-8c580ec5060c-kube-api-access-586dk\") pod \"c1f3a04a-7d93-4331-a612-8c580ec5060c\" (UID: \"c1f3a04a-7d93-4331-a612-8c580ec5060c\") "
Sep 10 19:30:47.860848 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:47.860660    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1f3a04a-7d93-4331-a612-8c580ec5060c-plugins-config-volume\") pod \"c1f3a04a-7d93-4331-a612-8c580ec5060c\" (UID: \"c1f3a04a-7d93-4331-a612-8c580ec5060c\") "
Sep 10 19:30:47.862178 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:47.862152    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1f3a04a-7d93-4331-a612-8c580ec5060c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c1f3a04a-7d93-4331-a612-8c580ec5060c" (UID: "c1f3a04a-7d93-4331-a612-8c580ec5060c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:30:47.870946 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:47.870811    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1f3a04a-7d93-4331-a612-8c580ec5060c-kube-api-access-586dk" (OuterVolumeSpecName: "kube-api-access-586dk") pod "c1f3a04a-7d93-4331-a612-8c580ec5060c" (UID: "c1f3a04a-7d93-4331-a612-8c580ec5060c"). InnerVolumeSpecName "kube-api-access-586dk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:30:47.896286 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:47.896248    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf"]
Sep 10 19:30:47.896569 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:47.896483    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf" podUID="22c0b2d9-7b39-43d7-8133-8982866c8130" containerName="istio-proxy" containerID="cri-o://a954197f5ffcce6ddc7b9c93f380dac018c07ea067893b000c5b8253da44f22f" gracePeriod=30
Sep 10 19:30:47.929761 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:47.929699    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:47.962950 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:47.962904    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-586dk\" (UniqueName: \"kubernetes.io/projected/c1f3a04a-7d93-4331-a612-8c580ec5060c-kube-api-access-586dk\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:30:47.962950 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:47.962946    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1f3a04a-7d93-4331-a612-8c580ec5060c-plugins-config-volume\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:30:48.364078 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:48.364036    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:48.367569 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:48.367536    2564 generic.go:356] "Generic (PLEG): container finished" podID="c1f3a04a-7d93-4331-a612-8c580ec5060c" containerID="20bae07ad951f796112b90fd830419c4e1eedc1a259af70e5ef9bcdc89271f06" exitCode=0
Sep 10 19:30:48.367760 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:48.367593    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-ac5183cd-69cd54b878-r2x2b"
Sep 10 19:30:48.367760 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:48.367616    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-ac5183cd-69cd54b878-r2x2b" event={"ID":"c1f3a04a-7d93-4331-a612-8c580ec5060c","Type":"ContainerDied","Data":"20bae07ad951f796112b90fd830419c4e1eedc1a259af70e5ef9bcdc89271f06"}
Sep 10 19:30:48.367760 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:48.367692    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-ac5183cd-69cd54b878-r2x2b" event={"ID":"c1f3a04a-7d93-4331-a612-8c580ec5060c","Type":"ContainerDied","Data":"7abb53f21358c0a662669bede0caf630c02198c9a2323ea7f6d349940023c0e3"}
Sep 10 19:30:48.367760 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:48.367722    2564 scope.go:117] "RemoveContainer" containerID="20bae07ad951f796112b90fd830419c4e1eedc1a259af70e5ef9bcdc89271f06"
Sep 10 19:30:48.376808 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:48.376788    2564 scope.go:117] "RemoveContainer" containerID="20bae07ad951f796112b90fd830419c4e1eedc1a259af70e5ef9bcdc89271f06"
Sep 10 19:30:48.377098 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:30:48.377080    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20bae07ad951f796112b90fd830419c4e1eedc1a259af70e5ef9bcdc89271f06\": container with ID starting with 20bae07ad951f796112b90fd830419c4e1eedc1a259af70e5ef9bcdc89271f06 not found: ID does not exist" containerID="20bae07ad951f796112b90fd830419c4e1eedc1a259af70e5ef9bcdc89271f06"
Sep 10 19:30:48.377139 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:48.377108    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20bae07ad951f796112b90fd830419c4e1eedc1a259af70e5ef9bcdc89271f06"} err="failed to get container status \"20bae07ad951f796112b90fd830419c4e1eedc1a259af70e5ef9bcdc89271f06\": rpc error: code = NotFound desc = could not find container \"20bae07ad951f796112b90fd830419c4e1eedc1a259af70e5ef9bcdc89271f06\": container with ID starting with 20bae07ad951f796112b90fd830419c4e1eedc1a259af70e5ef9bcdc89271f06 not found: ID does not exist"
Sep 10 19:30:48.389797 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:48.389768    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-ac5183cd-69cd54b878-r2x2b"]
Sep 10 19:30:48.394813 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:48.394791    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-ac5183cd-69cd54b878-r2x2b"]
Sep 10 19:30:48.725827 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:48.725742    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:48.727341 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:48.727319    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz"]
Sep 10 19:30:48.727755 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:48.727742    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="050f7885-379b-4256-b25f-4e8f437dc8b9" containerName="istio-proxy"
Sep 10 19:30:48.727799 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:48.727757    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="050f7885-379b-4256-b25f-4e8f437dc8b9" containerName="istio-proxy"
Sep 10 19:30:48.727799 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:48.727767    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="050f7885-379b-4256-b25f-4e8f437dc8b9" containerName="istio-proxy"
Sep 10 19:30:48.727799 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:48.727788    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="050f7885-379b-4256-b25f-4e8f437dc8b9" containerName="istio-proxy"
Sep 10 19:30:48.727921 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:48.727801    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1f3a04a-7d93-4331-a612-8c580ec5060c" containerName="payload-processing"
Sep 10 19:30:48.727921 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:48.727807    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1f3a04a-7d93-4331-a612-8c580ec5060c" containerName="payload-processing"
Sep 10 19:30:48.727921 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:48.727892    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="050f7885-379b-4256-b25f-4e8f437dc8b9" containerName="istio-proxy"
Sep 10 19:30:48.727921 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:48.727901    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="050f7885-379b-4256-b25f-4e8f437dc8b9" containerName="istio-proxy"
Sep 10 19:30:48.727921 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:48.727907    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1f3a04a-7d93-4331-a612-8c580ec5060c" containerName="payload-processing"
Sep 10 19:30:48.732168 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:48.732152    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz"
Sep 10 19:30:48.735032 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:48.735014    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-30f2651a-openshift-default-dockercfg-27bcb\""
Sep 10 19:30:48.744876 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:48.744832    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz"]
Sep 10 19:30:48.769866 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:48.769833    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ba42d44-358e-4101-a5a8-cd3a86d5f513-workload-certs\") pod \"e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz\" (UID: \"2ba42d44-358e-4101-a5a8-cd3a86d5f513\") " pod="openshift-ingress/e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz"
Sep 10 19:30:48.770064 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:48.769874    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ba42d44-358e-4101-a5a8-cd3a86d5f513-istio-data\") pod \"e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz\" (UID: \"2ba42d44-358e-4101-a5a8-cd3a86d5f513\") " pod="openshift-ingress/e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz"
Sep 10 19:30:48.770064 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:48.769907    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ba42d44-358e-4101-a5a8-cd3a86d5f513-istio-envoy\") pod \"e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz\" (UID: \"2ba42d44-358e-4101-a5a8-cd3a86d5f513\") " pod="openshift-ingress/e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz"
Sep 10 19:30:48.770064 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:48.769931    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ba42d44-358e-4101-a5a8-cd3a86d5f513-istiod-ca-cert\") pod \"e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz\" (UID: \"2ba42d44-358e-4101-a5a8-cd3a86d5f513\") " pod="openshift-ingress/e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz"
Sep 10 19:30:48.770064 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:48.769959    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ba42d44-358e-4101-a5a8-cd3a86d5f513-workload-socket\") pod \"e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz\" (UID: \"2ba42d44-358e-4101-a5a8-cd3a86d5f513\") " pod="openshift-ingress/e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz"
Sep 10 19:30:48.770064 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:48.770009    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ba42d44-358e-4101-a5a8-cd3a86d5f513-istio-token\") pod \"e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz\" (UID: \"2ba42d44-358e-4101-a5a8-cd3a86d5f513\") " pod="openshift-ingress/e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz"
Sep 10 19:30:48.770064 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:48.770034    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ba42d44-358e-4101-a5a8-cd3a86d5f513-credential-socket\") pod \"e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz\" (UID: \"2ba42d44-358e-4101-a5a8-cd3a86d5f513\") " pod="openshift-ingress/e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz"
Sep 10 19:30:48.770381 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:48.770093    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ba42d44-358e-4101-a5a8-cd3a86d5f513-istio-podinfo\") pod \"e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz\" (UID: \"2ba42d44-358e-4101-a5a8-cd3a86d5f513\") " pod="openshift-ingress/e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz"
Sep 10 19:30:48.770381 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:48.770125    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2lm9\" (UniqueName: \"kubernetes.io/projected/2ba42d44-358e-4101-a5a8-cd3a86d5f513-kube-api-access-v2lm9\") pod \"e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz\" (UID: \"2ba42d44-358e-4101-a5a8-cd3a86d5f513\") " pod="openshift-ingress/e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz"
Sep 10 19:30:48.871114 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:48.871080    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ba42d44-358e-4101-a5a8-cd3a86d5f513-workload-certs\") pod \"e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz\" (UID: \"2ba42d44-358e-4101-a5a8-cd3a86d5f513\") " pod="openshift-ingress/e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz"
Sep 10 19:30:48.871318 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:48.871121    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ba42d44-358e-4101-a5a8-cd3a86d5f513-istio-data\") pod \"e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz\" (UID: \"2ba42d44-358e-4101-a5a8-cd3a86d5f513\") " pod="openshift-ingress/e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz"
Sep 10 19:30:48.871318 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:48.871146    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ba42d44-358e-4101-a5a8-cd3a86d5f513-istio-envoy\") pod \"e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz\" (UID: \"2ba42d44-358e-4101-a5a8-cd3a86d5f513\") " pod="openshift-ingress/e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz"
Sep 10 19:30:48.871318 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:48.871259    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ba42d44-358e-4101-a5a8-cd3a86d5f513-istiod-ca-cert\") pod \"e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz\" (UID: \"2ba42d44-358e-4101-a5a8-cd3a86d5f513\") " pod="openshift-ingress/e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz"
Sep 10 19:30:48.871318 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:48.871307    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ba42d44-358e-4101-a5a8-cd3a86d5f513-workload-socket\") pod \"e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz\" (UID: \"2ba42d44-358e-4101-a5a8-cd3a86d5f513\") " pod="openshift-ingress/e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz"
Sep 10 19:30:48.871552 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:48.871343    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ba42d44-358e-4101-a5a8-cd3a86d5f513-istio-token\") pod \"e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz\" (UID: \"2ba42d44-358e-4101-a5a8-cd3a86d5f513\") " pod="openshift-ingress/e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz"
Sep 10 19:30:48.871552 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:48.871370    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ba42d44-358e-4101-a5a8-cd3a86d5f513-credential-socket\") pod \"e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz\" (UID: \"2ba42d44-358e-4101-a5a8-cd3a86d5f513\") " pod="openshift-ingress/e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz"
Sep 10 19:30:48.871552 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:48.871446    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ba42d44-358e-4101-a5a8-cd3a86d5f513-istio-podinfo\") pod \"e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz\" (UID: \"2ba42d44-358e-4101-a5a8-cd3a86d5f513\") " pod="openshift-ingress/e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz"
Sep 10 19:30:48.871552 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:48.871486    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2lm9\" (UniqueName: \"kubernetes.io/projected/2ba42d44-358e-4101-a5a8-cd3a86d5f513-kube-api-access-v2lm9\") pod \"e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz\" (UID: \"2ba42d44-358e-4101-a5a8-cd3a86d5f513\") " pod="openshift-ingress/e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz"
Sep 10 19:30:48.871552 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:48.871542    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ba42d44-358e-4101-a5a8-cd3a86d5f513-istio-data\") pod \"e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz\" (UID: \"2ba42d44-358e-4101-a5a8-cd3a86d5f513\") " pod="openshift-ingress/e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz"
Sep 10 19:30:48.871851 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:48.871696    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ba42d44-358e-4101-a5a8-cd3a86d5f513-workload-socket\") pod \"e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz\" (UID: \"2ba42d44-358e-4101-a5a8-cd3a86d5f513\") " pod="openshift-ingress/e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz"
Sep 10 19:30:48.871934 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:48.871916    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ba42d44-358e-4101-a5a8-cd3a86d5f513-workload-certs\") pod \"e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz\" (UID: \"2ba42d44-358e-4101-a5a8-cd3a86d5f513\") " pod="openshift-ingress/e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz"
Sep 10 19:30:48.872048 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:48.872026    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ba42d44-358e-4101-a5a8-cd3a86d5f513-istiod-ca-cert\") pod \"e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz\" (UID: \"2ba42d44-358e-4101-a5a8-cd3a86d5f513\") " pod="openshift-ingress/e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz"
Sep 10 19:30:48.872048 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:48.872040    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ba42d44-358e-4101-a5a8-cd3a86d5f513-credential-socket\") pod \"e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz\" (UID: \"2ba42d44-358e-4101-a5a8-cd3a86d5f513\") " pod="openshift-ingress/e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz"
Sep 10 19:30:48.873466 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:48.873446    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ba42d44-358e-4101-a5a8-cd3a86d5f513-istio-envoy\") pod \"e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz\" (UID: \"2ba42d44-358e-4101-a5a8-cd3a86d5f513\") " pod="openshift-ingress/e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz"
Sep 10 19:30:48.874232 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:48.874211    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ba42d44-358e-4101-a5a8-cd3a86d5f513-istio-podinfo\") pod \"e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz\" (UID: \"2ba42d44-358e-4101-a5a8-cd3a86d5f513\") " pod="openshift-ingress/e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz"
Sep 10 19:30:48.879709 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:48.879687    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2lm9\" (UniqueName: \"kubernetes.io/projected/2ba42d44-358e-4101-a5a8-cd3a86d5f513-kube-api-access-v2lm9\") pod \"e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz\" (UID: \"2ba42d44-358e-4101-a5a8-cd3a86d5f513\") " pod="openshift-ingress/e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz"
Sep 10 19:30:48.879930 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:48.879911    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ba42d44-358e-4101-a5a8-cd3a86d5f513-istio-token\") pod \"e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz\" (UID: \"2ba42d44-358e-4101-a5a8-cd3a86d5f513\") " pod="openshift-ingress/e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz"
Sep 10 19:30:48.898100 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:48.898070    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:48.950435 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:48.950399    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1f3a04a-7d93-4331-a612-8c580ec5060c" path="/var/lib/kubelet/pods/c1f3a04a-7d93-4331-a612-8c580ec5060c/volumes"
Sep 10 19:30:49.043378 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:49.043346    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz"
Sep 10 19:30:49.321386 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:49.321351    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz"]
Sep 10 19:30:49.322526 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:30:49.322497    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ba42d44_358e_4101_a5a8_cd3a86d5f513.slice/crio-d78548ac704a98a0bbb8781a6ac3575767aa5e9534c219686644b5e4e97d3b9f WatchSource:0}: Error finding container d78548ac704a98a0bbb8781a6ac3575767aa5e9534c219686644b5e4e97d3b9f: Status 404 returned error can't find the container with id d78548ac704a98a0bbb8781a6ac3575767aa5e9534c219686644b5e4e97d3b9f
Sep 10 19:30:49.325002 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:49.324962    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:30:49.325079 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:49.325063    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:30:49.325127 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:49.325106    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:30:49.378280 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:49.378246    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz" event={"ID":"2ba42d44-358e-4101-a5a8-cd3a86d5f513","Type":"ContainerStarted","Data":"d78548ac704a98a0bbb8781a6ac3575767aa5e9534c219686644b5e4e97d3b9f"}
Sep 10 19:30:50.385701 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:50.385664    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz" event={"ID":"2ba42d44-358e-4101-a5a8-cd3a86d5f513","Type":"ContainerStarted","Data":"c927fe620e325629d344741220c5e83282d04b9df921e76b02e9ecf5af25c262"}
Sep 10 19:30:50.407819 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:50.407773    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz" podStartSLOduration=2.407754674 podStartE2EDuration="2.407754674s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:30:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:30:50.406402655 +0000 UTC m=+906.007309895" watchObservedRunningTime="2026-09-10 19:30:50.407754674 +0000 UTC m=+906.008661914"
Sep 10 19:30:50.584179 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:50.584140    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:50.684383 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:50.684298    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:51.043791 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:51.043753    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz"
Sep 10 19:30:51.049422 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:51.049396    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz"
Sep 10 19:30:51.391477 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:51.391379    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz"
Sep 10 19:30:51.392367 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:51.392342    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz"
Sep 10 19:30:51.744039 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:51.743948    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:51.841794 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:51.841759    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:52.971038 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:52.971002    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:53.037596 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.037562    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:53.138383 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.138344    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:53.265181 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.265146    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf"
Sep 10 19:30:53.313147 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.313107    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22c0b2d9-7b39-43d7-8133-8982866c8130-workload-certs\") pod \"22c0b2d9-7b39-43d7-8133-8982866c8130\" (UID: \"22c0b2d9-7b39-43d7-8133-8982866c8130\") "
Sep 10 19:30:53.313147 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.313148    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22c0b2d9-7b39-43d7-8133-8982866c8130-credential-socket\") pod \"22c0b2d9-7b39-43d7-8133-8982866c8130\" (UID: \"22c0b2d9-7b39-43d7-8133-8982866c8130\") "
Sep 10 19:30:53.313412 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.313188    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22c0b2d9-7b39-43d7-8133-8982866c8130-istio-podinfo\") pod \"22c0b2d9-7b39-43d7-8133-8982866c8130\" (UID: \"22c0b2d9-7b39-43d7-8133-8982866c8130\") "
Sep 10 19:30:53.313412 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.313223    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22c0b2d9-7b39-43d7-8133-8982866c8130-istio-token\") pod \"22c0b2d9-7b39-43d7-8133-8982866c8130\" (UID: \"22c0b2d9-7b39-43d7-8133-8982866c8130\") "
Sep 10 19:30:53.313412 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.313263    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22c0b2d9-7b39-43d7-8133-8982866c8130-workload-socket\") pod \"22c0b2d9-7b39-43d7-8133-8982866c8130\" (UID: \"22c0b2d9-7b39-43d7-8133-8982866c8130\") "
Sep 10 19:30:53.313412 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.313289    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22c0b2d9-7b39-43d7-8133-8982866c8130-istio-envoy\") pod \"22c0b2d9-7b39-43d7-8133-8982866c8130\" (UID: \"22c0b2d9-7b39-43d7-8133-8982866c8130\") "
Sep 10 19:30:53.313412 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.313383    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22c0b2d9-7b39-43d7-8133-8982866c8130-istiod-ca-cert\") pod \"22c0b2d9-7b39-43d7-8133-8982866c8130\" (UID: \"22c0b2d9-7b39-43d7-8133-8982866c8130\") "
Sep 10 19:30:53.313412 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.313411    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-llk8q\" (UniqueName: \"kubernetes.io/projected/22c0b2d9-7b39-43d7-8133-8982866c8130-kube-api-access-llk8q\") pod \"22c0b2d9-7b39-43d7-8133-8982866c8130\" (UID: \"22c0b2d9-7b39-43d7-8133-8982866c8130\") "
Sep 10 19:30:53.313800 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.313472    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22c0b2d9-7b39-43d7-8133-8982866c8130-istio-data\") pod \"22c0b2d9-7b39-43d7-8133-8982866c8130\" (UID: \"22c0b2d9-7b39-43d7-8133-8982866c8130\") "
Sep 10 19:30:53.313800 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.313405    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22c0b2d9-7b39-43d7-8133-8982866c8130-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "22c0b2d9-7b39-43d7-8133-8982866c8130" (UID: "22c0b2d9-7b39-43d7-8133-8982866c8130"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:30:53.313800 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.313533    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22c0b2d9-7b39-43d7-8133-8982866c8130-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "22c0b2d9-7b39-43d7-8133-8982866c8130" (UID: "22c0b2d9-7b39-43d7-8133-8982866c8130"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:30:53.313968 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.313825    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22c0b2d9-7b39-43d7-8133-8982866c8130-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "22c0b2d9-7b39-43d7-8133-8982866c8130" (UID: "22c0b2d9-7b39-43d7-8133-8982866c8130"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:30:53.314028 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.313992    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22c0b2d9-7b39-43d7-8133-8982866c8130-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "22c0b2d9-7b39-43d7-8133-8982866c8130" (UID: "22c0b2d9-7b39-43d7-8133-8982866c8130"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:30:53.314085 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.314043    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22c0b2d9-7b39-43d7-8133-8982866c8130-workload-certs\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:30:53.314085 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.314062    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22c0b2d9-7b39-43d7-8133-8982866c8130-credential-socket\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:30:53.314085 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.314075    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22c0b2d9-7b39-43d7-8133-8982866c8130-istiod-ca-cert\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:30:53.314721 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.314681    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22c0b2d9-7b39-43d7-8133-8982866c8130-istio-data" (OuterVolumeSpecName: "istio-data") pod "22c0b2d9-7b39-43d7-8133-8982866c8130" (UID: "22c0b2d9-7b39-43d7-8133-8982866c8130"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:30:53.316348 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.316072    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22c0b2d9-7b39-43d7-8133-8982866c8130-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "22c0b2d9-7b39-43d7-8133-8982866c8130" (UID: "22c0b2d9-7b39-43d7-8133-8982866c8130"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:30:53.316348 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.316104    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22c0b2d9-7b39-43d7-8133-8982866c8130-istio-token" (OuterVolumeSpecName: "istio-token") pod "22c0b2d9-7b39-43d7-8133-8982866c8130" (UID: "22c0b2d9-7b39-43d7-8133-8982866c8130"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:30:53.316348 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.316200    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/22c0b2d9-7b39-43d7-8133-8982866c8130-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "22c0b2d9-7b39-43d7-8133-8982866c8130" (UID: "22c0b2d9-7b39-43d7-8133-8982866c8130"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 19:30:53.316587 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.316374    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22c0b2d9-7b39-43d7-8133-8982866c8130-kube-api-access-llk8q" (OuterVolumeSpecName: "kube-api-access-llk8q") pod "22c0b2d9-7b39-43d7-8133-8982866c8130" (UID: "22c0b2d9-7b39-43d7-8133-8982866c8130"). InnerVolumeSpecName "kube-api-access-llk8q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:30:53.402973 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.402940    2564 generic.go:356] "Generic (PLEG): container finished" podID="22c0b2d9-7b39-43d7-8133-8982866c8130" containerID="a954197f5ffcce6ddc7b9c93f380dac018c07ea067893b000c5b8253da44f22f" exitCode=0
Sep 10 19:30:53.403142 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.402985    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf"
Sep 10 19:30:53.403142 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.403019    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf" event={"ID":"22c0b2d9-7b39-43d7-8133-8982866c8130","Type":"ContainerDied","Data":"a954197f5ffcce6ddc7b9c93f380dac018c07ea067893b000c5b8253da44f22f"}
Sep 10 19:30:53.403142 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.403072    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf" event={"ID":"22c0b2d9-7b39-43d7-8133-8982866c8130","Type":"ContainerDied","Data":"a46949ce505d9fe91a0b03a5a0f93e44932032032769c2b3d1ff9a00b65388ff"}
Sep 10 19:30:53.403142 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.403095    2564 scope.go:117] "RemoveContainer" containerID="a954197f5ffcce6ddc7b9c93f380dac018c07ea067893b000c5b8253da44f22f"
Sep 10 19:30:53.412815 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.412762    2564 scope.go:117] "RemoveContainer" containerID="f7df2c433a07436c98322458e4cbaf458a33f30cd3813284ad8545a52b6d8498"
Sep 10 19:30:53.415223 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.415199    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22c0b2d9-7b39-43d7-8133-8982866c8130-istio-podinfo\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:30:53.415317 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.415228    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22c0b2d9-7b39-43d7-8133-8982866c8130-istio-token\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:30:53.415317 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.415244    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22c0b2d9-7b39-43d7-8133-8982866c8130-workload-socket\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:30:53.415317 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.415257    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22c0b2d9-7b39-43d7-8133-8982866c8130-istio-envoy\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:30:53.415317 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.415270    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-llk8q\" (UniqueName: \"kubernetes.io/projected/22c0b2d9-7b39-43d7-8133-8982866c8130-kube-api-access-llk8q\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:30:53.415317 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.415282    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22c0b2d9-7b39-43d7-8133-8982866c8130-istio-data\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:30:53.421146 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.421128    2564 scope.go:117] "RemoveContainer" containerID="a954197f5ffcce6ddc7b9c93f380dac018c07ea067893b000c5b8253da44f22f"
Sep 10 19:30:53.421495 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:30:53.421469    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a954197f5ffcce6ddc7b9c93f380dac018c07ea067893b000c5b8253da44f22f\": container with ID starting with a954197f5ffcce6ddc7b9c93f380dac018c07ea067893b000c5b8253da44f22f not found: ID does not exist" containerID="a954197f5ffcce6ddc7b9c93f380dac018c07ea067893b000c5b8253da44f22f"
Sep 10 19:30:53.421587 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.421506    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a954197f5ffcce6ddc7b9c93f380dac018c07ea067893b000c5b8253da44f22f"} err="failed to get container status \"a954197f5ffcce6ddc7b9c93f380dac018c07ea067893b000c5b8253da44f22f\": rpc error: code = NotFound desc = could not find container \"a954197f5ffcce6ddc7b9c93f380dac018c07ea067893b000c5b8253da44f22f\": container with ID starting with a954197f5ffcce6ddc7b9c93f380dac018c07ea067893b000c5b8253da44f22f not found: ID does not exist"
Sep 10 19:30:53.421587 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.421525    2564 scope.go:117] "RemoveContainer" containerID="f7df2c433a07436c98322458e4cbaf458a33f30cd3813284ad8545a52b6d8498"
Sep 10 19:30:53.421913 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:30:53.421897    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7df2c433a07436c98322458e4cbaf458a33f30cd3813284ad8545a52b6d8498\": container with ID starting with f7df2c433a07436c98322458e4cbaf458a33f30cd3813284ad8545a52b6d8498 not found: ID does not exist" containerID="f7df2c433a07436c98322458e4cbaf458a33f30cd3813284ad8545a52b6d8498"
Sep 10 19:30:53.421955 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.421917    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7df2c433a07436c98322458e4cbaf458a33f30cd3813284ad8545a52b6d8498"} err="failed to get container status \"f7df2c433a07436c98322458e4cbaf458a33f30cd3813284ad8545a52b6d8498\": rpc error: code = NotFound desc = could not find container \"f7df2c433a07436c98322458e4cbaf458a33f30cd3813284ad8545a52b6d8498\": container with ID starting with f7df2c433a07436c98322458e4cbaf458a33f30cd3813284ad8545a52b6d8498 not found: ID does not exist"
Sep 10 19:30:53.427039 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.427011    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf"]
Sep 10 19:30:53.433841 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.433816    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-ac5183cd-gw-openshift-default-77b45c6cd4-cmhpf"]
Sep 10 19:30:53.471240 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.471205    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:53.536133 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.536104    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:53.581428 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.581393    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-30f2651a-ccdc4b646-zkcd4"]
Sep 10 19:30:53.581842 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.581828    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22c0b2d9-7b39-43d7-8133-8982866c8130" containerName="istio-proxy"
Sep 10 19:30:53.581903 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.581844    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="22c0b2d9-7b39-43d7-8133-8982866c8130" containerName="istio-proxy"
Sep 10 19:30:53.581936 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.581923    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="22c0b2d9-7b39-43d7-8133-8982866c8130" containerName="istio-proxy"
Sep 10 19:30:53.581936 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.581931    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="22c0b2d9-7b39-43d7-8133-8982866c8130" containerName="istio-proxy"
Sep 10 19:30:53.586558 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.586538    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-30f2651a-ccdc4b646-zkcd4"
Sep 10 19:30:53.589277 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.589259    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-30f2651a\""
Sep 10 19:30:53.596001 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.595973    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-30f2651a-ccdc4b646-zkcd4"]
Sep 10 19:30:53.611031 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.611003    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-30f2651a-6f74d55fb7-84kq5"]
Sep 10 19:30:53.611485 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.611468    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22c0b2d9-7b39-43d7-8133-8982866c8130" containerName="istio-proxy"
Sep 10 19:30:53.611574 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.611488    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="22c0b2d9-7b39-43d7-8133-8982866c8130" containerName="istio-proxy"
Sep 10 19:30:53.614766 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.614747    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-30f2651a-6f74d55fb7-84kq5"
Sep 10 19:30:53.617316 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.617294    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-30f2651a-dockercfg-22xgm\""
Sep 10 19:30:53.617419 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.617370    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e9ce863-6c55-45ee-b28d-51ac152578dc-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-30f2651a-ccdc4b646-zkcd4\" (UID: \"5e9ce863-6c55-45ee-b28d-51ac152578dc\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-30f2651a-ccdc4b646-zkcd4"
Sep 10 19:30:53.617499 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.617430    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lddd\" (UniqueName: \"kubernetes.io/projected/5e9ce863-6c55-45ee-b28d-51ac152578dc-kube-api-access-9lddd\") pod \"payload-pre-processing-e2e-ait-30f2651a-ccdc4b646-zkcd4\" (UID: \"5e9ce863-6c55-45ee-b28d-51ac152578dc\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-30f2651a-ccdc4b646-zkcd4"
Sep 10 19:30:53.625545 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.625519    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-30f2651a-6f74d55fb7-84kq5"]
Sep 10 19:30:53.719646 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.719084    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tprlk\" (UniqueName: \"kubernetes.io/projected/8bbf5360-fae1-41c0-88a9-7f73d4627736-kube-api-access-tprlk\") pod \"payload-processing-e2e-ait-30f2651a-6f74d55fb7-84kq5\" (UID: \"8bbf5360-fae1-41c0-88a9-7f73d4627736\") " pod="openshift-ingress/payload-processing-e2e-ait-30f2651a-6f74d55fb7-84kq5"
Sep 10 19:30:53.719646 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.719167    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e9ce863-6c55-45ee-b28d-51ac152578dc-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-30f2651a-ccdc4b646-zkcd4\" (UID: \"5e9ce863-6c55-45ee-b28d-51ac152578dc\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-30f2651a-ccdc4b646-zkcd4"
Sep 10 19:30:53.719646 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.719199    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8bbf5360-fae1-41c0-88a9-7f73d4627736-plugins-config-volume\") pod \"payload-processing-e2e-ait-30f2651a-6f74d55fb7-84kq5\" (UID: \"8bbf5360-fae1-41c0-88a9-7f73d4627736\") " pod="openshift-ingress/payload-processing-e2e-ait-30f2651a-6f74d55fb7-84kq5"
Sep 10 19:30:53.719646 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.719299    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lddd\" (UniqueName: \"kubernetes.io/projected/5e9ce863-6c55-45ee-b28d-51ac152578dc-kube-api-access-9lddd\") pod \"payload-pre-processing-e2e-ait-30f2651a-ccdc4b646-zkcd4\" (UID: \"5e9ce863-6c55-45ee-b28d-51ac152578dc\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-30f2651a-ccdc4b646-zkcd4"
Sep 10 19:30:53.726184 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.720098    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e9ce863-6c55-45ee-b28d-51ac152578dc-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-30f2651a-ccdc4b646-zkcd4\" (UID: \"5e9ce863-6c55-45ee-b28d-51ac152578dc\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-30f2651a-ccdc4b646-zkcd4"
Sep 10 19:30:53.729068 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.729037    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lddd\" (UniqueName: \"kubernetes.io/projected/5e9ce863-6c55-45ee-b28d-51ac152578dc-kube-api-access-9lddd\") pod \"payload-pre-processing-e2e-ait-30f2651a-ccdc4b646-zkcd4\" (UID: \"5e9ce863-6c55-45ee-b28d-51ac152578dc\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-30f2651a-ccdc4b646-zkcd4"
Sep 10 19:30:53.820428 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.820333    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tprlk\" (UniqueName: \"kubernetes.io/projected/8bbf5360-fae1-41c0-88a9-7f73d4627736-kube-api-access-tprlk\") pod \"payload-processing-e2e-ait-30f2651a-6f74d55fb7-84kq5\" (UID: \"8bbf5360-fae1-41c0-88a9-7f73d4627736\") " pod="openshift-ingress/payload-processing-e2e-ait-30f2651a-6f74d55fb7-84kq5"
Sep 10 19:30:53.820428 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.820389    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8bbf5360-fae1-41c0-88a9-7f73d4627736-plugins-config-volume\") pod \"payload-processing-e2e-ait-30f2651a-6f74d55fb7-84kq5\" (UID: \"8bbf5360-fae1-41c0-88a9-7f73d4627736\") " pod="openshift-ingress/payload-processing-e2e-ait-30f2651a-6f74d55fb7-84kq5"
Sep 10 19:30:53.821148 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.821129    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8bbf5360-fae1-41c0-88a9-7f73d4627736-plugins-config-volume\") pod \"payload-processing-e2e-ait-30f2651a-6f74d55fb7-84kq5\" (UID: \"8bbf5360-fae1-41c0-88a9-7f73d4627736\") " pod="openshift-ingress/payload-processing-e2e-ait-30f2651a-6f74d55fb7-84kq5"
Sep 10 19:30:53.829650 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.829602    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tprlk\" (UniqueName: \"kubernetes.io/projected/8bbf5360-fae1-41c0-88a9-7f73d4627736-kube-api-access-tprlk\") pod \"payload-processing-e2e-ait-30f2651a-6f74d55fb7-84kq5\" (UID: \"8bbf5360-fae1-41c0-88a9-7f73d4627736\") " pod="openshift-ingress/payload-processing-e2e-ait-30f2651a-6f74d55fb7-84kq5"
Sep 10 19:30:53.898397 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.898359    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-30f2651a-ccdc4b646-zkcd4"
Sep 10 19:30:53.928112 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:53.928075    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-30f2651a-6f74d55fb7-84kq5"
Sep 10 19:30:54.172330 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:54.172303    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-30f2651a-ccdc4b646-zkcd4"]
Sep 10 19:30:54.174045 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:30:54.174012    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e9ce863_6c55_45ee_b28d_51ac152578dc.slice/crio-de75a931ca4f3dd34f3d9fc37dfd10e1dd936296711ba99d225d92df1b83bc19 WatchSource:0}: Error finding container de75a931ca4f3dd34f3d9fc37dfd10e1dd936296711ba99d225d92df1b83bc19: Status 404 returned error can't find the container with id de75a931ca4f3dd34f3d9fc37dfd10e1dd936296711ba99d225d92df1b83bc19
Sep 10 19:30:54.210508 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:54.210476    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-30f2651a-6f74d55fb7-84kq5"]
Sep 10 19:30:54.411536 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:54.411498    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-30f2651a-6f74d55fb7-84kq5" event={"ID":"8bbf5360-fae1-41c0-88a9-7f73d4627736","Type":"ContainerStarted","Data":"1a5d8c0ec0a75f3e166123d176295f4b21a5d90af1b2861df8faba9b7610b304"}
Sep 10 19:30:54.411779 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:54.411557    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-30f2651a-6f74d55fb7-84kq5" event={"ID":"8bbf5360-fae1-41c0-88a9-7f73d4627736","Type":"ContainerStarted","Data":"6874ab1b9468e832100568d3e679ae012caa1a35f9c014121aaafe67e137e39f"}
Sep 10 19:30:54.411779 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:54.411611    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-30f2651a-6f74d55fb7-84kq5"
Sep 10 19:30:54.413414 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:54.413391    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-30f2651a-ccdc4b646-zkcd4" event={"ID":"5e9ce863-6c55-45ee-b28d-51ac152578dc","Type":"ContainerStarted","Data":"365cee5b3a36be0c9590a57e7a4310343548cb097a047ec35e4d3f983398d1d5"}
Sep 10 19:30:54.413517 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:54.413429    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-30f2651a-ccdc4b646-zkcd4" event={"ID":"5e9ce863-6c55-45ee-b28d-51ac152578dc","Type":"ContainerStarted","Data":"de75a931ca4f3dd34f3d9fc37dfd10e1dd936296711ba99d225d92df1b83bc19"}
Sep 10 19:30:54.413517 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:54.413474    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-30f2651a-ccdc4b646-zkcd4"
Sep 10 19:30:54.428606 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:54.428498    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-30f2651a-6f74d55fb7-84kq5" podStartSLOduration=1.428480575 podStartE2EDuration="1.428480575s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:30:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:30:54.427735885 +0000 UTC m=+910.028643129" watchObservedRunningTime="2026-09-10 19:30:54.428480575 +0000 UTC m=+910.029387815"
Sep 10 19:30:54.445296 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:54.445235    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-30f2651a-ccdc4b646-zkcd4" podStartSLOduration=1.44521933 podStartE2EDuration="1.44521933s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:30:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:30:54.443575351 +0000 UTC m=+910.044482593" watchObservedRunningTime="2026-09-10 19:30:54.44521933 +0000 UTC m=+910.046126571"
Sep 10 19:30:54.961806 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:54.961767    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="22c0b2d9-7b39-43d7-8133-8982866c8130" path="/var/lib/kubelet/pods/22c0b2d9-7b39-43d7-8133-8982866c8130/volumes"
Sep 10 19:30:57.361713 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:30:57.361660    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:05.421189 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:05.421158    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-30f2651a-ccdc4b646-zkcd4"
Sep 10 19:31:05.421686 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:05.421230    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-30f2651a-6f74d55fb7-84kq5"
Sep 10 19:31:07.650185 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:07.650146    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:07.650725 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:07.650681    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-5352fd-29817810-2866b"]
Sep 10 19:31:07.653197 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:07.653173    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-5352fd-29817810-2866b"]
Sep 10 19:31:08.950512 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:08.950474    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd06d6e1-9456-427b-951b-93aeec4025d4" path="/var/lib/kubelet/pods/fd06d6e1-9456-427b-951b-93aeec4025d4/volumes"
Sep 10 19:31:08.959118 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:08.959085    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:11.536532 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:11.536497    2564 generic.go:356] "Generic (PLEG): container finished" podID="2ba42d44-358e-4101-a5a8-cd3a86d5f513" containerID="c927fe620e325629d344741220c5e83282d04b9df921e76b02e9ecf5af25c262" exitCode=0
Sep 10 19:31:11.536963 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:11.536570    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz" event={"ID":"2ba42d44-358e-4101-a5a8-cd3a86d5f513","Type":"ContainerDied","Data":"c927fe620e325629d344741220c5e83282d04b9df921e76b02e9ecf5af25c262"}
Sep 10 19:31:11.537006 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:11.536983    2564 scope.go:117] "RemoveContainer" containerID="c927fe620e325629d344741220c5e83282d04b9df921e76b02e9ecf5af25c262"
Sep 10 19:31:11.538097 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:11.538068    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:31:11.538173 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:11.538140    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:31:11.538212 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:11.538171    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:31:12.545996 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:12.545960    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz" event={"ID":"2ba42d44-358e-4101-a5a8-cd3a86d5f513","Type":"ContainerStarted","Data":"f33e5c23372eaf9c67e6f8990beb4ab252364aab3f14413cfb20255ee549cdc4"}
Sep 10 19:31:13.043536 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:13.043498    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz"
Sep 10 19:31:13.043831 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:13.043805    2564 patch_prober.go:28] interesting pod/e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Sep 10 19:31:13.043884 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:13.043857    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz" podUID="2ba42d44-358e-4101-a5a8-cd3a86d5f513" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Sep 10 19:31:13.253394 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:13.253354    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:14.044182 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:14.044143    2564 patch_prober.go:28] interesting pod/e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Sep 10 19:31:14.044663 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:14.044201    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz" podUID="2ba42d44-358e-4101-a5a8-cd3a86d5f513" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Sep 10 19:31:15.044010 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:15.043972    2564 patch_prober.go:28] interesting pod/e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Sep 10 19:31:15.044187 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:15.044047    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz" podUID="2ba42d44-358e-4101-a5a8-cd3a86d5f513" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Sep 10 19:31:15.072295 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:15.072259    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:16.044853 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:16.044818    2564 patch_prober.go:28] interesting pod/e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Sep 10 19:31:16.045044 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:16.044875    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz" podUID="2ba42d44-358e-4101-a5a8-cd3a86d5f513" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Sep 10 19:31:17.044113 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:17.044077    2564 patch_prober.go:28] interesting pod/e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Sep 10 19:31:17.044579 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:17.044138    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz" podUID="2ba42d44-358e-4101-a5a8-cd3a86d5f513" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Sep 10 19:31:18.048147 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:18.048115    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz"
Sep 10 19:31:18.048610 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:18.048519    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz"
Sep 10 19:31:18.049402 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:18.049378    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz"
Sep 10 19:31:19.061985 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:19.061922    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:23.446816 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:23.446782    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:23.475460 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:23.475413    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-30f2651a-6f74d55fb7-84kq5"]
Sep 10 19:31:23.475917 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:23.475723    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-30f2651a-6f74d55fb7-84kq5" podUID="8bbf5360-fae1-41c0-88a9-7f73d4627736" containerName="payload-processing" containerID="cri-o://1a5d8c0ec0a75f3e166123d176295f4b21a5d90af1b2861df8faba9b7610b304" gracePeriod=30
Sep 10 19:31:23.482683 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:23.482600    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-30f2651a-ccdc4b646-zkcd4"]
Sep 10 19:31:23.486848 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:23.483095    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-30f2651a-ccdc4b646-zkcd4" podUID="5e9ce863-6c55-45ee-b28d-51ac152578dc" containerName="payload-pre-processing" containerID="cri-o://365cee5b3a36be0c9590a57e7a4310343548cb097a047ec35e4d3f983398d1d5" gracePeriod=30
Sep 10 19:31:23.520151 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:23.520109    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:23.574898 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:23.574857    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:23.616206 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:23.616177    2564 generic.go:356] "Generic (PLEG): container finished" podID="8bbf5360-fae1-41c0-88a9-7f73d4627736" containerID="1a5d8c0ec0a75f3e166123d176295f4b21a5d90af1b2861df8faba9b7610b304" exitCode=0
Sep 10 19:31:23.616360 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:23.616255    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-30f2651a-6f74d55fb7-84kq5" event={"ID":"8bbf5360-fae1-41c0-88a9-7f73d4627736","Type":"ContainerDied","Data":"1a5d8c0ec0a75f3e166123d176295f4b21a5d90af1b2861df8faba9b7610b304"}
Sep 10 19:31:23.617867 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:23.617841    2564 generic.go:356] "Generic (PLEG): container finished" podID="5e9ce863-6c55-45ee-b28d-51ac152578dc" containerID="365cee5b3a36be0c9590a57e7a4310343548cb097a047ec35e4d3f983398d1d5" exitCode=0
Sep 10 19:31:23.617978 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:23.617920    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-30f2651a-ccdc4b646-zkcd4" event={"ID":"5e9ce863-6c55-45ee-b28d-51ac152578dc","Type":"ContainerDied","Data":"365cee5b3a36be0c9590a57e7a4310343548cb097a047ec35e4d3f983398d1d5"}
Sep 10 19:31:23.658960 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:23.658931    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:23.719694 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:23.719580    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:23.774089 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:23.774064    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-30f2651a-6f74d55fb7-84kq5"
Sep 10 19:31:23.777821 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:23.777799    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-30f2651a-ccdc4b646-zkcd4"
Sep 10 19:31:23.932412 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:23.932371    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8bbf5360-fae1-41c0-88a9-7f73d4627736-plugins-config-volume\") pod \"8bbf5360-fae1-41c0-88a9-7f73d4627736\" (UID: \"8bbf5360-fae1-41c0-88a9-7f73d4627736\") "
Sep 10 19:31:23.932601 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:23.932443    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tprlk\" (UniqueName: \"kubernetes.io/projected/8bbf5360-fae1-41c0-88a9-7f73d4627736-kube-api-access-tprlk\") pod \"8bbf5360-fae1-41c0-88a9-7f73d4627736\" (UID: \"8bbf5360-fae1-41c0-88a9-7f73d4627736\") "
Sep 10 19:31:23.932601 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:23.932563    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9lddd\" (UniqueName: \"kubernetes.io/projected/5e9ce863-6c55-45ee-b28d-51ac152578dc-kube-api-access-9lddd\") pod \"5e9ce863-6c55-45ee-b28d-51ac152578dc\" (UID: \"5e9ce863-6c55-45ee-b28d-51ac152578dc\") "
Sep 10 19:31:23.932759 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:23.932643    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e9ce863-6c55-45ee-b28d-51ac152578dc-plugins-config-volume\") pod \"5e9ce863-6c55-45ee-b28d-51ac152578dc\" (UID: \"5e9ce863-6c55-45ee-b28d-51ac152578dc\") "
Sep 10 19:31:23.932759 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:23.932718    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8bbf5360-fae1-41c0-88a9-7f73d4627736-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8bbf5360-fae1-41c0-88a9-7f73d4627736" (UID: "8bbf5360-fae1-41c0-88a9-7f73d4627736"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:31:23.932989 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:23.932971    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8bbf5360-fae1-41c0-88a9-7f73d4627736-plugins-config-volume\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:31:23.933045 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:23.932982    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e9ce863-6c55-45ee-b28d-51ac152578dc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5e9ce863-6c55-45ee-b28d-51ac152578dc" (UID: "5e9ce863-6c55-45ee-b28d-51ac152578dc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:31:23.934550 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:23.934531    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8bbf5360-fae1-41c0-88a9-7f73d4627736-kube-api-access-tprlk" (OuterVolumeSpecName: "kube-api-access-tprlk") pod "8bbf5360-fae1-41c0-88a9-7f73d4627736" (UID: "8bbf5360-fae1-41c0-88a9-7f73d4627736"). InnerVolumeSpecName "kube-api-access-tprlk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:31:23.934598 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:23.934581    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e9ce863-6c55-45ee-b28d-51ac152578dc-kube-api-access-9lddd" (OuterVolumeSpecName: "kube-api-access-9lddd") pod "5e9ce863-6c55-45ee-b28d-51ac152578dc" (UID: "5e9ce863-6c55-45ee-b28d-51ac152578dc"). InnerVolumeSpecName "kube-api-access-9lddd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:31:24.033851 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:24.033816    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9lddd\" (UniqueName: \"kubernetes.io/projected/5e9ce863-6c55-45ee-b28d-51ac152578dc-kube-api-access-9lddd\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:31:24.033851 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:24.033847    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e9ce863-6c55-45ee-b28d-51ac152578dc-plugins-config-volume\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:31:24.033851 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:24.033858    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tprlk\" (UniqueName: \"kubernetes.io/projected/8bbf5360-fae1-41c0-88a9-7f73d4627736-kube-api-access-tprlk\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:31:24.631027 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:24.630984    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-30f2651a-6f74d55fb7-84kq5" event={"ID":"8bbf5360-fae1-41c0-88a9-7f73d4627736","Type":"ContainerDied","Data":"6874ab1b9468e832100568d3e679ae012caa1a35f9c014121aaafe67e137e39f"}
Sep 10 19:31:24.631462 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:24.630996    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-30f2651a-6f74d55fb7-84kq5"
Sep 10 19:31:24.631462 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:24.631080    2564 scope.go:117] "RemoveContainer" containerID="1a5d8c0ec0a75f3e166123d176295f4b21a5d90af1b2861df8faba9b7610b304"
Sep 10 19:31:24.632548 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:24.632458    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-30f2651a-ccdc4b646-zkcd4" event={"ID":"5e9ce863-6c55-45ee-b28d-51ac152578dc","Type":"ContainerDied","Data":"de75a931ca4f3dd34f3d9fc37dfd10e1dd936296711ba99d225d92df1b83bc19"}
Sep 10 19:31:24.632640 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:24.632553    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-30f2651a-ccdc4b646-zkcd4"
Sep 10 19:31:24.640298 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:24.640277    2564 scope.go:117] "RemoveContainer" containerID="365cee5b3a36be0c9590a57e7a4310343548cb097a047ec35e4d3f983398d1d5"
Sep 10 19:31:24.660237 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:24.660204    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-30f2651a-ccdc4b646-zkcd4"]
Sep 10 19:31:24.667829 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:24.667801    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-30f2651a-ccdc4b646-zkcd4"]
Sep 10 19:31:24.679707 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:24.679678    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-30f2651a-6f74d55fb7-84kq5"]
Sep 10 19:31:24.685309 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:24.685282    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-30f2651a-6f74d55fb7-84kq5"]
Sep 10 19:31:24.817739 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:24.817695    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz"]
Sep 10 19:31:24.817980 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:24.817949    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz" podUID="2ba42d44-358e-4101-a5a8-cd3a86d5f513" containerName="istio-proxy" containerID="cri-o://f33e5c23372eaf9c67e6f8990beb4ab252364aab3f14413cfb20255ee549cdc4" gracePeriod=30
Sep 10 19:31:24.839188 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:24.839147    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:24.951536 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:24.951504    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e9ce863-6c55-45ee-b28d-51ac152578dc" path="/var/lib/kubelet/pods/5e9ce863-6c55-45ee-b28d-51ac152578dc/volumes"
Sep 10 19:31:24.951943 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:24.951926    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8bbf5360-fae1-41c0-88a9-7f73d4627736" path="/var/lib/kubelet/pods/8bbf5360-fae1-41c0-88a9-7f73d4627736/volumes"
Sep 10 19:31:25.286659 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:25.286611    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:25.593269 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:25.593185    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:25.780724 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:25.780691    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:27.460365 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:27.460316    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:28.722160 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:28.722117    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:30.089202 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:30.089141    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz"
Sep 10 19:31:30.200901 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:30.200866    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ba42d44-358e-4101-a5a8-cd3a86d5f513-istio-data\") pod \"2ba42d44-358e-4101-a5a8-cd3a86d5f513\" (UID: \"2ba42d44-358e-4101-a5a8-cd3a86d5f513\") "
Sep 10 19:31:30.200901 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:30.200904    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ba42d44-358e-4101-a5a8-cd3a86d5f513-istio-token\") pod \"2ba42d44-358e-4101-a5a8-cd3a86d5f513\" (UID: \"2ba42d44-358e-4101-a5a8-cd3a86d5f513\") "
Sep 10 19:31:30.201165 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:30.200924    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ba42d44-358e-4101-a5a8-cd3a86d5f513-istio-envoy\") pod \"2ba42d44-358e-4101-a5a8-cd3a86d5f513\" (UID: \"2ba42d44-358e-4101-a5a8-cd3a86d5f513\") "
Sep 10 19:31:30.201165 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:30.200956    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v2lm9\" (UniqueName: \"kubernetes.io/projected/2ba42d44-358e-4101-a5a8-cd3a86d5f513-kube-api-access-v2lm9\") pod \"2ba42d44-358e-4101-a5a8-cd3a86d5f513\" (UID: \"2ba42d44-358e-4101-a5a8-cd3a86d5f513\") "
Sep 10 19:31:30.201165 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:30.200993    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ba42d44-358e-4101-a5a8-cd3a86d5f513-workload-certs\") pod \"2ba42d44-358e-4101-a5a8-cd3a86d5f513\" (UID: \"2ba42d44-358e-4101-a5a8-cd3a86d5f513\") "
Sep 10 19:31:30.201165 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:30.201050    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ba42d44-358e-4101-a5a8-cd3a86d5f513-workload-socket\") pod \"2ba42d44-358e-4101-a5a8-cd3a86d5f513\" (UID: \"2ba42d44-358e-4101-a5a8-cd3a86d5f513\") "
Sep 10 19:31:30.201165 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:30.201068    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ba42d44-358e-4101-a5a8-cd3a86d5f513-istiod-ca-cert\") pod \"2ba42d44-358e-4101-a5a8-cd3a86d5f513\" (UID: \"2ba42d44-358e-4101-a5a8-cd3a86d5f513\") "
Sep 10 19:31:30.201165 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:30.201089    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ba42d44-358e-4101-a5a8-cd3a86d5f513-istio-podinfo\") pod \"2ba42d44-358e-4101-a5a8-cd3a86d5f513\" (UID: \"2ba42d44-358e-4101-a5a8-cd3a86d5f513\") "
Sep 10 19:31:30.201165 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:30.201108    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ba42d44-358e-4101-a5a8-cd3a86d5f513-credential-socket\") pod \"2ba42d44-358e-4101-a5a8-cd3a86d5f513\" (UID: \"2ba42d44-358e-4101-a5a8-cd3a86d5f513\") "
Sep 10 19:31:30.201521 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:30.201353    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ba42d44-358e-4101-a5a8-cd3a86d5f513-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2ba42d44-358e-4101-a5a8-cd3a86d5f513" (UID: "2ba42d44-358e-4101-a5a8-cd3a86d5f513"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:31:30.201521 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:30.201363    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ba42d44-358e-4101-a5a8-cd3a86d5f513-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2ba42d44-358e-4101-a5a8-cd3a86d5f513" (UID: "2ba42d44-358e-4101-a5a8-cd3a86d5f513"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:31:30.201521 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:30.201498    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ba42d44-358e-4101-a5a8-cd3a86d5f513-workload-certs\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:31:30.201521 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:30.201519    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ba42d44-358e-4101-a5a8-cd3a86d5f513-workload-socket\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:31:30.201768 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:30.201557    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ba42d44-358e-4101-a5a8-cd3a86d5f513-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2ba42d44-358e-4101-a5a8-cd3a86d5f513" (UID: "2ba42d44-358e-4101-a5a8-cd3a86d5f513"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:31:30.201768 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:30.201596    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ba42d44-358e-4101-a5a8-cd3a86d5f513-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2ba42d44-358e-4101-a5a8-cd3a86d5f513" (UID: "2ba42d44-358e-4101-a5a8-cd3a86d5f513"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:31:30.201902 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:30.201862    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ba42d44-358e-4101-a5a8-cd3a86d5f513-istio-data" (OuterVolumeSpecName: "istio-data") pod "2ba42d44-358e-4101-a5a8-cd3a86d5f513" (UID: "2ba42d44-358e-4101-a5a8-cd3a86d5f513"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:31:30.203190 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:30.203164    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ba42d44-358e-4101-a5a8-cd3a86d5f513-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2ba42d44-358e-4101-a5a8-cd3a86d5f513" (UID: "2ba42d44-358e-4101-a5a8-cd3a86d5f513"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:31:30.203335 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:30.203305    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2ba42d44-358e-4101-a5a8-cd3a86d5f513-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2ba42d44-358e-4101-a5a8-cd3a86d5f513" (UID: "2ba42d44-358e-4101-a5a8-cd3a86d5f513"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 19:31:30.203572 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:30.203548    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ba42d44-358e-4101-a5a8-cd3a86d5f513-kube-api-access-v2lm9" (OuterVolumeSpecName: "kube-api-access-v2lm9") pod "2ba42d44-358e-4101-a5a8-cd3a86d5f513" (UID: "2ba42d44-358e-4101-a5a8-cd3a86d5f513"). InnerVolumeSpecName "kube-api-access-v2lm9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:31:30.203611 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:30.203559    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ba42d44-358e-4101-a5a8-cd3a86d5f513-istio-token" (OuterVolumeSpecName: "istio-token") pod "2ba42d44-358e-4101-a5a8-cd3a86d5f513" (UID: "2ba42d44-358e-4101-a5a8-cd3a86d5f513"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:31:30.302894 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:30.302859    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ba42d44-358e-4101-a5a8-cd3a86d5f513-istio-data\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:31:30.302894 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:30.302890    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ba42d44-358e-4101-a5a8-cd3a86d5f513-istio-token\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:31:30.302894 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:30.302901    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ba42d44-358e-4101-a5a8-cd3a86d5f513-istio-envoy\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:31:30.303129 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:30.302911    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v2lm9\" (UniqueName: \"kubernetes.io/projected/2ba42d44-358e-4101-a5a8-cd3a86d5f513-kube-api-access-v2lm9\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:31:30.303129 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:30.302922    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ba42d44-358e-4101-a5a8-cd3a86d5f513-istiod-ca-cert\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:31:30.303129 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:30.302931    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ba42d44-358e-4101-a5a8-cd3a86d5f513-istio-podinfo\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:31:30.303129 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:30.302938    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ba42d44-358e-4101-a5a8-cd3a86d5f513-credential-socket\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:31:30.676376 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:30.676339    2564 generic.go:356] "Generic (PLEG): container finished" podID="2ba42d44-358e-4101-a5a8-cd3a86d5f513" containerID="f33e5c23372eaf9c67e6f8990beb4ab252364aab3f14413cfb20255ee549cdc4" exitCode=0
Sep 10 19:31:30.676536 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:30.676387    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz"
Sep 10 19:31:30.676536 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:30.676424    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz" event={"ID":"2ba42d44-358e-4101-a5a8-cd3a86d5f513","Type":"ContainerDied","Data":"f33e5c23372eaf9c67e6f8990beb4ab252364aab3f14413cfb20255ee549cdc4"}
Sep 10 19:31:30.676536 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:30.676474    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz" event={"ID":"2ba42d44-358e-4101-a5a8-cd3a86d5f513","Type":"ContainerDied","Data":"d78548ac704a98a0bbb8781a6ac3575767aa5e9534c219686644b5e4e97d3b9f"}
Sep 10 19:31:30.676536 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:30.676496    2564 scope.go:117] "RemoveContainer" containerID="f33e5c23372eaf9c67e6f8990beb4ab252364aab3f14413cfb20255ee549cdc4"
Sep 10 19:31:30.687878 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:30.687861    2564 scope.go:117] "RemoveContainer" containerID="c927fe620e325629d344741220c5e83282d04b9df921e76b02e9ecf5af25c262"
Sep 10 19:31:30.696150 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:30.696127    2564 scope.go:117] "RemoveContainer" containerID="f33e5c23372eaf9c67e6f8990beb4ab252364aab3f14413cfb20255ee549cdc4"
Sep 10 19:31:30.696409 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:31:30.696391    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f33e5c23372eaf9c67e6f8990beb4ab252364aab3f14413cfb20255ee549cdc4\": container with ID starting with f33e5c23372eaf9c67e6f8990beb4ab252364aab3f14413cfb20255ee549cdc4 not found: ID does not exist" containerID="f33e5c23372eaf9c67e6f8990beb4ab252364aab3f14413cfb20255ee549cdc4"
Sep 10 19:31:30.696453 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:30.696419    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f33e5c23372eaf9c67e6f8990beb4ab252364aab3f14413cfb20255ee549cdc4"} err="failed to get container status \"f33e5c23372eaf9c67e6f8990beb4ab252364aab3f14413cfb20255ee549cdc4\": rpc error: code = NotFound desc = could not find container \"f33e5c23372eaf9c67e6f8990beb4ab252364aab3f14413cfb20255ee549cdc4\": container with ID starting with f33e5c23372eaf9c67e6f8990beb4ab252364aab3f14413cfb20255ee549cdc4 not found: ID does not exist"
Sep 10 19:31:30.696453 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:30.696435    2564 scope.go:117] "RemoveContainer" containerID="c927fe620e325629d344741220c5e83282d04b9df921e76b02e9ecf5af25c262"
Sep 10 19:31:30.696717 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:31:30.696697    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c927fe620e325629d344741220c5e83282d04b9df921e76b02e9ecf5af25c262\": container with ID starting with c927fe620e325629d344741220c5e83282d04b9df921e76b02e9ecf5af25c262 not found: ID does not exist" containerID="c927fe620e325629d344741220c5e83282d04b9df921e76b02e9ecf5af25c262"
Sep 10 19:31:30.696783 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:30.696723    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c927fe620e325629d344741220c5e83282d04b9df921e76b02e9ecf5af25c262"} err="failed to get container status \"c927fe620e325629d344741220c5e83282d04b9df921e76b02e9ecf5af25c262\": rpc error: code = NotFound desc = could not find container \"c927fe620e325629d344741220c5e83282d04b9df921e76b02e9ecf5af25c262\": container with ID starting with c927fe620e325629d344741220c5e83282d04b9df921e76b02e9ecf5af25c262 not found: ID does not exist"
Sep 10 19:31:30.703564 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:30.703539    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz"]
Sep 10 19:31:30.708182 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:30.708163    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-30f2651a-openshift-default-5d6595dc48-pb6gz"]
Sep 10 19:31:30.949695 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:30.949595    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ba42d44-358e-4101-a5a8-cd3a86d5f513" path="/var/lib/kubelet/pods/2ba42d44-358e-4101-a5a8-cd3a86d5f513/volumes"
Sep 10 19:31:32.154846 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:32.154808    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:32.247590 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:32.247546    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:33.347284 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:33.347071    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:33.440983 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:33.440945    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:33.540956 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:33.540918    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:33.852795 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:33.852759    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:33.952594 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:33.952555    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:38.219186 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:38.219143    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:41.523965 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:41.523931    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:50.479066 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:50.479027    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:50.713743 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:50.713705    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b"]
Sep 10 19:31:50.714015 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:50.713988    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b" podUID="1a02f4a9-bd86-4658-aa0a-a82758731f76" containerName="istio-proxy" containerID="cri-o://b8d0baf5c584f74e1aa02444d6ba7138b85ee3396c1d248b1fe5bbed846f5e94" gracePeriod=30
Sep 10 19:31:50.729932 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:50.729849    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:51.195073 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:51.195035    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:51.296936 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:51.296898    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:51.489566 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:51.489479    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j"]
Sep 10 19:31:51.490300 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:51.490275    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e9ce863-6c55-45ee-b28d-51ac152578dc" containerName="payload-pre-processing"
Sep 10 19:31:51.490300 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:51.490301    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e9ce863-6c55-45ee-b28d-51ac152578dc" containerName="payload-pre-processing"
Sep 10 19:31:51.490451 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:51.490318    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ba42d44-358e-4101-a5a8-cd3a86d5f513" containerName="istio-proxy"
Sep 10 19:31:51.490451 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:51.490328    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ba42d44-358e-4101-a5a8-cd3a86d5f513" containerName="istio-proxy"
Sep 10 19:31:51.490451 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:51.490352    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ba42d44-358e-4101-a5a8-cd3a86d5f513" containerName="istio-proxy"
Sep 10 19:31:51.490451 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:51.490360    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ba42d44-358e-4101-a5a8-cd3a86d5f513" containerName="istio-proxy"
Sep 10 19:31:51.490451 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:51.490378    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bbf5360-fae1-41c0-88a9-7f73d4627736" containerName="payload-processing"
Sep 10 19:31:51.490451 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:51.490385    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bbf5360-fae1-41c0-88a9-7f73d4627736" containerName="payload-processing"
Sep 10 19:31:51.490712 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:51.490483    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e9ce863-6c55-45ee-b28d-51ac152578dc" containerName="payload-pre-processing"
Sep 10 19:31:51.490712 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:51.490496    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bbf5360-fae1-41c0-88a9-7f73d4627736" containerName="payload-processing"
Sep 10 19:31:51.490712 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:51.490506    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ba42d44-358e-4101-a5a8-cd3a86d5f513" containerName="istio-proxy"
Sep 10 19:31:51.490712 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:51.490516    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ba42d44-358e-4101-a5a8-cd3a86d5f513" containerName="istio-proxy"
Sep 10 19:31:51.495126 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:51.495107    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j"
Sep 10 19:31:51.497522 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:51.497497    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-9a4e54-openshift-default-dockercfg-26svs\""
Sep 10 19:31:51.506326 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:51.506302    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j"]
Sep 10 19:31:51.523696 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:51.523668    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:51.616302 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:51.616268    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27890338-9e2e-40c3-a800-ee4f7e2030e2-istio-envoy\") pod \"e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j\" (UID: \"27890338-9e2e-40c3-a800-ee4f7e2030e2\") " pod="openshift-ingress/e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j"
Sep 10 19:31:51.616302 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:51.616308    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27890338-9e2e-40c3-a800-ee4f7e2030e2-istio-data\") pod \"e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j\" (UID: \"27890338-9e2e-40c3-a800-ee4f7e2030e2\") " pod="openshift-ingress/e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j"
Sep 10 19:31:51.616522 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:51.616383    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27890338-9e2e-40c3-a800-ee4f7e2030e2-workload-certs\") pod \"e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j\" (UID: \"27890338-9e2e-40c3-a800-ee4f7e2030e2\") " pod="openshift-ingress/e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j"
Sep 10 19:31:51.616522 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:51.616433    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27890338-9e2e-40c3-a800-ee4f7e2030e2-workload-socket\") pod \"e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j\" (UID: \"27890338-9e2e-40c3-a800-ee4f7e2030e2\") " pod="openshift-ingress/e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j"
Sep 10 19:31:51.616522 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:51.616471    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27890338-9e2e-40c3-a800-ee4f7e2030e2-credential-socket\") pod \"e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j\" (UID: \"27890338-9e2e-40c3-a800-ee4f7e2030e2\") " pod="openshift-ingress/e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j"
Sep 10 19:31:51.616522 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:51.616505    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27890338-9e2e-40c3-a800-ee4f7e2030e2-istio-token\") pod \"e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j\" (UID: \"27890338-9e2e-40c3-a800-ee4f7e2030e2\") " pod="openshift-ingress/e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j"
Sep 10 19:31:51.616674 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:51.616532    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgfz9\" (UniqueName: \"kubernetes.io/projected/27890338-9e2e-40c3-a800-ee4f7e2030e2-kube-api-access-jgfz9\") pod \"e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j\" (UID: \"27890338-9e2e-40c3-a800-ee4f7e2030e2\") " pod="openshift-ingress/e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j"
Sep 10 19:31:51.616674 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:51.616587    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27890338-9e2e-40c3-a800-ee4f7e2030e2-istiod-ca-cert\") pod \"e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j\" (UID: \"27890338-9e2e-40c3-a800-ee4f7e2030e2\") " pod="openshift-ingress/e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j"
Sep 10 19:31:51.616674 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:51.616616    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27890338-9e2e-40c3-a800-ee4f7e2030e2-istio-podinfo\") pod \"e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j\" (UID: \"27890338-9e2e-40c3-a800-ee4f7e2030e2\") " pod="openshift-ingress/e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j"
Sep 10 19:31:51.620467 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:51.620428    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:51.717473 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:51.717441    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27890338-9e2e-40c3-a800-ee4f7e2030e2-workload-socket\") pod \"e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j\" (UID: \"27890338-9e2e-40c3-a800-ee4f7e2030e2\") " pod="openshift-ingress/e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j"
Sep 10 19:31:51.717710 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:51.717487    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27890338-9e2e-40c3-a800-ee4f7e2030e2-credential-socket\") pod \"e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j\" (UID: \"27890338-9e2e-40c3-a800-ee4f7e2030e2\") " pod="openshift-ingress/e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j"
Sep 10 19:31:51.717710 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:51.717509    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27890338-9e2e-40c3-a800-ee4f7e2030e2-istio-token\") pod \"e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j\" (UID: \"27890338-9e2e-40c3-a800-ee4f7e2030e2\") " pod="openshift-ingress/e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j"
Sep 10 19:31:51.717710 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:51.717537    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgfz9\" (UniqueName: \"kubernetes.io/projected/27890338-9e2e-40c3-a800-ee4f7e2030e2-kube-api-access-jgfz9\") pod \"e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j\" (UID: \"27890338-9e2e-40c3-a800-ee4f7e2030e2\") " pod="openshift-ingress/e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j"
Sep 10 19:31:51.717710 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:51.717582    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27890338-9e2e-40c3-a800-ee4f7e2030e2-istiod-ca-cert\") pod \"e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j\" (UID: \"27890338-9e2e-40c3-a800-ee4f7e2030e2\") " pod="openshift-ingress/e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j"
Sep 10 19:31:51.717710 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:51.717612    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27890338-9e2e-40c3-a800-ee4f7e2030e2-istio-podinfo\") pod \"e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j\" (UID: \"27890338-9e2e-40c3-a800-ee4f7e2030e2\") " pod="openshift-ingress/e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j"
Sep 10 19:31:51.718005 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:51.717715    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27890338-9e2e-40c3-a800-ee4f7e2030e2-istio-envoy\") pod \"e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j\" (UID: \"27890338-9e2e-40c3-a800-ee4f7e2030e2\") " pod="openshift-ingress/e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j"
Sep 10 19:31:51.718005 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:51.717747    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27890338-9e2e-40c3-a800-ee4f7e2030e2-istio-data\") pod \"e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j\" (UID: \"27890338-9e2e-40c3-a800-ee4f7e2030e2\") " pod="openshift-ingress/e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j"
Sep 10 19:31:51.718005 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:51.717771    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27890338-9e2e-40c3-a800-ee4f7e2030e2-workload-certs\") pod \"e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j\" (UID: \"27890338-9e2e-40c3-a800-ee4f7e2030e2\") " pod="openshift-ingress/e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j"
Sep 10 19:31:51.718186 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:51.717998    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27890338-9e2e-40c3-a800-ee4f7e2030e2-credential-socket\") pod \"e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j\" (UID: \"27890338-9e2e-40c3-a800-ee4f7e2030e2\") " pod="openshift-ingress/e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j"
Sep 10 19:31:51.718186 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:51.717997    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27890338-9e2e-40c3-a800-ee4f7e2030e2-workload-socket\") pod \"e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j\" (UID: \"27890338-9e2e-40c3-a800-ee4f7e2030e2\") " pod="openshift-ingress/e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j"
Sep 10 19:31:51.718186 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:51.718087    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27890338-9e2e-40c3-a800-ee4f7e2030e2-workload-certs\") pod \"e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j\" (UID: \"27890338-9e2e-40c3-a800-ee4f7e2030e2\") " pod="openshift-ingress/e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j"
Sep 10 19:31:51.718343 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:51.718244    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27890338-9e2e-40c3-a800-ee4f7e2030e2-istio-data\") pod \"e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j\" (UID: \"27890338-9e2e-40c3-a800-ee4f7e2030e2\") " pod="openshift-ingress/e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j"
Sep 10 19:31:51.718343 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:51.718331    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27890338-9e2e-40c3-a800-ee4f7e2030e2-istiod-ca-cert\") pod \"e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j\" (UID: \"27890338-9e2e-40c3-a800-ee4f7e2030e2\") " pod="openshift-ingress/e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j"
Sep 10 19:31:51.720153 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:51.720119    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27890338-9e2e-40c3-a800-ee4f7e2030e2-istio-envoy\") pod \"e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j\" (UID: \"27890338-9e2e-40c3-a800-ee4f7e2030e2\") " pod="openshift-ingress/e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j"
Sep 10 19:31:51.720500 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:51.720480    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27890338-9e2e-40c3-a800-ee4f7e2030e2-istio-podinfo\") pod \"e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j\" (UID: \"27890338-9e2e-40c3-a800-ee4f7e2030e2\") " pod="openshift-ingress/e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j"
Sep 10 19:31:51.726540 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:51.726512    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgfz9\" (UniqueName: \"kubernetes.io/projected/27890338-9e2e-40c3-a800-ee4f7e2030e2-kube-api-access-jgfz9\") pod \"e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j\" (UID: \"27890338-9e2e-40c3-a800-ee4f7e2030e2\") " pod="openshift-ingress/e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j"
Sep 10 19:31:51.726634 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:51.726605    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27890338-9e2e-40c3-a800-ee4f7e2030e2-istio-token\") pod \"e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j\" (UID: \"27890338-9e2e-40c3-a800-ee4f7e2030e2\") " pod="openshift-ingress/e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j"
Sep 10 19:31:51.806884 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:51.806850    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j"
Sep 10 19:31:52.096184 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:52.096152    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j"]
Sep 10 19:31:52.098032 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:31:52.098001    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27890338_9e2e_40c3_a800_ee4f7e2030e2.slice/crio-118d2d193620c638464e0e30a37341a6ad450f56191066d7dd516e0cea856540 WatchSource:0}: Error finding container 118d2d193620c638464e0e30a37341a6ad450f56191066d7dd516e0cea856540: Status 404 returned error can't find the container with id 118d2d193620c638464e0e30a37341a6ad450f56191066d7dd516e0cea856540
Sep 10 19:31:52.100412 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:52.100381    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:31:52.100478 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:52.100445    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:31:52.100478 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:52.100474    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:31:52.804941 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:52.804906    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j" event={"ID":"27890338-9e2e-40c3-a800-ee4f7e2030e2","Type":"ContainerStarted","Data":"ce8c7785a3788b1020897c250d2de60341942f81572b76c4dd3ab8b5006dc8f5"}
Sep 10 19:31:52.805329 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:52.804970    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j" event={"ID":"27890338-9e2e-40c3-a800-ee4f7e2030e2","Type":"ContainerStarted","Data":"118d2d193620c638464e0e30a37341a6ad450f56191066d7dd516e0cea856540"}
Sep 10 19:31:52.807307 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:52.807275    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j"
Sep 10 19:31:52.828821 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:52.828770    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j" podStartSLOduration=1.828754915 podStartE2EDuration="1.828754915s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:31:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:31:52.826594113 +0000 UTC m=+968.427501352" watchObservedRunningTime="2026-09-10 19:31:52.828754915 +0000 UTC m=+968.429662154"
Sep 10 19:31:53.812538 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:53.812510    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j"
Sep 10 19:31:54.762208 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:54.762157    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:54.815134 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:54.815099    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j"
Sep 10 19:31:54.816139 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:54.816117    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j"
Sep 10 19:31:54.857764 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:54.857729    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:55.981459 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:55.981395    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b"
Sep 10 19:31:56.097082 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.097052    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:56.167004 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.166960    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a02f4a9-bd86-4658-aa0a-a82758731f76-workload-socket\") pod \"1a02f4a9-bd86-4658-aa0a-a82758731f76\" (UID: \"1a02f4a9-bd86-4658-aa0a-a82758731f76\") "
Sep 10 19:31:56.168405 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.167428    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a02f4a9-bd86-4658-aa0a-a82758731f76-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1a02f4a9-bd86-4658-aa0a-a82758731f76" (UID: "1a02f4a9-bd86-4658-aa0a-a82758731f76"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:31:56.168683 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.168568    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a02f4a9-bd86-4658-aa0a-a82758731f76-istiod-ca-cert\") pod \"1a02f4a9-bd86-4658-aa0a-a82758731f76\" (UID: \"1a02f4a9-bd86-4658-aa0a-a82758731f76\") "
Sep 10 19:31:56.169938 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.168932    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a02f4a9-bd86-4658-aa0a-a82758731f76-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1a02f4a9-bd86-4658-aa0a-a82758731f76" (UID: "1a02f4a9-bd86-4658-aa0a-a82758731f76"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:31:56.169938 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.168948    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a02f4a9-bd86-4658-aa0a-a82758731f76-credential-socket\") pod \"1a02f4a9-bd86-4658-aa0a-a82758731f76\" (UID: \"1a02f4a9-bd86-4658-aa0a-a82758731f76\") "
Sep 10 19:31:56.169938 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.168984    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a02f4a9-bd86-4658-aa0a-a82758731f76-istio-token\") pod \"1a02f4a9-bd86-4658-aa0a-a82758731f76\" (UID: \"1a02f4a9-bd86-4658-aa0a-a82758731f76\") "
Sep 10 19:31:56.169938 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.169025    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a02f4a9-bd86-4658-aa0a-a82758731f76-istio-podinfo\") pod \"1a02f4a9-bd86-4658-aa0a-a82758731f76\" (UID: \"1a02f4a9-bd86-4658-aa0a-a82758731f76\") "
Sep 10 19:31:56.169938 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.169044    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a02f4a9-bd86-4658-aa0a-a82758731f76-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1a02f4a9-bd86-4658-aa0a-a82758731f76" (UID: "1a02f4a9-bd86-4658-aa0a-a82758731f76"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:31:56.169938 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.169063    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bf4xk\" (UniqueName: \"kubernetes.io/projected/1a02f4a9-bd86-4658-aa0a-a82758731f76-kube-api-access-bf4xk\") pod \"1a02f4a9-bd86-4658-aa0a-a82758731f76\" (UID: \"1a02f4a9-bd86-4658-aa0a-a82758731f76\") "
Sep 10 19:31:56.169938 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.169115    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a02f4a9-bd86-4658-aa0a-a82758731f76-workload-certs\") pod \"1a02f4a9-bd86-4658-aa0a-a82758731f76\" (UID: \"1a02f4a9-bd86-4658-aa0a-a82758731f76\") "
Sep 10 19:31:56.169938 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.169146    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a02f4a9-bd86-4658-aa0a-a82758731f76-istio-data\") pod \"1a02f4a9-bd86-4658-aa0a-a82758731f76\" (UID: \"1a02f4a9-bd86-4658-aa0a-a82758731f76\") "
Sep 10 19:31:56.169938 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.169186    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a02f4a9-bd86-4658-aa0a-a82758731f76-istio-envoy\") pod \"1a02f4a9-bd86-4658-aa0a-a82758731f76\" (UID: \"1a02f4a9-bd86-4658-aa0a-a82758731f76\") "
Sep 10 19:31:56.169938 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.169532    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:56.169938 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.169671    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a02f4a9-bd86-4658-aa0a-a82758731f76-credential-socket\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:31:56.169938 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.169701    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a02f4a9-bd86-4658-aa0a-a82758731f76-workload-socket\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:31:56.169938 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.169716    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a02f4a9-bd86-4658-aa0a-a82758731f76-istiod-ca-cert\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:31:56.170810 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.170487    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a02f4a9-bd86-4658-aa0a-a82758731f76-istio-data" (OuterVolumeSpecName: "istio-data") pod "1a02f4a9-bd86-4658-aa0a-a82758731f76" (UID: "1a02f4a9-bd86-4658-aa0a-a82758731f76"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:31:56.170875 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.170808    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a02f4a9-bd86-4658-aa0a-a82758731f76-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1a02f4a9-bd86-4658-aa0a-a82758731f76" (UID: "1a02f4a9-bd86-4658-aa0a-a82758731f76"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:31:56.171974 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.171916    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a02f4a9-bd86-4658-aa0a-a82758731f76-istio-token" (OuterVolumeSpecName: "istio-token") pod "1a02f4a9-bd86-4658-aa0a-a82758731f76" (UID: "1a02f4a9-bd86-4658-aa0a-a82758731f76"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:31:56.173127 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.173102    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a02f4a9-bd86-4658-aa0a-a82758731f76-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1a02f4a9-bd86-4658-aa0a-a82758731f76" (UID: "1a02f4a9-bd86-4658-aa0a-a82758731f76"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:31:56.174725 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.174665    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a02f4a9-bd86-4658-aa0a-a82758731f76-kube-api-access-bf4xk" (OuterVolumeSpecName: "kube-api-access-bf4xk") pod "1a02f4a9-bd86-4658-aa0a-a82758731f76" (UID: "1a02f4a9-bd86-4658-aa0a-a82758731f76"). InnerVolumeSpecName "kube-api-access-bf4xk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:31:56.175109 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.175081    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1a02f4a9-bd86-4658-aa0a-a82758731f76-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1a02f4a9-bd86-4658-aa0a-a82758731f76" (UID: "1a02f4a9-bd86-4658-aa0a-a82758731f76"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 19:31:56.266273 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.266199    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:56.270391 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.270361    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a02f4a9-bd86-4658-aa0a-a82758731f76-istio-token\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:31:56.270391 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.270385    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a02f4a9-bd86-4658-aa0a-a82758731f76-istio-podinfo\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:31:56.270562 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.270395    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bf4xk\" (UniqueName: \"kubernetes.io/projected/1a02f4a9-bd86-4658-aa0a-a82758731f76-kube-api-access-bf4xk\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:31:56.270562 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.270407    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a02f4a9-bd86-4658-aa0a-a82758731f76-workload-certs\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:31:56.270562 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.270415    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a02f4a9-bd86-4658-aa0a-a82758731f76-istio-data\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:31:56.270562 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.270424    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a02f4a9-bd86-4658-aa0a-a82758731f76-istio-envoy\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:31:56.638794 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.638702    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:56.739737 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.739702    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:56.769489 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.769453    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-9a4e54-85bc97b69-qnr2v"]
Sep 10 19:31:56.769940 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.769925    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a02f4a9-bd86-4658-aa0a-a82758731f76" containerName="istio-proxy"
Sep 10 19:31:56.769987 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.769943    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a02f4a9-bd86-4658-aa0a-a82758731f76" containerName="istio-proxy"
Sep 10 19:31:56.769987 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.769955    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a02f4a9-bd86-4658-aa0a-a82758731f76" containerName="istio-proxy"
Sep 10 19:31:56.769987 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.769961    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a02f4a9-bd86-4658-aa0a-a82758731f76" containerName="istio-proxy"
Sep 10 19:31:56.770082 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.770040    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a02f4a9-bd86-4658-aa0a-a82758731f76" containerName="istio-proxy"
Sep 10 19:31:56.773503 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.773486    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-9a4e54-85bc97b69-qnr2v"
Sep 10 19:31:56.775444 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.775422    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rf7gn\" (UniqueName: \"kubernetes.io/projected/8a719d26-9394-4381-938d-c45a85db658c-kube-api-access-rf7gn\") pod \"payload-processing-e2e-explicit-9a4e54-85bc97b69-qnr2v\" (UID: \"8a719d26-9394-4381-938d-c45a85db658c\") " pod="openshift-ingress/payload-processing-e2e-explicit-9a4e54-85bc97b69-qnr2v"
Sep 10 19:31:56.775546 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.775488    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a719d26-9394-4381-938d-c45a85db658c-plugins-config-volume\") pod \"payload-processing-e2e-explicit-9a4e54-85bc97b69-qnr2v\" (UID: \"8a719d26-9394-4381-938d-c45a85db658c\") " pod="openshift-ingress/payload-processing-e2e-explicit-9a4e54-85bc97b69-qnr2v"
Sep 10 19:31:56.775934 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.775913    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-9a4e54\""
Sep 10 19:31:56.776020 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.775919    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-9a4e54-dockercfg-md5qd\""
Sep 10 19:31:56.784023 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.783994    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-9a4e54-85bc97b69-qnr2v"]
Sep 10 19:31:56.829600 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.829566    2564 generic.go:356] "Generic (PLEG): container finished" podID="1a02f4a9-bd86-4658-aa0a-a82758731f76" containerID="b8d0baf5c584f74e1aa02444d6ba7138b85ee3396c1d248b1fe5bbed846f5e94" exitCode=0
Sep 10 19:31:56.829817 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.829633    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b"
Sep 10 19:31:56.829817 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.829653    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b" event={"ID":"1a02f4a9-bd86-4658-aa0a-a82758731f76","Type":"ContainerDied","Data":"b8d0baf5c584f74e1aa02444d6ba7138b85ee3396c1d248b1fe5bbed846f5e94"}
Sep 10 19:31:56.829817 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.829743    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b" event={"ID":"1a02f4a9-bd86-4658-aa0a-a82758731f76","Type":"ContainerDied","Data":"ae7306c5d82c336f0a2ae4391f62f200679aa3655abc4506c900692dacf49c40"}
Sep 10 19:31:56.829817 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.829786    2564 scope.go:117] "RemoveContainer" containerID="b8d0baf5c584f74e1aa02444d6ba7138b85ee3396c1d248b1fe5bbed846f5e94"
Sep 10 19:31:56.830604 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.830303    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-9a4e54-7c48d9d566-92wlr"]
Sep 10 19:31:56.831124 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.831100    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a02f4a9-bd86-4658-aa0a-a82758731f76" containerName="istio-proxy"
Sep 10 19:31:56.834560 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.834543    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-9a4e54-7c48d9d566-92wlr"
Sep 10 19:31:56.840781 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.840757    2564 scope.go:117] "RemoveContainer" containerID="0719669fed48400ec1ffee0ad0988611190291cfd6751063730d5e6835b052a3"
Sep 10 19:31:56.844149 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.844130    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-9a4e54-7c48d9d566-92wlr"]
Sep 10 19:31:56.852146 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.852125    2564 scope.go:117] "RemoveContainer" containerID="b8d0baf5c584f74e1aa02444d6ba7138b85ee3396c1d248b1fe5bbed846f5e94"
Sep 10 19:31:56.852498 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:31:56.852473    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8d0baf5c584f74e1aa02444d6ba7138b85ee3396c1d248b1fe5bbed846f5e94\": container with ID starting with b8d0baf5c584f74e1aa02444d6ba7138b85ee3396c1d248b1fe5bbed846f5e94 not found: ID does not exist" containerID="b8d0baf5c584f74e1aa02444d6ba7138b85ee3396c1d248b1fe5bbed846f5e94"
Sep 10 19:31:56.852581 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.852505    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8d0baf5c584f74e1aa02444d6ba7138b85ee3396c1d248b1fe5bbed846f5e94"} err="failed to get container status \"b8d0baf5c584f74e1aa02444d6ba7138b85ee3396c1d248b1fe5bbed846f5e94\": rpc error: code = NotFound desc = could not find container \"b8d0baf5c584f74e1aa02444d6ba7138b85ee3396c1d248b1fe5bbed846f5e94\": container with ID starting with b8d0baf5c584f74e1aa02444d6ba7138b85ee3396c1d248b1fe5bbed846f5e94 not found: ID does not exist"
Sep 10 19:31:56.852581 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.852523    2564 scope.go:117] "RemoveContainer" containerID="0719669fed48400ec1ffee0ad0988611190291cfd6751063730d5e6835b052a3"
Sep 10 19:31:56.852851 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:31:56.852828    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0719669fed48400ec1ffee0ad0988611190291cfd6751063730d5e6835b052a3\": container with ID starting with 0719669fed48400ec1ffee0ad0988611190291cfd6751063730d5e6835b052a3 not found: ID does not exist" containerID="0719669fed48400ec1ffee0ad0988611190291cfd6751063730d5e6835b052a3"
Sep 10 19:31:56.852935 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.852860    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0719669fed48400ec1ffee0ad0988611190291cfd6751063730d5e6835b052a3"} err="failed to get container status \"0719669fed48400ec1ffee0ad0988611190291cfd6751063730d5e6835b052a3\": rpc error: code = NotFound desc = could not find container \"0719669fed48400ec1ffee0ad0988611190291cfd6751063730d5e6835b052a3\": container with ID starting with 0719669fed48400ec1ffee0ad0988611190291cfd6751063730d5e6835b052a3 not found: ID does not exist"
Sep 10 19:31:56.875249 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.875210    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b"]
Sep 10 19:31:56.877239 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.877204    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6p7t\" (UniqueName: \"kubernetes.io/projected/d712cb06-c8d2-4f37-99fe-30e1e3a68e84-kube-api-access-c6p7t\") pod \"payload-pre-processing-e2e-explicit-9a4e54-7c48d9d566-92wlr\" (UID: \"d712cb06-c8d2-4f37-99fe-30e1e3a68e84\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-9a4e54-7c48d9d566-92wlr"
Sep 10 19:31:56.877426 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.877279    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rf7gn\" (UniqueName: \"kubernetes.io/projected/8a719d26-9394-4381-938d-c45a85db658c-kube-api-access-rf7gn\") pod \"payload-processing-e2e-explicit-9a4e54-85bc97b69-qnr2v\" (UID: \"8a719d26-9394-4381-938d-c45a85db658c\") " pod="openshift-ingress/payload-processing-e2e-explicit-9a4e54-85bc97b69-qnr2v"
Sep 10 19:31:56.877426 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.877370    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a719d26-9394-4381-938d-c45a85db658c-plugins-config-volume\") pod \"payload-processing-e2e-explicit-9a4e54-85bc97b69-qnr2v\" (UID: \"8a719d26-9394-4381-938d-c45a85db658c\") " pod="openshift-ingress/payload-processing-e2e-explicit-9a4e54-85bc97b69-qnr2v"
Sep 10 19:31:56.877551 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.877451    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d712cb06-c8d2-4f37-99fe-30e1e3a68e84-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-9a4e54-7c48d9d566-92wlr\" (UID: \"d712cb06-c8d2-4f37-99fe-30e1e3a68e84\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-9a4e54-7c48d9d566-92wlr"
Sep 10 19:31:56.879387 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.879353    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a719d26-9394-4381-938d-c45a85db658c-plugins-config-volume\") pod \"payload-processing-e2e-explicit-9a4e54-85bc97b69-qnr2v\" (UID: \"8a719d26-9394-4381-938d-c45a85db658c\") " pod="openshift-ingress/payload-processing-e2e-explicit-9a4e54-85bc97b69-qnr2v"
Sep 10 19:31:56.880893 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.880863    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-autores-5352fd-openshift-default-5d8df88c5b-z9z6b"]
Sep 10 19:31:56.891764 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.891562    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rf7gn\" (UniqueName: \"kubernetes.io/projected/8a719d26-9394-4381-938d-c45a85db658c-kube-api-access-rf7gn\") pod \"payload-processing-e2e-explicit-9a4e54-85bc97b69-qnr2v\" (UID: \"8a719d26-9394-4381-938d-c45a85db658c\") " pod="openshift-ingress/payload-processing-e2e-explicit-9a4e54-85bc97b69-qnr2v"
Sep 10 19:31:56.950270 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.950234    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a02f4a9-bd86-4658-aa0a-a82758731f76" path="/var/lib/kubelet/pods/1a02f4a9-bd86-4658-aa0a-a82758731f76/volumes"
Sep 10 19:31:56.978362 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.978324    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d712cb06-c8d2-4f37-99fe-30e1e3a68e84-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-9a4e54-7c48d9d566-92wlr\" (UID: \"d712cb06-c8d2-4f37-99fe-30e1e3a68e84\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-9a4e54-7c48d9d566-92wlr"
Sep 10 19:31:56.978508 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.978412    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6p7t\" (UniqueName: \"kubernetes.io/projected/d712cb06-c8d2-4f37-99fe-30e1e3a68e84-kube-api-access-c6p7t\") pod \"payload-pre-processing-e2e-explicit-9a4e54-7c48d9d566-92wlr\" (UID: \"d712cb06-c8d2-4f37-99fe-30e1e3a68e84\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-9a4e54-7c48d9d566-92wlr"
Sep 10 19:31:56.979075 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.979053    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d712cb06-c8d2-4f37-99fe-30e1e3a68e84-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-9a4e54-7c48d9d566-92wlr\" (UID: \"d712cb06-c8d2-4f37-99fe-30e1e3a68e84\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-9a4e54-7c48d9d566-92wlr"
Sep 10 19:31:56.990003 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:56.989982    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6p7t\" (UniqueName: \"kubernetes.io/projected/d712cb06-c8d2-4f37-99fe-30e1e3a68e84-kube-api-access-c6p7t\") pod \"payload-pre-processing-e2e-explicit-9a4e54-7c48d9d566-92wlr\" (UID: \"d712cb06-c8d2-4f37-99fe-30e1e3a68e84\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-9a4e54-7c48d9d566-92wlr"
Sep 10 19:31:57.085521 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:57.085478    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-9a4e54-85bc97b69-qnr2v"
Sep 10 19:31:57.100689 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:57.100652    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:57.148305 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:57.148166    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-9a4e54-7c48d9d566-92wlr"
Sep 10 19:31:57.364899 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:57.364854    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-9a4e54-85bc97b69-qnr2v"]
Sep 10 19:31:57.367510 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:31:57.367480    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a719d26_9394_4381_938d_c45a85db658c.slice/crio-50819f43fef668d4397e10f229698ad517b17aa8ebe7f7629c08e302f3164252 WatchSource:0}: Error finding container 50819f43fef668d4397e10f229698ad517b17aa8ebe7f7629c08e302f3164252: Status 404 returned error can't find the container with id 50819f43fef668d4397e10f229698ad517b17aa8ebe7f7629c08e302f3164252
Sep 10 19:31:57.443216 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:57.443186    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-9a4e54-7c48d9d566-92wlr"]
Sep 10 19:31:57.445002 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:31:57.444973    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd712cb06_c8d2_4f37_99fe_30e1e3a68e84.slice/crio-dec4ad839fe0919cc2ff46acb9c0e4d56fb5c86359b4a9d09de519690da3f300 WatchSource:0}: Error finding container dec4ad839fe0919cc2ff46acb9c0e4d56fb5c86359b4a9d09de519690da3f300: Status 404 returned error can't find the container with id dec4ad839fe0919cc2ff46acb9c0e4d56fb5c86359b4a9d09de519690da3f300
Sep 10 19:31:57.837634 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:57.837579    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-9a4e54-7c48d9d566-92wlr" event={"ID":"d712cb06-c8d2-4f37-99fe-30e1e3a68e84","Type":"ContainerStarted","Data":"18341a5ed8948169f93b9ef593d95ad4635420738d7d2a169dfd348e2d4b5b82"}
Sep 10 19:31:57.837853 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:57.837654    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-9a4e54-7c48d9d566-92wlr" event={"ID":"d712cb06-c8d2-4f37-99fe-30e1e3a68e84","Type":"ContainerStarted","Data":"dec4ad839fe0919cc2ff46acb9c0e4d56fb5c86359b4a9d09de519690da3f300"}
Sep 10 19:31:57.837853 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:57.837692    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-9a4e54-7c48d9d566-92wlr"
Sep 10 19:31:57.839165 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:57.839135    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-9a4e54-85bc97b69-qnr2v" event={"ID":"8a719d26-9394-4381-938d-c45a85db658c","Type":"ContainerStarted","Data":"8f1e482ad46de8c5edc5ee21c94d1bfc715cdba98b35e595425e6b8af9953c10"}
Sep 10 19:31:57.839280 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:57.839192    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-9a4e54-85bc97b69-qnr2v" event={"ID":"8a719d26-9394-4381-938d-c45a85db658c","Type":"ContainerStarted","Data":"50819f43fef668d4397e10f229698ad517b17aa8ebe7f7629c08e302f3164252"}
Sep 10 19:31:57.839280 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:57.839223    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-explicit-9a4e54-85bc97b69-qnr2v"
Sep 10 19:31:57.855974 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:57.855927    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-explicit-9a4e54-7c48d9d566-92wlr" podStartSLOduration=1.855911782 podStartE2EDuration="1.855911782s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:31:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:31:57.855132425 +0000 UTC m=+973.456039666" watchObservedRunningTime="2026-09-10 19:31:57.855911782 +0000 UTC m=+973.456819021"
Sep 10 19:31:57.874851 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:57.874793    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-explicit-9a4e54-85bc97b69-qnr2v" podStartSLOduration=1.874771134 podStartE2EDuration="1.874771134s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:31:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:31:57.871927435 +0000 UTC m=+973.472834676" watchObservedRunningTime="2026-09-10 19:31:57.874771134 +0000 UTC m=+973.475678375"
Sep 10 19:31:57.959262 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:31:57.959229    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:02.802816 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:02.802778    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:03.534976 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:03.534935    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:04.233018 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:04.232979    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:04.323194 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:04.323159    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:04.381450 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:04.381405    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:04.481917 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:04.481881    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:04.591307 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:04.591275    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:04.688175 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:04.688135    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:05.399370 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:05.399313    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:05.492637 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:05.492579    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:05.593774 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:05.593736    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:06.991915 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:06.991878    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:08.846403 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:08.846372    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-9a4e54-7c48d9d566-92wlr"
Sep 10 19:32:08.846898 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:08.846452    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-explicit-9a4e54-85bc97b69-qnr2v"
Sep 10 19:32:13.490285 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:13.490231    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:13.549427 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:13.549393    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:13.620493 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:13.620452    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:13.724946 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:13.724905    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:13.827044 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:13.827006    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:14.204249 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:14.204145    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:14.942071 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:14.942036    2564 generic.go:356] "Generic (PLEG): container finished" podID="27890338-9e2e-40c3-a800-ee4f7e2030e2" containerID="ce8c7785a3788b1020897c250d2de60341942f81572b76c4dd3ab8b5006dc8f5" exitCode=0
Sep 10 19:32:14.942454 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:14.942112    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j" event={"ID":"27890338-9e2e-40c3-a800-ee4f7e2030e2","Type":"ContainerDied","Data":"ce8c7785a3788b1020897c250d2de60341942f81572b76c4dd3ab8b5006dc8f5"}
Sep 10 19:32:14.945375 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:14.945353    2564 scope.go:117] "RemoveContainer" containerID="ce8c7785a3788b1020897c250d2de60341942f81572b76c4dd3ab8b5006dc8f5"
Sep 10 19:32:14.946501 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:14.946471    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:32:14.946582 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:14.946559    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:32:14.946641 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:14.946598    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:32:15.949481 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:15.949444    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j" event={"ID":"27890338-9e2e-40c3-a800-ee4f7e2030e2","Type":"ContainerStarted","Data":"2239254da28b0e906850fa860ea03c9feb2f277c8065ca3ac8040a55426cdfe0"}
Sep 10 19:32:16.807891 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:16.807850    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j"
Sep 10 19:32:16.808213 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:16.808191    2564 patch_prober.go:28] interesting pod/e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Sep 10 19:32:16.808263 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:16.808237    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j" podUID="27890338-9e2e-40c3-a800-ee4f7e2030e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Sep 10 19:32:17.108956 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:17.108864    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:17.807864 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:17.807823    2564 patch_prober.go:28] interesting pod/e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Sep 10 19:32:17.808072 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:17.807891    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j" podUID="27890338-9e2e-40c3-a800-ee4f7e2030e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Sep 10 19:32:18.398639 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:18.398568    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:18.807422 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:18.807385    2564 patch_prober.go:28] interesting pod/e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Sep 10 19:32:18.807591 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:18.807447    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j" podUID="27890338-9e2e-40c3-a800-ee4f7e2030e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Sep 10 19:32:19.697593 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:19.697555    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:19.808292 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:19.808255    2564 patch_prober.go:28] interesting pod/e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Sep 10 19:32:19.808489 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:19.808321    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j" podUID="27890338-9e2e-40c3-a800-ee4f7e2030e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Sep 10 19:32:20.807725 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:20.807691    2564 patch_prober.go:28] interesting pod/e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Sep 10 19:32:20.808113 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:20.807763    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j" podUID="27890338-9e2e-40c3-a800-ee4f7e2030e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Sep 10 19:32:21.811998 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:21.811966    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j"
Sep 10 19:32:21.812417 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:21.812362    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j"
Sep 10 19:32:21.813084 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:21.813068    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j"
Sep 10 19:32:22.945640 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:22.945544    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:37.673936 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:37.673899    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:39.966309 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:39.966267    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:40.196185 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:40.196151    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:43.481046 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:43.481006    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:47.196331 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:47.196291    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:47.262045 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:47.262005    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:47.376558 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:47.376525    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:47.472721 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:47.472602    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:50.669527 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:50.669488    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:50.709849 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:50.709816    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-9a4e54-85bc97b69-qnr2v"]
Sep 10 19:32:50.710115 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:50.710087    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-explicit-9a4e54-85bc97b69-qnr2v" podUID="8a719d26-9394-4381-938d-c45a85db658c" containerName="payload-processing" containerID="cri-o://8f1e482ad46de8c5edc5ee21c94d1bfc715cdba98b35e595425e6b8af9953c10" gracePeriod=30
Sep 10 19:32:50.715543 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:50.715509    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-9a4e54-7c48d9d566-92wlr"]
Sep 10 19:32:50.715868 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:50.715839    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-explicit-9a4e54-7c48d9d566-92wlr" podUID="d712cb06-c8d2-4f37-99fe-30e1e3a68e84" containerName="payload-pre-processing" containerID="cri-o://18341a5ed8948169f93b9ef593d95ad4635420738d7d2a169dfd348e2d4b5b82" gracePeriod=30
Sep 10 19:32:50.744556 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:50.744392    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:50.814925 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:50.814882    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:50.973140 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:50.972935    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:50.996600 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:50.996575    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-9a4e54-85bc97b69-qnr2v"
Sep 10 19:32:51.014360 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:51.014337    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-9a4e54-7c48d9d566-92wlr"
Sep 10 19:32:51.027522 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:51.027492    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a719d26-9394-4381-938d-c45a85db658c-plugins-config-volume\") pod \"8a719d26-9394-4381-938d-c45a85db658c\" (UID: \"8a719d26-9394-4381-938d-c45a85db658c\") "
Sep 10 19:32:51.027698 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:51.027653    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rf7gn\" (UniqueName: \"kubernetes.io/projected/8a719d26-9394-4381-938d-c45a85db658c-kube-api-access-rf7gn\") pod \"8a719d26-9394-4381-938d-c45a85db658c\" (UID: \"8a719d26-9394-4381-938d-c45a85db658c\") "
Sep 10 19:32:51.027847 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:51.027825    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a719d26-9394-4381-938d-c45a85db658c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8a719d26-9394-4381-938d-c45a85db658c" (UID: "8a719d26-9394-4381-938d-c45a85db658c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:32:51.028073 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:51.028052    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a719d26-9394-4381-938d-c45a85db658c-plugins-config-volume\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:32:51.029877 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:51.029848    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a719d26-9394-4381-938d-c45a85db658c-kube-api-access-rf7gn" (OuterVolumeSpecName: "kube-api-access-rf7gn") pod "8a719d26-9394-4381-938d-c45a85db658c" (UID: "8a719d26-9394-4381-938d-c45a85db658c"). InnerVolumeSpecName "kube-api-access-rf7gn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:32:51.036495 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:51.036470    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:51.128787 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:51.128752    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d712cb06-c8d2-4f37-99fe-30e1e3a68e84-plugins-config-volume\") pod \"d712cb06-c8d2-4f37-99fe-30e1e3a68e84\" (UID: \"d712cb06-c8d2-4f37-99fe-30e1e3a68e84\") "
Sep 10 19:32:51.128966 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:51.128843    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c6p7t\" (UniqueName: \"kubernetes.io/projected/d712cb06-c8d2-4f37-99fe-30e1e3a68e84-kube-api-access-c6p7t\") pod \"d712cb06-c8d2-4f37-99fe-30e1e3a68e84\" (UID: \"d712cb06-c8d2-4f37-99fe-30e1e3a68e84\") "
Sep 10 19:32:51.129066 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:51.129055    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rf7gn\" (UniqueName: \"kubernetes.io/projected/8a719d26-9394-4381-938d-c45a85db658c-kube-api-access-rf7gn\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:32:51.129192 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:51.129166    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d712cb06-c8d2-4f37-99fe-30e1e3a68e84-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d712cb06-c8d2-4f37-99fe-30e1e3a68e84" (UID: "d712cb06-c8d2-4f37-99fe-30e1e3a68e84"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:32:51.130918 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:51.130886    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d712cb06-c8d2-4f37-99fe-30e1e3a68e84-kube-api-access-c6p7t" (OuterVolumeSpecName: "kube-api-access-c6p7t") pod "d712cb06-c8d2-4f37-99fe-30e1e3a68e84" (UID: "d712cb06-c8d2-4f37-99fe-30e1e3a68e84"). InnerVolumeSpecName "kube-api-access-c6p7t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:32:51.159651 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:51.159586    2564 generic.go:356] "Generic (PLEG): container finished" podID="d712cb06-c8d2-4f37-99fe-30e1e3a68e84" containerID="18341a5ed8948169f93b9ef593d95ad4635420738d7d2a169dfd348e2d4b5b82" exitCode=0
Sep 10 19:32:51.159819 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:51.159659    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-9a4e54-7c48d9d566-92wlr"
Sep 10 19:32:51.159819 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:51.159680    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-9a4e54-7c48d9d566-92wlr" event={"ID":"d712cb06-c8d2-4f37-99fe-30e1e3a68e84","Type":"ContainerDied","Data":"18341a5ed8948169f93b9ef593d95ad4635420738d7d2a169dfd348e2d4b5b82"}
Sep 10 19:32:51.159819 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:51.159747    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-9a4e54-7c48d9d566-92wlr" event={"ID":"d712cb06-c8d2-4f37-99fe-30e1e3a68e84","Type":"ContainerDied","Data":"dec4ad839fe0919cc2ff46acb9c0e4d56fb5c86359b4a9d09de519690da3f300"}
Sep 10 19:32:51.159819 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:51.159777    2564 scope.go:117] "RemoveContainer" containerID="18341a5ed8948169f93b9ef593d95ad4635420738d7d2a169dfd348e2d4b5b82"
Sep 10 19:32:51.161826 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:51.161804    2564 generic.go:356] "Generic (PLEG): container finished" podID="8a719d26-9394-4381-938d-c45a85db658c" containerID="8f1e482ad46de8c5edc5ee21c94d1bfc715cdba98b35e595425e6b8af9953c10" exitCode=0
Sep 10 19:32:51.161998 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:51.161974    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-9a4e54-85bc97b69-qnr2v" event={"ID":"8a719d26-9394-4381-938d-c45a85db658c","Type":"ContainerDied","Data":"8f1e482ad46de8c5edc5ee21c94d1bfc715cdba98b35e595425e6b8af9953c10"}
Sep 10 19:32:51.162131 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:51.162027    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-9a4e54-85bc97b69-qnr2v" event={"ID":"8a719d26-9394-4381-938d-c45a85db658c","Type":"ContainerDied","Data":"50819f43fef668d4397e10f229698ad517b17aa8ebe7f7629c08e302f3164252"}
Sep 10 19:32:51.162131 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:51.162094    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-9a4e54-85bc97b69-qnr2v"
Sep 10 19:32:51.180351 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:51.180331    2564 scope.go:117] "RemoveContainer" containerID="18341a5ed8948169f93b9ef593d95ad4635420738d7d2a169dfd348e2d4b5b82"
Sep 10 19:32:51.180675 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:32:51.180652    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18341a5ed8948169f93b9ef593d95ad4635420738d7d2a169dfd348e2d4b5b82\": container with ID starting with 18341a5ed8948169f93b9ef593d95ad4635420738d7d2a169dfd348e2d4b5b82 not found: ID does not exist" containerID="18341a5ed8948169f93b9ef593d95ad4635420738d7d2a169dfd348e2d4b5b82"
Sep 10 19:32:51.180773 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:51.180681    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18341a5ed8948169f93b9ef593d95ad4635420738d7d2a169dfd348e2d4b5b82"} err="failed to get container status \"18341a5ed8948169f93b9ef593d95ad4635420738d7d2a169dfd348e2d4b5b82\": rpc error: code = NotFound desc = could not find container \"18341a5ed8948169f93b9ef593d95ad4635420738d7d2a169dfd348e2d4b5b82\": container with ID starting with 18341a5ed8948169f93b9ef593d95ad4635420738d7d2a169dfd348e2d4b5b82 not found: ID does not exist"
Sep 10 19:32:51.180773 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:51.180699    2564 scope.go:117] "RemoveContainer" containerID="8f1e482ad46de8c5edc5ee21c94d1bfc715cdba98b35e595425e6b8af9953c10"
Sep 10 19:32:51.189100 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:51.189069    2564 scope.go:117] "RemoveContainer" containerID="8f1e482ad46de8c5edc5ee21c94d1bfc715cdba98b35e595425e6b8af9953c10"
Sep 10 19:32:51.189726 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:32:51.189699    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f1e482ad46de8c5edc5ee21c94d1bfc715cdba98b35e595425e6b8af9953c10\": container with ID starting with 8f1e482ad46de8c5edc5ee21c94d1bfc715cdba98b35e595425e6b8af9953c10 not found: ID does not exist" containerID="8f1e482ad46de8c5edc5ee21c94d1bfc715cdba98b35e595425e6b8af9953c10"
Sep 10 19:32:51.189817 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:51.189731    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f1e482ad46de8c5edc5ee21c94d1bfc715cdba98b35e595425e6b8af9953c10"} err="failed to get container status \"8f1e482ad46de8c5edc5ee21c94d1bfc715cdba98b35e595425e6b8af9953c10\": rpc error: code = NotFound desc = could not find container \"8f1e482ad46de8c5edc5ee21c94d1bfc715cdba98b35e595425e6b8af9953c10\": container with ID starting with 8f1e482ad46de8c5edc5ee21c94d1bfc715cdba98b35e595425e6b8af9953c10 not found: ID does not exist"
Sep 10 19:32:51.191573 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:51.191551    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-9a4e54-85bc97b69-qnr2v"]
Sep 10 19:32:51.195965 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:51.195943    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-9a4e54-85bc97b69-qnr2v"]
Sep 10 19:32:51.208652 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:51.208631    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-9a4e54-7c48d9d566-92wlr"]
Sep 10 19:32:51.214614 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:51.214591    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-9a4e54-7c48d9d566-92wlr"]
Sep 10 19:32:51.230162 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:51.230139    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d712cb06-c8d2-4f37-99fe-30e1e3a68e84-plugins-config-volume\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:32:51.230162 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:51.230162    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c6p7t\" (UniqueName: \"kubernetes.io/projected/d712cb06-c8d2-4f37-99fe-30e1e3a68e84-kube-api-access-c6p7t\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:32:51.369524 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:51.369454    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:52.949312 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:52.949279    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a719d26-9394-4381-938d-c45a85db658c" path="/var/lib/kubelet/pods/8a719d26-9394-4381-938d-c45a85db658c/volumes"
Sep 10 19:32:52.949716 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:52.949659    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d712cb06-c8d2-4f37-99fe-30e1e3a68e84" path="/var/lib/kubelet/pods/d712cb06-c8d2-4f37-99fe-30e1e3a68e84/volumes"
Sep 10 19:32:57.973214 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:57.973175    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:58.035304 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:58.035258    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:58.233192 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:58.233109    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j"]
Sep 10 19:32:58.233376 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:58.233333    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j" podUID="27890338-9e2e-40c3-a800-ee4f7e2030e2" containerName="istio-proxy" containerID="cri-o://2239254da28b0e906850fa860ea03c9feb2f277c8065ca3ac8040a55426cdfe0" gracePeriod=30
Sep 10 19:32:58.250450 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:58.250419    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:58.692455 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:58.692416    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:59.001889 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:59.001846    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:59.291665 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:32:59.291539    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:00.468356 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:00.468317    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:00.563052 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:00.563000    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:00.694666 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:00.694609    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:03.351438 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:03.351332    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:03.496167 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:03.496124    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j"
Sep 10 19:33:03.654887 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:03.654799    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27890338-9e2e-40c3-a800-ee4f7e2030e2-workload-certs\") pod \"27890338-9e2e-40c3-a800-ee4f7e2030e2\" (UID: \"27890338-9e2e-40c3-a800-ee4f7e2030e2\") "
Sep 10 19:33:03.654887 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:03.654837    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27890338-9e2e-40c3-a800-ee4f7e2030e2-workload-socket\") pod \"27890338-9e2e-40c3-a800-ee4f7e2030e2\" (UID: \"27890338-9e2e-40c3-a800-ee4f7e2030e2\") "
Sep 10 19:33:03.654887 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:03.654860    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27890338-9e2e-40c3-a800-ee4f7e2030e2-istio-envoy\") pod \"27890338-9e2e-40c3-a800-ee4f7e2030e2\" (UID: \"27890338-9e2e-40c3-a800-ee4f7e2030e2\") "
Sep 10 19:33:03.655168 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:03.654911    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27890338-9e2e-40c3-a800-ee4f7e2030e2-istiod-ca-cert\") pod \"27890338-9e2e-40c3-a800-ee4f7e2030e2\" (UID: \"27890338-9e2e-40c3-a800-ee4f7e2030e2\") "
Sep 10 19:33:03.655168 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:03.654944    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27890338-9e2e-40c3-a800-ee4f7e2030e2-istio-token\") pod \"27890338-9e2e-40c3-a800-ee4f7e2030e2\" (UID: \"27890338-9e2e-40c3-a800-ee4f7e2030e2\") "
Sep 10 19:33:03.655168 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:03.654959    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27890338-9e2e-40c3-a800-ee4f7e2030e2-istio-podinfo\") pod \"27890338-9e2e-40c3-a800-ee4f7e2030e2\" (UID: \"27890338-9e2e-40c3-a800-ee4f7e2030e2\") "
Sep 10 19:33:03.655168 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:03.654993    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27890338-9e2e-40c3-a800-ee4f7e2030e2-credential-socket\") pod \"27890338-9e2e-40c3-a800-ee4f7e2030e2\" (UID: \"27890338-9e2e-40c3-a800-ee4f7e2030e2\") "
Sep 10 19:33:03.655168 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:03.655049    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27890338-9e2e-40c3-a800-ee4f7e2030e2-istio-data\") pod \"27890338-9e2e-40c3-a800-ee4f7e2030e2\" (UID: \"27890338-9e2e-40c3-a800-ee4f7e2030e2\") "
Sep 10 19:33:03.655168 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:03.655074    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jgfz9\" (UniqueName: \"kubernetes.io/projected/27890338-9e2e-40c3-a800-ee4f7e2030e2-kube-api-access-jgfz9\") pod \"27890338-9e2e-40c3-a800-ee4f7e2030e2\" (UID: \"27890338-9e2e-40c3-a800-ee4f7e2030e2\") "
Sep 10 19:33:03.655168 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:03.655145    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27890338-9e2e-40c3-a800-ee4f7e2030e2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "27890338-9e2e-40c3-a800-ee4f7e2030e2" (UID: "27890338-9e2e-40c3-a800-ee4f7e2030e2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:33:03.655503 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:03.655142    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27890338-9e2e-40c3-a800-ee4f7e2030e2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "27890338-9e2e-40c3-a800-ee4f7e2030e2" (UID: "27890338-9e2e-40c3-a800-ee4f7e2030e2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:33:03.655503 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:03.655313    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27890338-9e2e-40c3-a800-ee4f7e2030e2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "27890338-9e2e-40c3-a800-ee4f7e2030e2" (UID: "27890338-9e2e-40c3-a800-ee4f7e2030e2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:33:03.655503 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:03.655354    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27890338-9e2e-40c3-a800-ee4f7e2030e2-istio-data" (OuterVolumeSpecName: "istio-data") pod "27890338-9e2e-40c3-a800-ee4f7e2030e2" (UID: "27890338-9e2e-40c3-a800-ee4f7e2030e2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:33:03.655503 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:03.655407    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27890338-9e2e-40c3-a800-ee4f7e2030e2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "27890338-9e2e-40c3-a800-ee4f7e2030e2" (UID: "27890338-9e2e-40c3-a800-ee4f7e2030e2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:33:03.655503 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:03.655494    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27890338-9e2e-40c3-a800-ee4f7e2030e2-credential-socket\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:33:03.655780 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:03.655511    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27890338-9e2e-40c3-a800-ee4f7e2030e2-istio-data\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:33:03.655780 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:03.655527    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27890338-9e2e-40c3-a800-ee4f7e2030e2-workload-certs\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:33:03.655780 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:03.655539    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27890338-9e2e-40c3-a800-ee4f7e2030e2-workload-socket\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:33:03.657255 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:03.657228    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27890338-9e2e-40c3-a800-ee4f7e2030e2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "27890338-9e2e-40c3-a800-ee4f7e2030e2" (UID: "27890338-9e2e-40c3-a800-ee4f7e2030e2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:33:03.657363 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:03.657253    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/27890338-9e2e-40c3-a800-ee4f7e2030e2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "27890338-9e2e-40c3-a800-ee4f7e2030e2" (UID: "27890338-9e2e-40c3-a800-ee4f7e2030e2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 19:33:03.657363 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:03.657339    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27890338-9e2e-40c3-a800-ee4f7e2030e2-kube-api-access-jgfz9" (OuterVolumeSpecName: "kube-api-access-jgfz9") pod "27890338-9e2e-40c3-a800-ee4f7e2030e2" (UID: "27890338-9e2e-40c3-a800-ee4f7e2030e2"). InnerVolumeSpecName "kube-api-access-jgfz9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:33:03.657363 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:03.657352    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27890338-9e2e-40c3-a800-ee4f7e2030e2-istio-token" (OuterVolumeSpecName: "istio-token") pod "27890338-9e2e-40c3-a800-ee4f7e2030e2" (UID: "27890338-9e2e-40c3-a800-ee4f7e2030e2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:33:03.757046 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:03.757008    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27890338-9e2e-40c3-a800-ee4f7e2030e2-istiod-ca-cert\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:33:03.757046 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:03.757040    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27890338-9e2e-40c3-a800-ee4f7e2030e2-istio-token\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:33:03.757046 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:03.757050    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27890338-9e2e-40c3-a800-ee4f7e2030e2-istio-podinfo\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:33:03.757292 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:03.757061    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jgfz9\" (UniqueName: \"kubernetes.io/projected/27890338-9e2e-40c3-a800-ee4f7e2030e2-kube-api-access-jgfz9\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:33:03.757292 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:03.757071    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27890338-9e2e-40c3-a800-ee4f7e2030e2-istio-envoy\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:33:04.236838 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:04.236801    2564 generic.go:356] "Generic (PLEG): container finished" podID="27890338-9e2e-40c3-a800-ee4f7e2030e2" containerID="2239254da28b0e906850fa860ea03c9feb2f277c8065ca3ac8040a55426cdfe0" exitCode=0
Sep 10 19:33:04.237011 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:04.236858    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j"
Sep 10 19:33:04.237011 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:04.236892    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j" event={"ID":"27890338-9e2e-40c3-a800-ee4f7e2030e2","Type":"ContainerDied","Data":"2239254da28b0e906850fa860ea03c9feb2f277c8065ca3ac8040a55426cdfe0"}
Sep 10 19:33:04.237011 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:04.236947    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j" event={"ID":"27890338-9e2e-40c3-a800-ee4f7e2030e2","Type":"ContainerDied","Data":"118d2d193620c638464e0e30a37341a6ad450f56191066d7dd516e0cea856540"}
Sep 10 19:33:04.237011 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:04.236978    2564 scope.go:117] "RemoveContainer" containerID="2239254da28b0e906850fa860ea03c9feb2f277c8065ca3ac8040a55426cdfe0"
Sep 10 19:33:04.248894 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:04.248871    2564 scope.go:117] "RemoveContainer" containerID="ce8c7785a3788b1020897c250d2de60341942f81572b76c4dd3ab8b5006dc8f5"
Sep 10 19:33:04.257921 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:04.257894    2564 scope.go:117] "RemoveContainer" containerID="2239254da28b0e906850fa860ea03c9feb2f277c8065ca3ac8040a55426cdfe0"
Sep 10 19:33:04.258247 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:33:04.258228    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2239254da28b0e906850fa860ea03c9feb2f277c8065ca3ac8040a55426cdfe0\": container with ID starting with 2239254da28b0e906850fa860ea03c9feb2f277c8065ca3ac8040a55426cdfe0 not found: ID does not exist" containerID="2239254da28b0e906850fa860ea03c9feb2f277c8065ca3ac8040a55426cdfe0"
Sep 10 19:33:04.258299 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:04.258259    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2239254da28b0e906850fa860ea03c9feb2f277c8065ca3ac8040a55426cdfe0"} err="failed to get container status \"2239254da28b0e906850fa860ea03c9feb2f277c8065ca3ac8040a55426cdfe0\": rpc error: code = NotFound desc = could not find container \"2239254da28b0e906850fa860ea03c9feb2f277c8065ca3ac8040a55426cdfe0\": container with ID starting with 2239254da28b0e906850fa860ea03c9feb2f277c8065ca3ac8040a55426cdfe0 not found: ID does not exist"
Sep 10 19:33:04.258299 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:04.258278    2564 scope.go:117] "RemoveContainer" containerID="ce8c7785a3788b1020897c250d2de60341942f81572b76c4dd3ab8b5006dc8f5"
Sep 10 19:33:04.258564 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:33:04.258544    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce8c7785a3788b1020897c250d2de60341942f81572b76c4dd3ab8b5006dc8f5\": container with ID starting with ce8c7785a3788b1020897c250d2de60341942f81572b76c4dd3ab8b5006dc8f5 not found: ID does not exist" containerID="ce8c7785a3788b1020897c250d2de60341942f81572b76c4dd3ab8b5006dc8f5"
Sep 10 19:33:04.258680 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:04.258567    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce8c7785a3788b1020897c250d2de60341942f81572b76c4dd3ab8b5006dc8f5"} err="failed to get container status \"ce8c7785a3788b1020897c250d2de60341942f81572b76c4dd3ab8b5006dc8f5\": rpc error: code = NotFound desc = could not find container \"ce8c7785a3788b1020897c250d2de60341942f81572b76c4dd3ab8b5006dc8f5\": container with ID starting with ce8c7785a3788b1020897c250d2de60341942f81572b76c4dd3ab8b5006dc8f5 not found: ID does not exist"
Sep 10 19:33:04.266270 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:04.266204    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j"]
Sep 10 19:33:04.268153 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:04.268131    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-9a4e54-openshift-default-6cf4cf44d4-p625j"]
Sep 10 19:33:04.950003 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:04.949971    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="27890338-9e2e-40c3-a800-ee4f7e2030e2" path="/var/lib/kubelet/pods/27890338-9e2e-40c3-a800-ee4f7e2030e2/volumes"
Sep 10 19:33:08.044189 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:08.044152    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:10.287173 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:10.287135    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:10.391615 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:10.391551    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:10.494827 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:10.494792    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:21.039224 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:21.039186    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:21.102029 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:21.101997    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:21.208241 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:21.208192    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:21.460172 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:21.460078    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:21.570498 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:21.570460    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:21.838195 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:21.838157    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:26.652934 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:26.652890    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:28.097773 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:28.097737    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:28.157363 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:28.157330    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:28.230865 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:28.230836    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:28.332227 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:28.332177    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:28.432908 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:28.432818    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:29.072091 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:29.072058    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:29.150743 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:29.150711    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:29.246933 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:29.246901    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:30.791611 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:30.791572    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:37.050889 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:37.050850    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:37.109081 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:37.109046    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:37.753678 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:37.753643    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:44.009321 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:44.009285    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:44.111083 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:44.111052    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:44.249008 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:44.248978    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:44.591574 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:44.591535    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:44.718773 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:44.718740    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:47.474591 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:47.474552    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:47.573195 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:47.573155    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:48.658920 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:48.658874    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:48.797001 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:48.796959    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:48.896287 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:48.896258    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:49.249907 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:49.249871    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:49.349088 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:49.349032    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:49.458024 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:49.457992    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-cecfebe7-84d6d4468b-ckwpq"]
Sep 10 19:33:49.458430 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:49.458417    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27890338-9e2e-40c3-a800-ee4f7e2030e2" containerName="istio-proxy"
Sep 10 19:33:49.458482 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:49.458434    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="27890338-9e2e-40c3-a800-ee4f7e2030e2" containerName="istio-proxy"
Sep 10 19:33:49.458482 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:49.458447    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a719d26-9394-4381-938d-c45a85db658c" containerName="payload-processing"
Sep 10 19:33:49.458482 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:49.458454    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a719d26-9394-4381-938d-c45a85db658c" containerName="payload-processing"
Sep 10 19:33:49.458482 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:49.458469    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27890338-9e2e-40c3-a800-ee4f7e2030e2" containerName="istio-proxy"
Sep 10 19:33:49.458482 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:49.458474    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="27890338-9e2e-40c3-a800-ee4f7e2030e2" containerName="istio-proxy"
Sep 10 19:33:49.458482 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:49.458481    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d712cb06-c8d2-4f37-99fe-30e1e3a68e84" containerName="payload-pre-processing"
Sep 10 19:33:49.458691 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:49.458486    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d712cb06-c8d2-4f37-99fe-30e1e3a68e84" containerName="payload-pre-processing"
Sep 10 19:33:49.458691 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:49.458562    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="27890338-9e2e-40c3-a800-ee4f7e2030e2" containerName="istio-proxy"
Sep 10 19:33:49.458691 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:49.458569    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d712cb06-c8d2-4f37-99fe-30e1e3a68e84" containerName="payload-pre-processing"
Sep 10 19:33:49.458691 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:49.458577    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a719d26-9394-4381-938d-c45a85db658c" containerName="payload-processing"
Sep 10 19:33:49.458691 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:49.458585    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="27890338-9e2e-40c3-a800-ee4f7e2030e2" containerName="istio-proxy"
Sep 10 19:33:49.463361 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:49.463340    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-cecfebe7-84d6d4468b-ckwpq"
Sep 10 19:33:49.465855 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:49.465833    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-cecfebe7\""
Sep 10 19:33:49.479640 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:49.479598    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-cecfebe7-84d6d4468b-ckwpq"]
Sep 10 19:33:49.508929 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:49.508849    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24987d09-fa94-47ea-b149-141b53c11c40-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-cecfebe7-84d6d4468b-ckwpq\" (UID: \"24987d09-fa94-47ea-b149-141b53c11c40\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-cecfebe7-84d6d4468b-ckwpq"
Sep 10 19:33:49.508929 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:49.508892    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zc69s\" (UniqueName: \"kubernetes.io/projected/24987d09-fa94-47ea-b149-141b53c11c40-kube-api-access-zc69s\") pod \"payload-pre-processing-e2e-mt-cecfebe7-84d6d4468b-ckwpq\" (UID: \"24987d09-fa94-47ea-b149-141b53c11c40\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-cecfebe7-84d6d4468b-ckwpq"
Sep 10 19:33:49.531231 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:49.531197    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-cecfebe7-7b4d6cb889-qkctd"]
Sep 10 19:33:49.534632 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:49.534607    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-cecfebe7-7b4d6cb889-qkctd"
Sep 10 19:33:49.537206 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:49.537187    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-cecfebe7-dockercfg-7hmk6\""
Sep 10 19:33:49.544836 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:49.544815    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-cecfebe7-7b4d6cb889-qkctd"]
Sep 10 19:33:49.610526 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:49.610492    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zc69s\" (UniqueName: \"kubernetes.io/projected/24987d09-fa94-47ea-b149-141b53c11c40-kube-api-access-zc69s\") pod \"payload-pre-processing-e2e-mt-cecfebe7-84d6d4468b-ckwpq\" (UID: \"24987d09-fa94-47ea-b149-141b53c11c40\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-cecfebe7-84d6d4468b-ckwpq"
Sep 10 19:33:49.610913 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:49.610884    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/adc68a84-f331-4104-926e-d835d58f443c-plugins-config-volume\") pod \"payload-processing-e2e-mt-cecfebe7-7b4d6cb889-qkctd\" (UID: \"adc68a84-f331-4104-926e-d835d58f443c\") " pod="openshift-ingress/payload-processing-e2e-mt-cecfebe7-7b4d6cb889-qkctd"
Sep 10 19:33:49.611035 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:49.610942    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ngkh\" (UniqueName: \"kubernetes.io/projected/adc68a84-f331-4104-926e-d835d58f443c-kube-api-access-8ngkh\") pod \"payload-processing-e2e-mt-cecfebe7-7b4d6cb889-qkctd\" (UID: \"adc68a84-f331-4104-926e-d835d58f443c\") " pod="openshift-ingress/payload-processing-e2e-mt-cecfebe7-7b4d6cb889-qkctd"
Sep 10 19:33:49.611100 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:49.611073    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24987d09-fa94-47ea-b149-141b53c11c40-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-cecfebe7-84d6d4468b-ckwpq\" (UID: \"24987d09-fa94-47ea-b149-141b53c11c40\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-cecfebe7-84d6d4468b-ckwpq"
Sep 10 19:33:49.612010 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:49.611988    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24987d09-fa94-47ea-b149-141b53c11c40-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-cecfebe7-84d6d4468b-ckwpq\" (UID: \"24987d09-fa94-47ea-b149-141b53c11c40\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-cecfebe7-84d6d4468b-ckwpq"
Sep 10 19:33:49.622341 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:49.622316    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zc69s\" (UniqueName: \"kubernetes.io/projected/24987d09-fa94-47ea-b149-141b53c11c40-kube-api-access-zc69s\") pod \"payload-pre-processing-e2e-mt-cecfebe7-84d6d4468b-ckwpq\" (UID: \"24987d09-fa94-47ea-b149-141b53c11c40\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-cecfebe7-84d6d4468b-ckwpq"
Sep 10 19:33:49.712310 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:49.712275    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/adc68a84-f331-4104-926e-d835d58f443c-plugins-config-volume\") pod \"payload-processing-e2e-mt-cecfebe7-7b4d6cb889-qkctd\" (UID: \"adc68a84-f331-4104-926e-d835d58f443c\") " pod="openshift-ingress/payload-processing-e2e-mt-cecfebe7-7b4d6cb889-qkctd"
Sep 10 19:33:49.712730 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:49.712318    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ngkh\" (UniqueName: \"kubernetes.io/projected/adc68a84-f331-4104-926e-d835d58f443c-kube-api-access-8ngkh\") pod \"payload-processing-e2e-mt-cecfebe7-7b4d6cb889-qkctd\" (UID: \"adc68a84-f331-4104-926e-d835d58f443c\") " pod="openshift-ingress/payload-processing-e2e-mt-cecfebe7-7b4d6cb889-qkctd"
Sep 10 19:33:49.712985 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:49.712965    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/adc68a84-f331-4104-926e-d835d58f443c-plugins-config-volume\") pod \"payload-processing-e2e-mt-cecfebe7-7b4d6cb889-qkctd\" (UID: \"adc68a84-f331-4104-926e-d835d58f443c\") " pod="openshift-ingress/payload-processing-e2e-mt-cecfebe7-7b4d6cb889-qkctd"
Sep 10 19:33:49.720994 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:49.720965    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ngkh\" (UniqueName: \"kubernetes.io/projected/adc68a84-f331-4104-926e-d835d58f443c-kube-api-access-8ngkh\") pod \"payload-processing-e2e-mt-cecfebe7-7b4d6cb889-qkctd\" (UID: \"adc68a84-f331-4104-926e-d835d58f443c\") " pod="openshift-ingress/payload-processing-e2e-mt-cecfebe7-7b4d6cb889-qkctd"
Sep 10 19:33:49.773535 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:49.773450    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-cecfebe7-84d6d4468b-ckwpq"
Sep 10 19:33:49.845700 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:49.845651    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-cecfebe7-7b4d6cb889-qkctd"
Sep 10 19:33:50.045560 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:50.045473    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-cecfebe7-84d6d4468b-ckwpq"]
Sep 10 19:33:50.048592 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:33:50.048565    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24987d09_fa94_47ea_b149_141b53c11c40.slice/crio-4a90e4970597b0380187d97461ce4305a3faa4f8a1dd6cea228901f0bd12fbce WatchSource:0}: Error finding container 4a90e4970597b0380187d97461ce4305a3faa4f8a1dd6cea228901f0bd12fbce: Status 404 returned error can't find the container with id 4a90e4970597b0380187d97461ce4305a3faa4f8a1dd6cea228901f0bd12fbce
Sep 10 19:33:50.125675 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:50.125652    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-cecfebe7-7b4d6cb889-qkctd"]
Sep 10 19:33:50.127521 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:33:50.127494    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadc68a84_f331_4104_926e_d835d58f443c.slice/crio-cde7a8f10aa3559ee91d07cea31087cc1137c939191049367f27147179620d4f WatchSource:0}: Error finding container cde7a8f10aa3559ee91d07cea31087cc1137c939191049367f27147179620d4f: Status 404 returned error can't find the container with id cde7a8f10aa3559ee91d07cea31087cc1137c939191049367f27147179620d4f
Sep 10 19:33:50.495574 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:50.495472    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-cecfebe7-84d6d4468b-ckwpq" event={"ID":"24987d09-fa94-47ea-b149-141b53c11c40","Type":"ContainerStarted","Data":"eafd7ebc61f2929b52f3d6277a2d0dbdf76ca0648f18740615ed1e1c665d91ce"}
Sep 10 19:33:50.495574 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:50.495555    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-cecfebe7-84d6d4468b-ckwpq" event={"ID":"24987d09-fa94-47ea-b149-141b53c11c40","Type":"ContainerStarted","Data":"4a90e4970597b0380187d97461ce4305a3faa4f8a1dd6cea228901f0bd12fbce"}
Sep 10 19:33:50.495834 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:50.495585    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-cecfebe7-84d6d4468b-ckwpq"
Sep 10 19:33:50.497364 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:50.497339    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-cecfebe7-7b4d6cb889-qkctd" event={"ID":"adc68a84-f331-4104-926e-d835d58f443c","Type":"ContainerStarted","Data":"c750b4e8556c9384aa2f5575d17c0e8017262ac5418585c9d772922367087ea8"}
Sep 10 19:33:50.497506 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:50.497382    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-cecfebe7-7b4d6cb889-qkctd" event={"ID":"adc68a84-f331-4104-926e-d835d58f443c","Type":"ContainerStarted","Data":"cde7a8f10aa3559ee91d07cea31087cc1137c939191049367f27147179620d4f"}
Sep 10 19:33:50.497506 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:50.497430    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-cecfebe7-7b4d6cb889-qkctd"
Sep 10 19:33:50.545815 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:50.545760    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-cecfebe7-84d6d4468b-ckwpq" podStartSLOduration=1.545743681 podStartE2EDuration="1.545743681s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:33:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:33:50.544753697 +0000 UTC m=+1086.145660938" watchObservedRunningTime="2026-09-10 19:33:50.545743681 +0000 UTC m=+1086.146650920"
Sep 10 19:33:51.126069 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:51.126028    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:52.565441 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:52.565402    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:52.811787 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:33:52.811750    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:01.505668 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:01.505601    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-cecfebe7-7b4d6cb889-qkctd"
Sep 10 19:34:01.506138 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:01.505736    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-cecfebe7-84d6d4468b-ckwpq"
Sep 10 19:34:01.531214 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:01.531160    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-cecfebe7-7b4d6cb889-qkctd" podStartSLOduration=12.531145234 podStartE2EDuration="12.531145234s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:33:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:33:50.579216448 +0000 UTC m=+1086.180123689" watchObservedRunningTime="2026-09-10 19:34:01.531145234 +0000 UTC m=+1097.132052473"
Sep 10 19:34:03.443290 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:03.443250    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:03.516140 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:03.516107    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-173596-586b787-5hxkf"]
Sep 10 19:34:03.516381 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:03.516332    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-173596-586b787-5hxkf" podUID="77fd5b21-2532-4eae-b847-64079cce6907" containerName="payload-processing" containerID="cri-o://083022a27dc279cdfcb60c5c49a291dea71d867bddd1880b190631abbfd380b7" gracePeriod=30
Sep 10 19:34:03.568132 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:03.568095    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:03.667412 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:03.667375    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:03.769846 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:03.769819    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-173596-586b787-5hxkf"
Sep 10 19:34:03.771958 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:03.771925    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:03.838291 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:03.838256    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:03.851120 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:03.851087    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wr77l\" (UniqueName: \"kubernetes.io/projected/77fd5b21-2532-4eae-b847-64079cce6907-kube-api-access-wr77l\") pod \"77fd5b21-2532-4eae-b847-64079cce6907\" (UID: \"77fd5b21-2532-4eae-b847-64079cce6907\") "
Sep 10 19:34:03.851497 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:03.851264    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/77fd5b21-2532-4eae-b847-64079cce6907-plugins-config-volume\") pod \"77fd5b21-2532-4eae-b847-64079cce6907\" (UID: \"77fd5b21-2532-4eae-b847-64079cce6907\") "
Sep 10 19:34:03.851826 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:03.851790    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77fd5b21-2532-4eae-b847-64079cce6907-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "77fd5b21-2532-4eae-b847-64079cce6907" (UID: "77fd5b21-2532-4eae-b847-64079cce6907"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:34:03.853320 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:03.853295    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77fd5b21-2532-4eae-b847-64079cce6907-kube-api-access-wr77l" (OuterVolumeSpecName: "kube-api-access-wr77l") pod "77fd5b21-2532-4eae-b847-64079cce6907" (UID: "77fd5b21-2532-4eae-b847-64079cce6907"). InnerVolumeSpecName "kube-api-access-wr77l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:34:03.952301 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:03.952267    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wr77l\" (UniqueName: \"kubernetes.io/projected/77fd5b21-2532-4eae-b847-64079cce6907-kube-api-access-wr77l\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:34:03.952301 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:03.952297    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/77fd5b21-2532-4eae-b847-64079cce6907-plugins-config-volume\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:34:04.194239 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:04.194140    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:04.576036 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:04.575994    2564 generic.go:356] "Generic (PLEG): container finished" podID="77fd5b21-2532-4eae-b847-64079cce6907" containerID="083022a27dc279cdfcb60c5c49a291dea71d867bddd1880b190631abbfd380b7" exitCode=0
Sep 10 19:34:04.576036 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:04.576040    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-173596-586b787-5hxkf"
Sep 10 19:34:04.576525 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:04.576074    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-173596-586b787-5hxkf" event={"ID":"77fd5b21-2532-4eae-b847-64079cce6907","Type":"ContainerDied","Data":"083022a27dc279cdfcb60c5c49a291dea71d867bddd1880b190631abbfd380b7"}
Sep 10 19:34:04.576525 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:04.576130    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-173596-586b787-5hxkf" event={"ID":"77fd5b21-2532-4eae-b847-64079cce6907","Type":"ContainerDied","Data":"6deb4595ec0268cbcfc4d5d13e75d0c24573dd6ec5a71c5bfd540d4a1b9b74ea"}
Sep 10 19:34:04.576525 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:04.576156    2564 scope.go:117] "RemoveContainer" containerID="083022a27dc279cdfcb60c5c49a291dea71d867bddd1880b190631abbfd380b7"
Sep 10 19:34:04.585175 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:04.585158    2564 scope.go:117] "RemoveContainer" containerID="083022a27dc279cdfcb60c5c49a291dea71d867bddd1880b190631abbfd380b7"
Sep 10 19:34:04.585430 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:34:04.585410    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"083022a27dc279cdfcb60c5c49a291dea71d867bddd1880b190631abbfd380b7\": container with ID starting with 083022a27dc279cdfcb60c5c49a291dea71d867bddd1880b190631abbfd380b7 not found: ID does not exist" containerID="083022a27dc279cdfcb60c5c49a291dea71d867bddd1880b190631abbfd380b7"
Sep 10 19:34:04.585475 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:04.585437    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"083022a27dc279cdfcb60c5c49a291dea71d867bddd1880b190631abbfd380b7"} err="failed to get container status \"083022a27dc279cdfcb60c5c49a291dea71d867bddd1880b190631abbfd380b7\": rpc error: code = NotFound desc = could not find container \"083022a27dc279cdfcb60c5c49a291dea71d867bddd1880b190631abbfd380b7\": container with ID starting with 083022a27dc279cdfcb60c5c49a291dea71d867bddd1880b190631abbfd380b7 not found: ID does not exist"
Sep 10 19:34:04.601064 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:04.601035    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-173596-586b787-5hxkf"]
Sep 10 19:34:04.604150 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:04.604128    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-173596-586b787-5hxkf"]
Sep 10 19:34:04.949658 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:04.949567    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77fd5b21-2532-4eae-b847-64079cce6907" path="/var/lib/kubelet/pods/77fd5b21-2532-4eae-b847-64079cce6907/volumes"
Sep 10 19:34:05.386967 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:05.386931    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:10.010755 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:10.010717    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:11.657176 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:11.657139    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:11.754077 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:11.754036    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:11.945651 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:11.945549    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b"]
Sep 10 19:34:11.945829 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:11.945807    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b" podUID="5f43262c-1566-404b-8e4b-fbe09a10913f" containerName="istio-proxy" containerID="cri-o://2060e830b128eefa2acd7a82d4eef525b24a1d7fb8d99c17bed1dd1799bc1061" gracePeriod=30
Sep 10 19:34:11.953199 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:11.953169    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:12.496411 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:12.496375    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:12.605443 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:12.605410    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:13.195738 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:13.195696    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:13.298999 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:13.298962    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:13.702662 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:13.702611    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:17.033805 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:17.033764    2564 patch_prober.go:28] interesting pod/e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Sep 10 19:34:17.034184 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:17.033848    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b" podUID="5f43262c-1566-404b-8e4b-fbe09a10913f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Sep 10 19:34:17.200638 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:17.200604    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b"
Sep 10 19:34:17.283972 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:17.283937    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f43262c-1566-404b-8e4b-fbe09a10913f-istiod-ca-cert\") pod \"5f43262c-1566-404b-8e4b-fbe09a10913f\" (UID: \"5f43262c-1566-404b-8e4b-fbe09a10913f\") "
Sep 10 19:34:17.284123 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:17.284025    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f43262c-1566-404b-8e4b-fbe09a10913f-istio-envoy\") pod \"5f43262c-1566-404b-8e4b-fbe09a10913f\" (UID: \"5f43262c-1566-404b-8e4b-fbe09a10913f\") "
Sep 10 19:34:17.284123 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:17.284057    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5t4nl\" (UniqueName: \"kubernetes.io/projected/5f43262c-1566-404b-8e4b-fbe09a10913f-kube-api-access-5t4nl\") pod \"5f43262c-1566-404b-8e4b-fbe09a10913f\" (UID: \"5f43262c-1566-404b-8e4b-fbe09a10913f\") "
Sep 10 19:34:17.284123 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:17.284082    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f43262c-1566-404b-8e4b-fbe09a10913f-istio-podinfo\") pod \"5f43262c-1566-404b-8e4b-fbe09a10913f\" (UID: \"5f43262c-1566-404b-8e4b-fbe09a10913f\") "
Sep 10 19:34:17.284123 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:17.284106    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f43262c-1566-404b-8e4b-fbe09a10913f-credential-socket\") pod \"5f43262c-1566-404b-8e4b-fbe09a10913f\" (UID: \"5f43262c-1566-404b-8e4b-fbe09a10913f\") "
Sep 10 19:34:17.284339 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:17.284160    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f43262c-1566-404b-8e4b-fbe09a10913f-workload-socket\") pod \"5f43262c-1566-404b-8e4b-fbe09a10913f\" (UID: \"5f43262c-1566-404b-8e4b-fbe09a10913f\") "
Sep 10 19:34:17.284339 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:17.284204    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f43262c-1566-404b-8e4b-fbe09a10913f-workload-certs\") pod \"5f43262c-1566-404b-8e4b-fbe09a10913f\" (UID: \"5f43262c-1566-404b-8e4b-fbe09a10913f\") "
Sep 10 19:34:17.284339 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:17.284246    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f43262c-1566-404b-8e4b-fbe09a10913f-istio-data\") pod \"5f43262c-1566-404b-8e4b-fbe09a10913f\" (UID: \"5f43262c-1566-404b-8e4b-fbe09a10913f\") "
Sep 10 19:34:17.284339 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:17.284288    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f43262c-1566-404b-8e4b-fbe09a10913f-istio-token\") pod \"5f43262c-1566-404b-8e4b-fbe09a10913f\" (UID: \"5f43262c-1566-404b-8e4b-fbe09a10913f\") "
Sep 10 19:34:17.284339 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:17.284309    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f43262c-1566-404b-8e4b-fbe09a10913f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5f43262c-1566-404b-8e4b-fbe09a10913f" (UID: "5f43262c-1566-404b-8e4b-fbe09a10913f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:34:17.284590 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:17.284464    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f43262c-1566-404b-8e4b-fbe09a10913f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5f43262c-1566-404b-8e4b-fbe09a10913f" (UID: "5f43262c-1566-404b-8e4b-fbe09a10913f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:34:17.284590 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:17.284484    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f43262c-1566-404b-8e4b-fbe09a10913f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5f43262c-1566-404b-8e4b-fbe09a10913f" (UID: "5f43262c-1566-404b-8e4b-fbe09a10913f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:34:17.284736 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:17.284613    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f43262c-1566-404b-8e4b-fbe09a10913f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5f43262c-1566-404b-8e4b-fbe09a10913f" (UID: "5f43262c-1566-404b-8e4b-fbe09a10913f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:34:17.284736 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:17.284675    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f43262c-1566-404b-8e4b-fbe09a10913f-istio-data" (OuterVolumeSpecName: "istio-data") pod "5f43262c-1566-404b-8e4b-fbe09a10913f" (UID: "5f43262c-1566-404b-8e4b-fbe09a10913f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:34:17.284850 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:17.284747    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f43262c-1566-404b-8e4b-fbe09a10913f-credential-socket\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:34:17.284850 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:17.284771    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f43262c-1566-404b-8e4b-fbe09a10913f-workload-socket\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:34:17.284850 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:17.284786    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f43262c-1566-404b-8e4b-fbe09a10913f-workload-certs\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:34:17.284850 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:17.284800    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f43262c-1566-404b-8e4b-fbe09a10913f-istio-data\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:34:17.284850 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:17.284813    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f43262c-1566-404b-8e4b-fbe09a10913f-istiod-ca-cert\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:34:17.286354 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:17.286331    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f43262c-1566-404b-8e4b-fbe09a10913f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5f43262c-1566-404b-8e4b-fbe09a10913f" (UID: "5f43262c-1566-404b-8e4b-fbe09a10913f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:34:17.286466 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:17.286422    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f43262c-1566-404b-8e4b-fbe09a10913f-istio-token" (OuterVolumeSpecName: "istio-token") pod "5f43262c-1566-404b-8e4b-fbe09a10913f" (UID: "5f43262c-1566-404b-8e4b-fbe09a10913f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:34:17.286718 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:17.286699    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5f43262c-1566-404b-8e4b-fbe09a10913f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5f43262c-1566-404b-8e4b-fbe09a10913f" (UID: "5f43262c-1566-404b-8e4b-fbe09a10913f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 19:34:17.286795 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:17.286700    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f43262c-1566-404b-8e4b-fbe09a10913f-kube-api-access-5t4nl" (OuterVolumeSpecName: "kube-api-access-5t4nl") pod "5f43262c-1566-404b-8e4b-fbe09a10913f" (UID: "5f43262c-1566-404b-8e4b-fbe09a10913f"). InnerVolumeSpecName "kube-api-access-5t4nl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:34:17.386074 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:17.386041    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f43262c-1566-404b-8e4b-fbe09a10913f-istio-token\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:34:17.386074 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:17.386068    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f43262c-1566-404b-8e4b-fbe09a10913f-istio-envoy\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:34:17.386074 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:17.386079    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5t4nl\" (UniqueName: \"kubernetes.io/projected/5f43262c-1566-404b-8e4b-fbe09a10913f-kube-api-access-5t4nl\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:34:17.386300 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:17.386087    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f43262c-1566-404b-8e4b-fbe09a10913f-istio-podinfo\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:34:17.647932 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:17.647834    2564 generic.go:356] "Generic (PLEG): container finished" podID="5f43262c-1566-404b-8e4b-fbe09a10913f" containerID="2060e830b128eefa2acd7a82d4eef525b24a1d7fb8d99c17bed1dd1799bc1061" exitCode=0
Sep 10 19:34:17.647932 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:17.647873    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b" event={"ID":"5f43262c-1566-404b-8e4b-fbe09a10913f","Type":"ContainerDied","Data":"2060e830b128eefa2acd7a82d4eef525b24a1d7fb8d99c17bed1dd1799bc1061"}
Sep 10 19:34:17.647932 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:17.647892    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b"
Sep 10 19:34:17.647932 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:17.647916    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b" event={"ID":"5f43262c-1566-404b-8e4b-fbe09a10913f","Type":"ContainerDied","Data":"5a9d2303e5d3ce9a7abf7a60872a7a641115b466ee3227bb300fd23f7c1798f5"}
Sep 10 19:34:17.648253 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:17.647939    2564 scope.go:117] "RemoveContainer" containerID="2060e830b128eefa2acd7a82d4eef525b24a1d7fb8d99c17bed1dd1799bc1061"
Sep 10 19:34:17.659511 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:17.659450    2564 scope.go:117] "RemoveContainer" containerID="a6054b6d9512bee4d937398714ecd847ddad16130ef81391277a7dde2b5862b2"
Sep 10 19:34:17.667756 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:17.667734    2564 scope.go:117] "RemoveContainer" containerID="2060e830b128eefa2acd7a82d4eef525b24a1d7fb8d99c17bed1dd1799bc1061"
Sep 10 19:34:17.668011 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:34:17.667994    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2060e830b128eefa2acd7a82d4eef525b24a1d7fb8d99c17bed1dd1799bc1061\": container with ID starting with 2060e830b128eefa2acd7a82d4eef525b24a1d7fb8d99c17bed1dd1799bc1061 not found: ID does not exist" containerID="2060e830b128eefa2acd7a82d4eef525b24a1d7fb8d99c17bed1dd1799bc1061"
Sep 10 19:34:17.668069 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:17.668018    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2060e830b128eefa2acd7a82d4eef525b24a1d7fb8d99c17bed1dd1799bc1061"} err="failed to get container status \"2060e830b128eefa2acd7a82d4eef525b24a1d7fb8d99c17bed1dd1799bc1061\": rpc error: code = NotFound desc = could not find container \"2060e830b128eefa2acd7a82d4eef525b24a1d7fb8d99c17bed1dd1799bc1061\": container with ID starting with 2060e830b128eefa2acd7a82d4eef525b24a1d7fb8d99c17bed1dd1799bc1061 not found: ID does not exist"
Sep 10 19:34:17.668069 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:17.668033    2564 scope.go:117] "RemoveContainer" containerID="a6054b6d9512bee4d937398714ecd847ddad16130ef81391277a7dde2b5862b2"
Sep 10 19:34:17.668283 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:34:17.668259    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6054b6d9512bee4d937398714ecd847ddad16130ef81391277a7dde2b5862b2\": container with ID starting with a6054b6d9512bee4d937398714ecd847ddad16130ef81391277a7dde2b5862b2 not found: ID does not exist" containerID="a6054b6d9512bee4d937398714ecd847ddad16130ef81391277a7dde2b5862b2"
Sep 10 19:34:17.668335 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:17.668287    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6054b6d9512bee4d937398714ecd847ddad16130ef81391277a7dde2b5862b2"} err="failed to get container status \"a6054b6d9512bee4d937398714ecd847ddad16130ef81391277a7dde2b5862b2\": rpc error: code = NotFound desc = could not find container \"a6054b6d9512bee4d937398714ecd847ddad16130ef81391277a7dde2b5862b2\": container with ID starting with a6054b6d9512bee4d937398714ecd847ddad16130ef81391277a7dde2b5862b2 not found: ID does not exist"
Sep 10 19:34:17.673042 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:17.673019    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b"]
Sep 10 19:34:17.678999 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:17.678976    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-173596-openshift-default-6f4784ffb8-r726b"]
Sep 10 19:34:18.950204 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:18.950174    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f43262c-1566-404b-8e4b-fbe09a10913f" path="/var/lib/kubelet/pods/5f43262c-1566-404b-8e4b-fbe09a10913f/volumes"
Sep 10 19:34:20.009762 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:20.009726    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:20.073274 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:20.073237    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:20.573094 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:20.573063    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:22.446100 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:22.446065    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:23.262129 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:23.262093    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:23.314385 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:23.314337    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-e7e4c3-7c7ff4c845-rtghl"]
Sep 10 19:34:23.314611 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:23.314581    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-e7e4c3-7c7ff4c845-rtghl" podUID="3b338b98-3519-46ae-9fb9-9b594ab5cd86" containerName="payload-processing" containerID="cri-o://4a1f104b04548cfc551b7915dfdd6574d258c20376515550c332070d3f422ba5" gracePeriod=30
Sep 10 19:34:23.398566 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:23.398525    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:23.503923 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:23.503862    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:23.581017 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:23.580994    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-e7e4c3-7c7ff4c845-rtghl"
Sep 10 19:34:23.613291 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:23.613133    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:23.683085 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:23.683052    2564 generic.go:356] "Generic (PLEG): container finished" podID="3b338b98-3519-46ae-9fb9-9b594ab5cd86" containerID="4a1f104b04548cfc551b7915dfdd6574d258c20376515550c332070d3f422ba5" exitCode=0
Sep 10 19:34:23.683247 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:23.683099    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-e7e4c3-7c7ff4c845-rtghl"
Sep 10 19:34:23.683247 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:23.683134    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-e7e4c3-7c7ff4c845-rtghl" event={"ID":"3b338b98-3519-46ae-9fb9-9b594ab5cd86","Type":"ContainerDied","Data":"4a1f104b04548cfc551b7915dfdd6574d258c20376515550c332070d3f422ba5"}
Sep 10 19:34:23.683247 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:23.683195    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-e7e4c3-7c7ff4c845-rtghl" event={"ID":"3b338b98-3519-46ae-9fb9-9b594ab5cd86","Type":"ContainerDied","Data":"4e4453eb33dbd13137239402dd594d0eac68a49ddea22b5518ba78df4a3946b3"}
Sep 10 19:34:23.683247 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:23.683219    2564 scope.go:117] "RemoveContainer" containerID="4a1f104b04548cfc551b7915dfdd6574d258c20376515550c332070d3f422ba5"
Sep 10 19:34:23.691974 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:23.691951    2564 scope.go:117] "RemoveContainer" containerID="4a1f104b04548cfc551b7915dfdd6574d258c20376515550c332070d3f422ba5"
Sep 10 19:34:23.692246 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:34:23.692228    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a1f104b04548cfc551b7915dfdd6574d258c20376515550c332070d3f422ba5\": container with ID starting with 4a1f104b04548cfc551b7915dfdd6574d258c20376515550c332070d3f422ba5 not found: ID does not exist" containerID="4a1f104b04548cfc551b7915dfdd6574d258c20376515550c332070d3f422ba5"
Sep 10 19:34:23.692315 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:23.692254    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a1f104b04548cfc551b7915dfdd6574d258c20376515550c332070d3f422ba5"} err="failed to get container status \"4a1f104b04548cfc551b7915dfdd6574d258c20376515550c332070d3f422ba5\": rpc error: code = NotFound desc = could not find container \"4a1f104b04548cfc551b7915dfdd6574d258c20376515550c332070d3f422ba5\": container with ID starting with 4a1f104b04548cfc551b7915dfdd6574d258c20376515550c332070d3f422ba5 not found: ID does not exist"
Sep 10 19:34:23.747285 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:23.747256    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b338b98-3519-46ae-9fb9-9b594ab5cd86-plugins-config-volume\") pod \"3b338b98-3519-46ae-9fb9-9b594ab5cd86\" (UID: \"3b338b98-3519-46ae-9fb9-9b594ab5cd86\") "
Sep 10 19:34:23.747444 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:23.747355    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-75m7v\" (UniqueName: \"kubernetes.io/projected/3b338b98-3519-46ae-9fb9-9b594ab5cd86-kube-api-access-75m7v\") pod \"3b338b98-3519-46ae-9fb9-9b594ab5cd86\" (UID: \"3b338b98-3519-46ae-9fb9-9b594ab5cd86\") "
Sep 10 19:34:23.747732 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:23.747709    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b338b98-3519-46ae-9fb9-9b594ab5cd86-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3b338b98-3519-46ae-9fb9-9b594ab5cd86" (UID: "3b338b98-3519-46ae-9fb9-9b594ab5cd86"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:34:23.749363 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:23.749346    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b338b98-3519-46ae-9fb9-9b594ab5cd86-kube-api-access-75m7v" (OuterVolumeSpecName: "kube-api-access-75m7v") pod "3b338b98-3519-46ae-9fb9-9b594ab5cd86" (UID: "3b338b98-3519-46ae-9fb9-9b594ab5cd86"). InnerVolumeSpecName "kube-api-access-75m7v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:34:23.848851 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:23.848772    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b338b98-3519-46ae-9fb9-9b594ab5cd86-plugins-config-volume\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:34:23.848851 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:23.848801    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-75m7v\" (UniqueName: \"kubernetes.io/projected/3b338b98-3519-46ae-9fb9-9b594ab5cd86-kube-api-access-75m7v\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:34:23.902263 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:23.902221    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:24.006191 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:24.006148    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-e7e4c3-7c7ff4c845-rtghl"]
Sep 10 19:34:24.013150 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:24.013126    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-e7e4c3-7c7ff4c845-rtghl"]
Sep 10 19:34:24.948832 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:24.948795    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b338b98-3519-46ae-9fb9-9b594ab5cd86" path="/var/lib/kubelet/pods/3b338b98-3519-46ae-9fb9-9b594ab5cd86/volumes"
Sep 10 19:34:30.698737 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:30.698695    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:31.311787 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:31.311752    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:31.471203 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:31.471169    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:31.549003 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:31.548973    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:31.614319 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:31.614240    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-cecfebe7-7b4d6cb889-qkctd"]
Sep 10 19:34:31.614524 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:31.614496    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-cecfebe7-7b4d6cb889-qkctd" podUID="adc68a84-f331-4104-926e-d835d58f443c" containerName="payload-processing" containerID="cri-o://c750b4e8556c9384aa2f5575d17c0e8017262ac5418585c9d772922367087ea8" gracePeriod=30
Sep 10 19:34:31.625670 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:31.625346    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-cecfebe7-84d6d4468b-ckwpq"]
Sep 10 19:34:31.625670 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:31.625571    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-cecfebe7-84d6d4468b-ckwpq" podUID="24987d09-fa94-47ea-b149-141b53c11c40" containerName="payload-pre-processing" containerID="cri-o://eafd7ebc61f2929b52f3d6277a2d0dbdf76ca0648f18740615ed1e1c665d91ce" gracePeriod=30
Sep 10 19:34:31.628923 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:31.628527    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:31.726242 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:31.726206    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:31.861612 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:31.861484    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:31.910613 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:31.910587    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-cecfebe7-7b4d6cb889-qkctd"
Sep 10 19:34:31.913583 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:31.913564    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-cecfebe7-84d6d4468b-ckwpq"
Sep 10 19:34:31.920137 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:31.920119    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/adc68a84-f331-4104-926e-d835d58f443c-plugins-config-volume\") pod \"adc68a84-f331-4104-926e-d835d58f443c\" (UID: \"adc68a84-f331-4104-926e-d835d58f443c\") "
Sep 10 19:34:31.920202 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:31.920160    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8ngkh\" (UniqueName: \"kubernetes.io/projected/adc68a84-f331-4104-926e-d835d58f443c-kube-api-access-8ngkh\") pod \"adc68a84-f331-4104-926e-d835d58f443c\" (UID: \"adc68a84-f331-4104-926e-d835d58f443c\") "
Sep 10 19:34:31.920464 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:31.920441    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/adc68a84-f331-4104-926e-d835d58f443c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "adc68a84-f331-4104-926e-d835d58f443c" (UID: "adc68a84-f331-4104-926e-d835d58f443c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:34:31.922183 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:31.922162    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/adc68a84-f331-4104-926e-d835d58f443c-kube-api-access-8ngkh" (OuterVolumeSpecName: "kube-api-access-8ngkh") pod "adc68a84-f331-4104-926e-d835d58f443c" (UID: "adc68a84-f331-4104-926e-d835d58f443c"). InnerVolumeSpecName "kube-api-access-8ngkh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:34:31.925307 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:31.925281    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:32.021359 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.021313    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24987d09-fa94-47ea-b149-141b53c11c40-plugins-config-volume\") pod \"24987d09-fa94-47ea-b149-141b53c11c40\" (UID: \"24987d09-fa94-47ea-b149-141b53c11c40\") "
Sep 10 19:34:32.021359 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.021371    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zc69s\" (UniqueName: \"kubernetes.io/projected/24987d09-fa94-47ea-b149-141b53c11c40-kube-api-access-zc69s\") pod \"24987d09-fa94-47ea-b149-141b53c11c40\" (UID: \"24987d09-fa94-47ea-b149-141b53c11c40\") "
Sep 10 19:34:32.021597 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.021564    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/adc68a84-f331-4104-926e-d835d58f443c-plugins-config-volume\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:34:32.021597 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.021574    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8ngkh\" (UniqueName: \"kubernetes.io/projected/adc68a84-f331-4104-926e-d835d58f443c-kube-api-access-8ngkh\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:34:32.021759 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.021729    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/24987d09-fa94-47ea-b149-141b53c11c40-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "24987d09-fa94-47ea-b149-141b53c11c40" (UID: "24987d09-fa94-47ea-b149-141b53c11c40"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:34:32.023552 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.023529    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24987d09-fa94-47ea-b149-141b53c11c40-kube-api-access-zc69s" (OuterVolumeSpecName: "kube-api-access-zc69s") pod "24987d09-fa94-47ea-b149-141b53c11c40" (UID: "24987d09-fa94-47ea-b149-141b53c11c40"). InnerVolumeSpecName "kube-api-access-zc69s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:34:32.122225 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.122172    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24987d09-fa94-47ea-b149-141b53c11c40-plugins-config-volume\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:34:32.122225 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.122216    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zc69s\" (UniqueName: \"kubernetes.io/projected/24987d09-fa94-47ea-b149-141b53c11c40-kube-api-access-zc69s\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:34:32.241746 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.241655    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:32.341241 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.341182    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:32.634117 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.634087    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx"]
Sep 10 19:34:32.634487 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.634469    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b338b98-3519-46ae-9fb9-9b594ab5cd86" containerName="payload-processing"
Sep 10 19:34:32.634487 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.634484    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b338b98-3519-46ae-9fb9-9b594ab5cd86" containerName="payload-processing"
Sep 10 19:34:32.634585 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.634494    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24987d09-fa94-47ea-b149-141b53c11c40" containerName="payload-pre-processing"
Sep 10 19:34:32.634585 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.634500    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="24987d09-fa94-47ea-b149-141b53c11c40" containerName="payload-pre-processing"
Sep 10 19:34:32.634585 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.634509    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77fd5b21-2532-4eae-b847-64079cce6907" containerName="payload-processing"
Sep 10 19:34:32.634585 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.634516    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="77fd5b21-2532-4eae-b847-64079cce6907" containerName="payload-processing"
Sep 10 19:34:32.634585 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.634524    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f43262c-1566-404b-8e4b-fbe09a10913f" containerName="istio-proxy"
Sep 10 19:34:32.634585 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.634530    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f43262c-1566-404b-8e4b-fbe09a10913f" containerName="istio-proxy"
Sep 10 19:34:32.634585 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.634543    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f43262c-1566-404b-8e4b-fbe09a10913f" containerName="istio-proxy"
Sep 10 19:34:32.634585 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.634549    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f43262c-1566-404b-8e4b-fbe09a10913f" containerName="istio-proxy"
Sep 10 19:34:32.634585 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.634562    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="adc68a84-f331-4104-926e-d835d58f443c" containerName="payload-processing"
Sep 10 19:34:32.634585 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.634567    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="adc68a84-f331-4104-926e-d835d58f443c" containerName="payload-processing"
Sep 10 19:34:32.634909 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.634631    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b338b98-3519-46ae-9fb9-9b594ab5cd86" containerName="payload-processing"
Sep 10 19:34:32.634909 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.634642    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f43262c-1566-404b-8e4b-fbe09a10913f" containerName="istio-proxy"
Sep 10 19:34:32.634909 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.634649    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f43262c-1566-404b-8e4b-fbe09a10913f" containerName="istio-proxy"
Sep 10 19:34:32.634909 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.634654    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="77fd5b21-2532-4eae-b847-64079cce6907" containerName="payload-processing"
Sep 10 19:34:32.634909 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.634659    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="24987d09-fa94-47ea-b149-141b53c11c40" containerName="payload-pre-processing"
Sep 10 19:34:32.634909 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.634665    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="adc68a84-f331-4104-926e-d835d58f443c" containerName="payload-processing"
Sep 10 19:34:32.637982 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.637961    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx"
Sep 10 19:34:32.644863 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.644843    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-28d250-openshift-default-dockercfg-z4q88\""
Sep 10 19:34:32.654529 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.654507    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx"]
Sep 10 19:34:32.655405 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.655364    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:32.728205 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.728170    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/837f44f6-1b71-41a2-8bcf-364fcfcb8a85-credential-socket\") pod \"e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx\" (UID: \"837f44f6-1b71-41a2-8bcf-364fcfcb8a85\") " pod="openshift-ingress/e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx"
Sep 10 19:34:32.728205 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.728202    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/837f44f6-1b71-41a2-8bcf-364fcfcb8a85-istio-token\") pod \"e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx\" (UID: \"837f44f6-1b71-41a2-8bcf-364fcfcb8a85\") " pod="openshift-ingress/e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx"
Sep 10 19:34:32.728685 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.728222    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/837f44f6-1b71-41a2-8bcf-364fcfcb8a85-istio-podinfo\") pod \"e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx\" (UID: \"837f44f6-1b71-41a2-8bcf-364fcfcb8a85\") " pod="openshift-ingress/e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx"
Sep 10 19:34:32.728685 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.728310    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/837f44f6-1b71-41a2-8bcf-364fcfcb8a85-istiod-ca-cert\") pod \"e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx\" (UID: \"837f44f6-1b71-41a2-8bcf-364fcfcb8a85\") " pod="openshift-ingress/e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx"
Sep 10 19:34:32.728685 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.728352    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/837f44f6-1b71-41a2-8bcf-364fcfcb8a85-istio-envoy\") pod \"e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx\" (UID: \"837f44f6-1b71-41a2-8bcf-364fcfcb8a85\") " pod="openshift-ingress/e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx"
Sep 10 19:34:32.728685 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.728395    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/837f44f6-1b71-41a2-8bcf-364fcfcb8a85-istio-data\") pod \"e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx\" (UID: \"837f44f6-1b71-41a2-8bcf-364fcfcb8a85\") " pod="openshift-ingress/e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx"
Sep 10 19:34:32.728685 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.728433    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vlkbg\" (UniqueName: \"kubernetes.io/projected/837f44f6-1b71-41a2-8bcf-364fcfcb8a85-kube-api-access-vlkbg\") pod \"e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx\" (UID: \"837f44f6-1b71-41a2-8bcf-364fcfcb8a85\") " pod="openshift-ingress/e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx"
Sep 10 19:34:32.728685 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.728453    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/837f44f6-1b71-41a2-8bcf-364fcfcb8a85-workload-certs\") pod \"e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx\" (UID: \"837f44f6-1b71-41a2-8bcf-364fcfcb8a85\") " pod="openshift-ingress/e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx"
Sep 10 19:34:32.728685 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.728476    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/837f44f6-1b71-41a2-8bcf-364fcfcb8a85-workload-socket\") pod \"e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx\" (UID: \"837f44f6-1b71-41a2-8bcf-364fcfcb8a85\") " pod="openshift-ingress/e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx"
Sep 10 19:34:32.735193 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.735164    2564 generic.go:356] "Generic (PLEG): container finished" podID="24987d09-fa94-47ea-b149-141b53c11c40" containerID="eafd7ebc61f2929b52f3d6277a2d0dbdf76ca0648f18740615ed1e1c665d91ce" exitCode=0
Sep 10 19:34:32.735338 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.735220    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-cecfebe7-84d6d4468b-ckwpq"
Sep 10 19:34:32.735338 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.735250    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-cecfebe7-84d6d4468b-ckwpq" event={"ID":"24987d09-fa94-47ea-b149-141b53c11c40","Type":"ContainerDied","Data":"eafd7ebc61f2929b52f3d6277a2d0dbdf76ca0648f18740615ed1e1c665d91ce"}
Sep 10 19:34:32.735338 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.735305    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-cecfebe7-84d6d4468b-ckwpq" event={"ID":"24987d09-fa94-47ea-b149-141b53c11c40","Type":"ContainerDied","Data":"4a90e4970597b0380187d97461ce4305a3faa4f8a1dd6cea228901f0bd12fbce"}
Sep 10 19:34:32.735338 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.735332    2564 scope.go:117] "RemoveContainer" containerID="eafd7ebc61f2929b52f3d6277a2d0dbdf76ca0648f18740615ed1e1c665d91ce"
Sep 10 19:34:32.737104 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.737078    2564 generic.go:356] "Generic (PLEG): container finished" podID="adc68a84-f331-4104-926e-d835d58f443c" containerID="c750b4e8556c9384aa2f5575d17c0e8017262ac5418585c9d772922367087ea8" exitCode=0
Sep 10 19:34:32.737244 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.737210    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-cecfebe7-7b4d6cb889-qkctd" event={"ID":"adc68a84-f331-4104-926e-d835d58f443c","Type":"ContainerDied","Data":"c750b4e8556c9384aa2f5575d17c0e8017262ac5418585c9d772922367087ea8"}
Sep 10 19:34:32.737311 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.737259    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-cecfebe7-7b4d6cb889-qkctd" event={"ID":"adc68a84-f331-4104-926e-d835d58f443c","Type":"ContainerDied","Data":"cde7a8f10aa3559ee91d07cea31087cc1137c939191049367f27147179620d4f"}
Sep 10 19:34:32.737370 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.737353    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-cecfebe7-7b4d6cb889-qkctd"
Sep 10 19:34:32.747251 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.747234    2564 scope.go:117] "RemoveContainer" containerID="eafd7ebc61f2929b52f3d6277a2d0dbdf76ca0648f18740615ed1e1c665d91ce"
Sep 10 19:34:32.747584 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:34:32.747561    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eafd7ebc61f2929b52f3d6277a2d0dbdf76ca0648f18740615ed1e1c665d91ce\": container with ID starting with eafd7ebc61f2929b52f3d6277a2d0dbdf76ca0648f18740615ed1e1c665d91ce not found: ID does not exist" containerID="eafd7ebc61f2929b52f3d6277a2d0dbdf76ca0648f18740615ed1e1c665d91ce"
Sep 10 19:34:32.747702 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.747595    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eafd7ebc61f2929b52f3d6277a2d0dbdf76ca0648f18740615ed1e1c665d91ce"} err="failed to get container status \"eafd7ebc61f2929b52f3d6277a2d0dbdf76ca0648f18740615ed1e1c665d91ce\": rpc error: code = NotFound desc = could not find container \"eafd7ebc61f2929b52f3d6277a2d0dbdf76ca0648f18740615ed1e1c665d91ce\": container with ID starting with eafd7ebc61f2929b52f3d6277a2d0dbdf76ca0648f18740615ed1e1c665d91ce not found: ID does not exist"
Sep 10 19:34:32.747702 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.747614    2564 scope.go:117] "RemoveContainer" containerID="c750b4e8556c9384aa2f5575d17c0e8017262ac5418585c9d772922367087ea8"
Sep 10 19:34:32.755980 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.755962    2564 scope.go:117] "RemoveContainer" containerID="c750b4e8556c9384aa2f5575d17c0e8017262ac5418585c9d772922367087ea8"
Sep 10 19:34:32.756292 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:34:32.756247    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c750b4e8556c9384aa2f5575d17c0e8017262ac5418585c9d772922367087ea8\": container with ID starting with c750b4e8556c9384aa2f5575d17c0e8017262ac5418585c9d772922367087ea8 not found: ID does not exist" containerID="c750b4e8556c9384aa2f5575d17c0e8017262ac5418585c9d772922367087ea8"
Sep 10 19:34:32.756292 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.756276    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c750b4e8556c9384aa2f5575d17c0e8017262ac5418585c9d772922367087ea8"} err="failed to get container status \"c750b4e8556c9384aa2f5575d17c0e8017262ac5418585c9d772922367087ea8\": rpc error: code = NotFound desc = could not find container \"c750b4e8556c9384aa2f5575d17c0e8017262ac5418585c9d772922367087ea8\": container with ID starting with c750b4e8556c9384aa2f5575d17c0e8017262ac5418585c9d772922367087ea8 not found: ID does not exist"
Sep 10 19:34:32.780240 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.780196    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-cecfebe7-84d6d4468b-ckwpq"]
Sep 10 19:34:32.783291 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.783268    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-cecfebe7-84d6d4468b-ckwpq"]
Sep 10 19:34:32.799736 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.799709    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:32.806572 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.806546    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-cecfebe7-7b4d6cb889-qkctd"]
Sep 10 19:34:32.809267 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.809241    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-cecfebe7-7b4d6cb889-qkctd"]
Sep 10 19:34:32.829485 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.829462    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/837f44f6-1b71-41a2-8bcf-364fcfcb8a85-istio-data\") pod \"e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx\" (UID: \"837f44f6-1b71-41a2-8bcf-364fcfcb8a85\") " pod="openshift-ingress/e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx"
Sep 10 19:34:32.829588 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.829502    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlkbg\" (UniqueName: \"kubernetes.io/projected/837f44f6-1b71-41a2-8bcf-364fcfcb8a85-kube-api-access-vlkbg\") pod \"e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx\" (UID: \"837f44f6-1b71-41a2-8bcf-364fcfcb8a85\") " pod="openshift-ingress/e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx"
Sep 10 19:34:32.829588 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.829524    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/837f44f6-1b71-41a2-8bcf-364fcfcb8a85-workload-certs\") pod \"e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx\" (UID: \"837f44f6-1b71-41a2-8bcf-364fcfcb8a85\") " pod="openshift-ingress/e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx"
Sep 10 19:34:32.829718 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.829687    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/837f44f6-1b71-41a2-8bcf-364fcfcb8a85-workload-socket\") pod \"e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx\" (UID: \"837f44f6-1b71-41a2-8bcf-364fcfcb8a85\") " pod="openshift-ingress/e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx"
Sep 10 19:34:32.829803 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.829782    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/837f44f6-1b71-41a2-8bcf-364fcfcb8a85-credential-socket\") pod \"e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx\" (UID: \"837f44f6-1b71-41a2-8bcf-364fcfcb8a85\") " pod="openshift-ingress/e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx"
Sep 10 19:34:32.829867 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.829816    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/837f44f6-1b71-41a2-8bcf-364fcfcb8a85-istio-token\") pod \"e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx\" (UID: \"837f44f6-1b71-41a2-8bcf-364fcfcb8a85\") " pod="openshift-ingress/e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx"
Sep 10 19:34:32.829867 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.829844    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/837f44f6-1b71-41a2-8bcf-364fcfcb8a85-istio-podinfo\") pod \"e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx\" (UID: \"837f44f6-1b71-41a2-8bcf-364fcfcb8a85\") " pod="openshift-ingress/e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx"
Sep 10 19:34:32.829974 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.829887    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/837f44f6-1b71-41a2-8bcf-364fcfcb8a85-istiod-ca-cert\") pod \"e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx\" (UID: \"837f44f6-1b71-41a2-8bcf-364fcfcb8a85\") " pod="openshift-ingress/e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx"
Sep 10 19:34:32.829974 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.829932    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/837f44f6-1b71-41a2-8bcf-364fcfcb8a85-istio-envoy\") pod \"e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx\" (UID: \"837f44f6-1b71-41a2-8bcf-364fcfcb8a85\") " pod="openshift-ingress/e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx"
Sep 10 19:34:32.829974 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.829953    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/837f44f6-1b71-41a2-8bcf-364fcfcb8a85-workload-certs\") pod \"e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx\" (UID: \"837f44f6-1b71-41a2-8bcf-364fcfcb8a85\") " pod="openshift-ingress/e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx"
Sep 10 19:34:32.830134 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.830072    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/837f44f6-1b71-41a2-8bcf-364fcfcb8a85-workload-socket\") pod \"e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx\" (UID: \"837f44f6-1b71-41a2-8bcf-364fcfcb8a85\") " pod="openshift-ingress/e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx"
Sep 10 19:34:32.830192 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.830162    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/837f44f6-1b71-41a2-8bcf-364fcfcb8a85-istio-data\") pod \"e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx\" (UID: \"837f44f6-1b71-41a2-8bcf-364fcfcb8a85\") " pod="openshift-ingress/e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx"
Sep 10 19:34:32.830292 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.830270    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/837f44f6-1b71-41a2-8bcf-364fcfcb8a85-credential-socket\") pod \"e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx\" (UID: \"837f44f6-1b71-41a2-8bcf-364fcfcb8a85\") " pod="openshift-ingress/e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx"
Sep 10 19:34:32.830694 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.830671    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/837f44f6-1b71-41a2-8bcf-364fcfcb8a85-istiod-ca-cert\") pod \"e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx\" (UID: \"837f44f6-1b71-41a2-8bcf-364fcfcb8a85\") " pod="openshift-ingress/e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx"
Sep 10 19:34:32.832435 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.832407    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/837f44f6-1b71-41a2-8bcf-364fcfcb8a85-istio-envoy\") pod \"e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx\" (UID: \"837f44f6-1b71-41a2-8bcf-364fcfcb8a85\") " pod="openshift-ingress/e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx"
Sep 10 19:34:32.832954 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.832931    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/837f44f6-1b71-41a2-8bcf-364fcfcb8a85-istio-podinfo\") pod \"e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx\" (UID: \"837f44f6-1b71-41a2-8bcf-364fcfcb8a85\") " pod="openshift-ingress/e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx"
Sep 10 19:34:32.842224 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.842203    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlkbg\" (UniqueName: \"kubernetes.io/projected/837f44f6-1b71-41a2-8bcf-364fcfcb8a85-kube-api-access-vlkbg\") pod \"e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx\" (UID: \"837f44f6-1b71-41a2-8bcf-364fcfcb8a85\") " pod="openshift-ingress/e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx"
Sep 10 19:34:32.844536 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.844517    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/837f44f6-1b71-41a2-8bcf-364fcfcb8a85-istio-token\") pod \"e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx\" (UID: \"837f44f6-1b71-41a2-8bcf-364fcfcb8a85\") " pod="openshift-ingress/e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx"
Sep 10 19:34:32.948684 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.948577    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx"
Sep 10 19:34:32.949774 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.949750    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24987d09-fa94-47ea-b149-141b53c11c40" path="/var/lib/kubelet/pods/24987d09-fa94-47ea-b149-141b53c11c40/volumes"
Sep 10 19:34:32.950148 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:32.950135    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="adc68a84-f331-4104-926e-d835d58f443c" path="/var/lib/kubelet/pods/adc68a84-f331-4104-926e-d835d58f443c/volumes"
Sep 10 19:34:33.238166 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:33.238078    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx"]
Sep 10 19:34:33.240390 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:34:33.240364    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod837f44f6_1b71_41a2_8bcf_364fcfcb8a85.slice/crio-a16156b755c2985ad0de7d19c2d933468868dbcb2f1a6b9e7257f29946211c17 WatchSource:0}: Error finding container a16156b755c2985ad0de7d19c2d933468868dbcb2f1a6b9e7257f29946211c17: Status 404 returned error can't find the container with id a16156b755c2985ad0de7d19c2d933468868dbcb2f1a6b9e7257f29946211c17
Sep 10 19:34:33.242416 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:33.242384    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:34:33.242502 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:33.242451    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:34:33.242502 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:33.242498    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:34:33.745485 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:33.745447    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx" event={"ID":"837f44f6-1b71-41a2-8bcf-364fcfcb8a85","Type":"ContainerStarted","Data":"270126e4f4a48f525d1e9df11844143adc92e03d9f53ef018e497928c8ffee6d"}
Sep 10 19:34:33.745979 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:33.745504    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx" event={"ID":"837f44f6-1b71-41a2-8bcf-364fcfcb8a85","Type":"ContainerStarted","Data":"a16156b755c2985ad0de7d19c2d933468868dbcb2f1a6b9e7257f29946211c17"}
Sep 10 19:34:33.779163 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:33.779104    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx" podStartSLOduration=1.779085475 podStartE2EDuration="1.779085475s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:34:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:34:33.778272496 +0000 UTC m=+1129.379179737" watchObservedRunningTime="2026-09-10 19:34:33.779085475 +0000 UTC m=+1129.379992716"
Sep 10 19:34:33.949690 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:33.949653    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx"
Sep 10 19:34:34.954901 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:34.954865    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx"
Sep 10 19:34:35.757890 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:35.757857    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx"
Sep 10 19:34:35.758855 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:35.758833    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx"
Sep 10 19:34:35.996004 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:35.995963    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:36.097093 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:36.097000    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:36.186525 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:36.186488    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:37.224335 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:37.224300    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:37.319024 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:37.318982    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:38.004870 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:38.004835    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:38.101296 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:38.101259    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:38.256355 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:38.256264    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-28d250-5d86c7b9b6-v766w"]
Sep 10 19:34:38.260192 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:38.260169    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-28d250-5d86c7b9b6-v766w"
Sep 10 19:34:38.264665 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:38.264635    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-28d250-dockercfg-qsbk6\""
Sep 10 19:34:38.265018 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:38.264997    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-28d250\""
Sep 10 19:34:38.292299 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:38.292271    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-28d250-5d86c7b9b6-v766w"]
Sep 10 19:34:38.384285 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:38.384251    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hl5ml\" (UniqueName: \"kubernetes.io/projected/12bab5be-61db-4403-b133-064439213d04-kube-api-access-hl5ml\") pod \"payload-pre-processing-e2e-shared-a-w3-28d250-5d86c7b9b6-v766w\" (UID: \"12bab5be-61db-4403-b133-064439213d04\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-28d250-5d86c7b9b6-v766w"
Sep 10 19:34:38.384459 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:38.384334    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12bab5be-61db-4403-b133-064439213d04-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-28d250-5d86c7b9b6-v766w\" (UID: \"12bab5be-61db-4403-b133-064439213d04\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-28d250-5d86c7b9b6-v766w"
Sep 10 19:34:38.485437 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:38.485403    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12bab5be-61db-4403-b133-064439213d04-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-28d250-5d86c7b9b6-v766w\" (UID: \"12bab5be-61db-4403-b133-064439213d04\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-28d250-5d86c7b9b6-v766w"
Sep 10 19:34:38.485618 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:38.485494    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hl5ml\" (UniqueName: \"kubernetes.io/projected/12bab5be-61db-4403-b133-064439213d04-kube-api-access-hl5ml\") pod \"payload-pre-processing-e2e-shared-a-w3-28d250-5d86c7b9b6-v766w\" (UID: \"12bab5be-61db-4403-b133-064439213d04\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-28d250-5d86c7b9b6-v766w"
Sep 10 19:34:38.486144 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:38.486125    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12bab5be-61db-4403-b133-064439213d04-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-28d250-5d86c7b9b6-v766w\" (UID: \"12bab5be-61db-4403-b133-064439213d04\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-28d250-5d86c7b9b6-v766w"
Sep 10 19:34:38.495965 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:38.495937    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hl5ml\" (UniqueName: \"kubernetes.io/projected/12bab5be-61db-4403-b133-064439213d04-kube-api-access-hl5ml\") pod \"payload-pre-processing-e2e-shared-a-w3-28d250-5d86c7b9b6-v766w\" (UID: \"12bab5be-61db-4403-b133-064439213d04\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-28d250-5d86c7b9b6-v766w"
Sep 10 19:34:38.572276 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:38.572168    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-28d250-5d86c7b9b6-v766w"
Sep 10 19:34:38.862736 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:38.862711    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-28d250-5d86c7b9b6-v766w"]
Sep 10 19:34:38.865729 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:34:38.865702    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12bab5be_61db_4403_b133_064439213d04.slice/crio-2808db91542f736c5991b8792fe20eb0a469055865580bbfe512a7955a327d52 WatchSource:0}: Error finding container 2808db91542f736c5991b8792fe20eb0a469055865580bbfe512a7955a327d52: Status 404 returned error can't find the container with id 2808db91542f736c5991b8792fe20eb0a469055865580bbfe512a7955a327d52
Sep 10 19:34:39.786209 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:39.786170    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-28d250-5d86c7b9b6-v766w" event={"ID":"12bab5be-61db-4403-b133-064439213d04","Type":"ContainerStarted","Data":"4c3c97efcbdb091aa9fa2ff142d47f3a21254eccb249a218e19e874fad595f61"}
Sep 10 19:34:39.786691 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:39.786240    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-28d250-5d86c7b9b6-v766w" event={"ID":"12bab5be-61db-4403-b133-064439213d04","Type":"ContainerStarted","Data":"2808db91542f736c5991b8792fe20eb0a469055865580bbfe512a7955a327d52"}
Sep 10 19:34:39.786691 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:39.786273    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-28d250-5d86c7b9b6-v766w"
Sep 10 19:34:39.811034 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:39.810977    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-28d250-5d86c7b9b6-v766w" podStartSLOduration=1.810959969 podStartE2EDuration="1.810959969s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:34:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:34:39.806329823 +0000 UTC m=+1135.407237062" watchObservedRunningTime="2026-09-10 19:34:39.810959969 +0000 UTC m=+1135.411867206"
Sep 10 19:34:40.505656 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:40.505606    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:40.566538 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:40.566504    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:40.855857 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:40.855776    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:41.258428 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:41.258392    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:41.362409 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:41.362370    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:41.545773 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:41.545677    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:48.036335 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:48.036299    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:48.142923 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:48.142890    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:48.242923 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:48.242821    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:48.453901 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:48.453855    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:50.782039 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:50.782004    2564 patch_prober.go:28] interesting pod/e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": read tcp 10.133.0.2:35370->10.133.0.65:15021: read: connection reset by peer" start-of-body=
Sep 10 19:34:50.782419 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:50.782070    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx" podUID="837f44f6-1b71-41a2-8bcf-364fcfcb8a85" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": read tcp 10.133.0.2:35370->10.133.0.65:15021: read: connection reset by peer"
Sep 10 19:34:50.793281 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:50.793254    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-28d250-5d86c7b9b6-v766w"
Sep 10 19:34:50.847826 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:50.847783    2564 generic.go:356] "Generic (PLEG): container finished" podID="837f44f6-1b71-41a2-8bcf-364fcfcb8a85" containerID="270126e4f4a48f525d1e9df11844143adc92e03d9f53ef018e497928c8ffee6d" exitCode=0
Sep 10 19:34:50.847978 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:50.847856    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx" event={"ID":"837f44f6-1b71-41a2-8bcf-364fcfcb8a85","Type":"ContainerDied","Data":"270126e4f4a48f525d1e9df11844143adc92e03d9f53ef018e497928c8ffee6d"}
Sep 10 19:34:50.848293 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:50.848277    2564 scope.go:117] "RemoveContainer" containerID="270126e4f4a48f525d1e9df11844143adc92e03d9f53ef018e497928c8ffee6d"
Sep 10 19:34:50.849388 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:50.849359    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:34:50.849472 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:50.849444    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:34:50.849509 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:50.849481    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:34:51.855210 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:51.855175    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx" event={"ID":"837f44f6-1b71-41a2-8bcf-364fcfcb8a85","Type":"ContainerStarted","Data":"52caaa63fb8a34548e59f54fced77f89c08838e0085488253ee9946103ccd8b7"}
Sep 10 19:34:51.949748 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:51.949711    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx"
Sep 10 19:34:51.949918 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:51.949861    2564 patch_prober.go:28] interesting pod/e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Sep 10 19:34:51.949918 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:51.949905    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx" podUID="837f44f6-1b71-41a2-8bcf-364fcfcb8a85" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Sep 10 19:34:52.949905 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:52.949870    2564 patch_prober.go:28] interesting pod/e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Sep 10 19:34:52.950280 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:52.949923    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx" podUID="837f44f6-1b71-41a2-8bcf-364fcfcb8a85" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Sep 10 19:34:53.949001 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:53.948965    2564 patch_prober.go:28] interesting pod/e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Sep 10 19:34:53.949185 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:53.949089    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx" podUID="837f44f6-1b71-41a2-8bcf-364fcfcb8a85" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Sep 10 19:34:54.790284 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:54.790245    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:54.865520 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:54.865475    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:54.948927 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:54.948895    2564 patch_prober.go:28] interesting pod/e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Sep 10 19:34:54.949095 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:54.948944    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx" podUID="837f44f6-1b71-41a2-8bcf-364fcfcb8a85" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Sep 10 19:34:55.770026 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:55.769985    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:55.949610 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:55.949573    2564 patch_prober.go:28] interesting pod/e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Sep 10 19:34:55.950080 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:55.949662    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx" podUID="837f44f6-1b71-41a2-8bcf-364fcfcb8a85" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Sep 10 19:34:56.040234 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:56.040154    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98"]
Sep 10 19:34:56.043841 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:56.043821    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98"
Sep 10 19:34:56.046404 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:56.046378    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-32ce4f-openshift-default-dockercfg-ds49d\""
Sep 10 19:34:56.057160 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:56.057138    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98"]
Sep 10 19:34:56.092611 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:56.092562    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:56.159018 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:56.158971    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1301503-9300-43b1-913e-589ea3d31f71-istio-podinfo\") pod \"e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98\" (UID: \"e1301503-9300-43b1-913e-589ea3d31f71\") " pod="openshift-ingress/e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98"
Sep 10 19:34:56.159221 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:56.159126    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1301503-9300-43b1-913e-589ea3d31f71-istio-token\") pod \"e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98\" (UID: \"e1301503-9300-43b1-913e-589ea3d31f71\") " pod="openshift-ingress/e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98"
Sep 10 19:34:56.159221 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:56.159168    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1301503-9300-43b1-913e-589ea3d31f71-istio-envoy\") pod \"e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98\" (UID: \"e1301503-9300-43b1-913e-589ea3d31f71\") " pod="openshift-ingress/e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98"
Sep 10 19:34:56.159221 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:56.159200    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sb4lw\" (UniqueName: \"kubernetes.io/projected/e1301503-9300-43b1-913e-589ea3d31f71-kube-api-access-sb4lw\") pod \"e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98\" (UID: \"e1301503-9300-43b1-913e-589ea3d31f71\") " pod="openshift-ingress/e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98"
Sep 10 19:34:56.159372 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:56.159340    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1301503-9300-43b1-913e-589ea3d31f71-credential-socket\") pod \"e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98\" (UID: \"e1301503-9300-43b1-913e-589ea3d31f71\") " pod="openshift-ingress/e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98"
Sep 10 19:34:56.159417 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:56.159377    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1301503-9300-43b1-913e-589ea3d31f71-istio-data\") pod \"e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98\" (UID: \"e1301503-9300-43b1-913e-589ea3d31f71\") " pod="openshift-ingress/e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98"
Sep 10 19:34:56.159456 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:56.159422    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1301503-9300-43b1-913e-589ea3d31f71-workload-socket\") pod \"e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98\" (UID: \"e1301503-9300-43b1-913e-589ea3d31f71\") " pod="openshift-ingress/e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98"
Sep 10 19:34:56.159500 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:56.159484    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1301503-9300-43b1-913e-589ea3d31f71-workload-certs\") pod \"e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98\" (UID: \"e1301503-9300-43b1-913e-589ea3d31f71\") " pod="openshift-ingress/e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98"
Sep 10 19:34:56.159571 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:56.159549    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1301503-9300-43b1-913e-589ea3d31f71-istiod-ca-cert\") pod \"e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98\" (UID: \"e1301503-9300-43b1-913e-589ea3d31f71\") " pod="openshift-ingress/e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98"
Sep 10 19:34:56.261185 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:56.261138    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1301503-9300-43b1-913e-589ea3d31f71-istiod-ca-cert\") pod \"e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98\" (UID: \"e1301503-9300-43b1-913e-589ea3d31f71\") " pod="openshift-ingress/e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98"
Sep 10 19:34:56.261400 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:56.261198    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1301503-9300-43b1-913e-589ea3d31f71-istio-podinfo\") pod \"e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98\" (UID: \"e1301503-9300-43b1-913e-589ea3d31f71\") " pod="openshift-ingress/e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98"
Sep 10 19:34:56.261400 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:56.261223    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1301503-9300-43b1-913e-589ea3d31f71-istio-token\") pod \"e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98\" (UID: \"e1301503-9300-43b1-913e-589ea3d31f71\") " pod="openshift-ingress/e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98"
Sep 10 19:34:56.261400 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:56.261246    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1301503-9300-43b1-913e-589ea3d31f71-istio-envoy\") pod \"e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98\" (UID: \"e1301503-9300-43b1-913e-589ea3d31f71\") " pod="openshift-ingress/e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98"
Sep 10 19:34:56.261400 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:56.261269    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sb4lw\" (UniqueName: \"kubernetes.io/projected/e1301503-9300-43b1-913e-589ea3d31f71-kube-api-access-sb4lw\") pod \"e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98\" (UID: \"e1301503-9300-43b1-913e-589ea3d31f71\") " pod="openshift-ingress/e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98"
Sep 10 19:34:56.261400 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:56.261352    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1301503-9300-43b1-913e-589ea3d31f71-credential-socket\") pod \"e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98\" (UID: \"e1301503-9300-43b1-913e-589ea3d31f71\") " pod="openshift-ingress/e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98"
Sep 10 19:34:56.261400 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:56.261376    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1301503-9300-43b1-913e-589ea3d31f71-istio-data\") pod \"e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98\" (UID: \"e1301503-9300-43b1-913e-589ea3d31f71\") " pod="openshift-ingress/e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98"
Sep 10 19:34:56.261765 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:56.261418    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1301503-9300-43b1-913e-589ea3d31f71-workload-socket\") pod \"e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98\" (UID: \"e1301503-9300-43b1-913e-589ea3d31f71\") " pod="openshift-ingress/e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98"
Sep 10 19:34:56.261765 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:56.261481    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1301503-9300-43b1-913e-589ea3d31f71-workload-certs\") pod \"e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98\" (UID: \"e1301503-9300-43b1-913e-589ea3d31f71\") " pod="openshift-ingress/e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98"
Sep 10 19:34:56.261906 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:56.261878    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1301503-9300-43b1-913e-589ea3d31f71-workload-certs\") pod \"e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98\" (UID: \"e1301503-9300-43b1-913e-589ea3d31f71\") " pod="openshift-ingress/e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98"
Sep 10 19:34:56.262050 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:56.262017    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1301503-9300-43b1-913e-589ea3d31f71-istiod-ca-cert\") pod \"e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98\" (UID: \"e1301503-9300-43b1-913e-589ea3d31f71\") " pod="openshift-ingress/e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98"
Sep 10 19:34:56.262736 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:56.262708    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1301503-9300-43b1-913e-589ea3d31f71-istio-data\") pod \"e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98\" (UID: \"e1301503-9300-43b1-913e-589ea3d31f71\") " pod="openshift-ingress/e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98"
Sep 10 19:34:56.262862 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:56.262840    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1301503-9300-43b1-913e-589ea3d31f71-workload-socket\") pod \"e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98\" (UID: \"e1301503-9300-43b1-913e-589ea3d31f71\") " pod="openshift-ingress/e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98"
Sep 10 19:34:56.268642 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:56.264796    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1301503-9300-43b1-913e-589ea3d31f71-credential-socket\") pod \"e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98\" (UID: \"e1301503-9300-43b1-913e-589ea3d31f71\") " pod="openshift-ingress/e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98"
Sep 10 19:34:56.268642 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:56.265220    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1301503-9300-43b1-913e-589ea3d31f71-istio-envoy\") pod \"e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98\" (UID: \"e1301503-9300-43b1-913e-589ea3d31f71\") " pod="openshift-ingress/e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98"
Sep 10 19:34:56.268642 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:56.266072    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1301503-9300-43b1-913e-589ea3d31f71-istio-podinfo\") pod \"e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98\" (UID: \"e1301503-9300-43b1-913e-589ea3d31f71\") " pod="openshift-ingress/e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98"
Sep 10 19:34:56.272642 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:56.272344    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sb4lw\" (UniqueName: \"kubernetes.io/projected/e1301503-9300-43b1-913e-589ea3d31f71-kube-api-access-sb4lw\") pod \"e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98\" (UID: \"e1301503-9300-43b1-913e-589ea3d31f71\") " pod="openshift-ingress/e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98"
Sep 10 19:34:56.287649 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:56.273130    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1301503-9300-43b1-913e-589ea3d31f71-istio-token\") pod \"e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98\" (UID: \"e1301503-9300-43b1-913e-589ea3d31f71\") " pod="openshift-ingress/e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98"
Sep 10 19:34:56.354661 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:56.354546    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98"
Sep 10 19:34:56.357816 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:56.357789    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:56.658068 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:56.658044    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98"]
Sep 10 19:34:56.663791 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:56.663739    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:34:56.663902 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:56.663857    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:34:56.663965 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:56.663913    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:34:56.890924 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:56.890885    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98" event={"ID":"e1301503-9300-43b1-913e-589ea3d31f71","Type":"ContainerStarted","Data":"62cb85ae6a53dc79bef7ef86877292824e937d96b1869593a5092bb38c578bcf"}
Sep 10 19:34:56.891120 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:56.890950    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98" event={"ID":"e1301503-9300-43b1-913e-589ea3d31f71","Type":"ContainerStarted","Data":"097b02b7d45ca86644ec05bf6bfad658e8a640223f9eb0b2e49ad3b846696eb2"}
Sep 10 19:34:56.953373 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:56.953286    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx"
Sep 10 19:34:56.953768 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:56.953663    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx"
Sep 10 19:34:56.954588 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:56.954570    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx"
Sep 10 19:34:56.976330 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:56.976271    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98" podStartSLOduration=0.976250223 podStartE2EDuration="976.250223ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:34:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:34:56.918977931 +0000 UTC m=+1152.519885171" watchObservedRunningTime="2026-09-10 19:34:56.976250223 +0000 UTC m=+1152.577157463"
Sep 10 19:34:57.355224 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:57.355180    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98"
Sep 10 19:34:57.361310 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:57.361279    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98"
Sep 10 19:34:57.903689 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:57.903653    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98"
Sep 10 19:34:57.904525 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:57.904503    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98"
Sep 10 19:34:58.933385 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:58.933350    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:59.034406 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:34:59.034368    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:00.471746 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:00.471707    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:00.570016 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:00.569980    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:00.887248 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:00.887213    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:01.039485 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:01.039451    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:01.136065 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:01.136031    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:01.189875 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:01.189802    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-32ce4f-8896f4555-89kkl"]
Sep 10 19:35:01.193405 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:01.193384    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-32ce4f-8896f4555-89kkl"
Sep 10 19:35:01.196097 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:01.196066    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-32ce4f\""
Sep 10 19:35:01.196212 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:01.196066    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-32ce4f-dockercfg-s5thx\""
Sep 10 19:35:01.213659 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:01.213631    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-32ce4f-8896f4555-89kkl"]
Sep 10 19:35:01.239323 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:01.239287    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:01.252238 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:01.252203    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-32ce4f-649996998-6pzhs"]
Sep 10 19:35:01.255676 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:01.255657    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-32ce4f-649996998-6pzhs"
Sep 10 19:35:01.267933 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:01.267909    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-32ce4f-649996998-6pzhs"]
Sep 10 19:35:01.324515 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:01.324483    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zn65m\" (UniqueName: \"kubernetes.io/projected/731b5f5c-45a1-466d-b0b8-143680a1733d-kube-api-access-zn65m\") pod \"payload-processing-e2e-shared-a-w2-32ce4f-8896f4555-89kkl\" (UID: \"731b5f5c-45a1-466d-b0b8-143680a1733d\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-32ce4f-8896f4555-89kkl"
Sep 10 19:35:01.324707 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:01.324610    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/731b5f5c-45a1-466d-b0b8-143680a1733d-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-32ce4f-8896f4555-89kkl\" (UID: \"731b5f5c-45a1-466d-b0b8-143680a1733d\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-32ce4f-8896f4555-89kkl"
Sep 10 19:35:01.337911 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:01.337880    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:01.425315 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:01.425278    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/649c71f4-a31f-4934-a004-a0f41ddd57cc-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-32ce4f-649996998-6pzhs\" (UID: \"649c71f4-a31f-4934-a004-a0f41ddd57cc\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-32ce4f-649996998-6pzhs"
Sep 10 19:35:01.425515 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:01.425337    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zn65m\" (UniqueName: \"kubernetes.io/projected/731b5f5c-45a1-466d-b0b8-143680a1733d-kube-api-access-zn65m\") pod \"payload-processing-e2e-shared-a-w2-32ce4f-8896f4555-89kkl\" (UID: \"731b5f5c-45a1-466d-b0b8-143680a1733d\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-32ce4f-8896f4555-89kkl"
Sep 10 19:35:01.425515 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:01.425419    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4rgb\" (UniqueName: \"kubernetes.io/projected/649c71f4-a31f-4934-a004-a0f41ddd57cc-kube-api-access-m4rgb\") pod \"payload-pre-processing-e2e-shared-a-w2-32ce4f-649996998-6pzhs\" (UID: \"649c71f4-a31f-4934-a004-a0f41ddd57cc\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-32ce4f-649996998-6pzhs"
Sep 10 19:35:01.425515 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:01.425510    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/731b5f5c-45a1-466d-b0b8-143680a1733d-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-32ce4f-8896f4555-89kkl\" (UID: \"731b5f5c-45a1-466d-b0b8-143680a1733d\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-32ce4f-8896f4555-89kkl"
Sep 10 19:35:01.426197 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:01.426180    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/731b5f5c-45a1-466d-b0b8-143680a1733d-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-32ce4f-8896f4555-89kkl\" (UID: \"731b5f5c-45a1-466d-b0b8-143680a1733d\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-32ce4f-8896f4555-89kkl"
Sep 10 19:35:01.434061 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:01.434034    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zn65m\" (UniqueName: \"kubernetes.io/projected/731b5f5c-45a1-466d-b0b8-143680a1733d-kube-api-access-zn65m\") pod \"payload-processing-e2e-shared-a-w2-32ce4f-8896f4555-89kkl\" (UID: \"731b5f5c-45a1-466d-b0b8-143680a1733d\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-32ce4f-8896f4555-89kkl"
Sep 10 19:35:01.504693 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:01.504654    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-32ce4f-8896f4555-89kkl"
Sep 10 19:35:01.526465 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:01.526426    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/649c71f4-a31f-4934-a004-a0f41ddd57cc-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-32ce4f-649996998-6pzhs\" (UID: \"649c71f4-a31f-4934-a004-a0f41ddd57cc\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-32ce4f-649996998-6pzhs"
Sep 10 19:35:01.526653 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:01.526552    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4rgb\" (UniqueName: \"kubernetes.io/projected/649c71f4-a31f-4934-a004-a0f41ddd57cc-kube-api-access-m4rgb\") pod \"payload-pre-processing-e2e-shared-a-w2-32ce4f-649996998-6pzhs\" (UID: \"649c71f4-a31f-4934-a004-a0f41ddd57cc\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-32ce4f-649996998-6pzhs"
Sep 10 19:35:01.527340 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:01.527277    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/649c71f4-a31f-4934-a004-a0f41ddd57cc-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-32ce4f-649996998-6pzhs\" (UID: \"649c71f4-a31f-4934-a004-a0f41ddd57cc\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-32ce4f-649996998-6pzhs"
Sep 10 19:35:01.543118 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:01.543089    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4rgb\" (UniqueName: \"kubernetes.io/projected/649c71f4-a31f-4934-a004-a0f41ddd57cc-kube-api-access-m4rgb\") pod \"payload-pre-processing-e2e-shared-a-w2-32ce4f-649996998-6pzhs\" (UID: \"649c71f4-a31f-4934-a004-a0f41ddd57cc\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-32ce4f-649996998-6pzhs"
Sep 10 19:35:01.566389 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:01.566363    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-32ce4f-649996998-6pzhs"
Sep 10 19:35:01.567809 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:01.567781    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:01.786185 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:01.786159    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-32ce4f-8896f4555-89kkl"]
Sep 10 19:35:01.787972 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:35:01.787944    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod731b5f5c_45a1_466d_b0b8_143680a1733d.slice/crio-f6c78e131cae99dcca607b0fdefa22aac873eeef88ad2c85d263ab117582ee1d WatchSource:0}: Error finding container f6c78e131cae99dcca607b0fdefa22aac873eeef88ad2c85d263ab117582ee1d: Status 404 returned error can't find the container with id f6c78e131cae99dcca607b0fdefa22aac873eeef88ad2c85d263ab117582ee1d
Sep 10 19:35:01.874721 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:01.874692    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-32ce4f-649996998-6pzhs"]
Sep 10 19:35:01.876742 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:35:01.876699    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod649c71f4_a31f_4934_a004_a0f41ddd57cc.slice/crio-9ccff95d41a1105865c08c37b2d6771feddfe2493a4e82c73237ed909da90d65 WatchSource:0}: Error finding container 9ccff95d41a1105865c08c37b2d6771feddfe2493a4e82c73237ed909da90d65: Status 404 returned error can't find the container with id 9ccff95d41a1105865c08c37b2d6771feddfe2493a4e82c73237ed909da90d65
Sep 10 19:35:01.913736 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:01.913656    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:01.928358 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:01.928327    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-32ce4f-649996998-6pzhs" event={"ID":"649c71f4-a31f-4934-a004-a0f41ddd57cc","Type":"ContainerStarted","Data":"9ccff95d41a1105865c08c37b2d6771feddfe2493a4e82c73237ed909da90d65"}
Sep 10 19:35:01.930259 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:01.930223    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-32ce4f-8896f4555-89kkl" event={"ID":"731b5f5c-45a1-466d-b0b8-143680a1733d","Type":"ContainerStarted","Data":"4266291e64b680947830fdbb09fe222e1f27e244918efdf108ab0c00200bcab9"}
Sep 10 19:35:01.930412 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:01.930276    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-32ce4f-8896f4555-89kkl" event={"ID":"731b5f5c-45a1-466d-b0b8-143680a1733d","Type":"ContainerStarted","Data":"f6c78e131cae99dcca607b0fdefa22aac873eeef88ad2c85d263ab117582ee1d"}
Sep 10 19:35:01.930412 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:01.930332    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-32ce4f-8896f4555-89kkl"
Sep 10 19:35:01.950190 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:01.950139    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-32ce4f-8896f4555-89kkl" podStartSLOduration=0.950122332 podStartE2EDuration="950.122332ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:35:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:35:01.949024694 +0000 UTC m=+1157.549931935" watchObservedRunningTime="2026-09-10 19:35:01.950122332 +0000 UTC m=+1157.551029606"
Sep 10 19:35:02.939865 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:02.939816    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-32ce4f-649996998-6pzhs" event={"ID":"649c71f4-a31f-4934-a004-a0f41ddd57cc","Type":"ContainerStarted","Data":"6c0a40ba9e95c80c0c433aa21ebcec5bc706e70b056d262a9ed48cd0a8ffb128"}
Sep 10 19:35:02.940250 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:02.939886    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-32ce4f-649996998-6pzhs"
Sep 10 19:35:02.959079 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:02.959036    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-32ce4f-649996998-6pzhs" podStartSLOduration=1.9590223679999998 podStartE2EDuration="1.959022368s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:35:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:35:02.958228465 +0000 UTC m=+1158.559135709" watchObservedRunningTime="2026-09-10 19:35:02.959022368 +0000 UTC m=+1158.559929608"
Sep 10 19:35:04.707983 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:04.707936    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:04.786389 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:04.786346    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:05.906808 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:05.906761    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:05.967967 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:05.967930    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:06.256858 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:06.256824    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:06.492143 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:06.492106    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:06.587520 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:06.587393    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:06.607382 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:06.607349    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b83810-96d75b4fd-6rmvx"]
Sep 10 19:35:06.613999 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:06.613961    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b83810-96d75b4fd-6rmvx"
Sep 10 19:35:06.617111 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:06.617082    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-b83810\""
Sep 10 19:35:06.617258 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:06.617221    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-b83810-dockercfg-w479f\""
Sep 10 19:35:06.620834 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:06.620805    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b83810-96d75b4fd-6rmvx"]
Sep 10 19:35:06.677070 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:06.677038    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d78eafd1-ceaa-4e14-9d59-e71b95f08ae7-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-b83810-96d75b4fd-6rmvx\" (UID: \"d78eafd1-ceaa-4e14-9d59-e71b95f08ae7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b83810-96d75b4fd-6rmvx"
Sep 10 19:35:06.677271 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:06.677086    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdqbm\" (UniqueName: \"kubernetes.io/projected/d78eafd1-ceaa-4e14-9d59-e71b95f08ae7-kube-api-access-vdqbm\") pod \"payload-pre-processing-e2e-shared-b-w3-b83810-96d75b4fd-6rmvx\" (UID: \"d78eafd1-ceaa-4e14-9d59-e71b95f08ae7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b83810-96d75b4fd-6rmvx"
Sep 10 19:35:06.777725 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:06.777681    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d78eafd1-ceaa-4e14-9d59-e71b95f08ae7-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-b83810-96d75b4fd-6rmvx\" (UID: \"d78eafd1-ceaa-4e14-9d59-e71b95f08ae7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b83810-96d75b4fd-6rmvx"
Sep 10 19:35:06.777914 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:06.777753    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdqbm\" (UniqueName: \"kubernetes.io/projected/d78eafd1-ceaa-4e14-9d59-e71b95f08ae7-kube-api-access-vdqbm\") pod \"payload-pre-processing-e2e-shared-b-w3-b83810-96d75b4fd-6rmvx\" (UID: \"d78eafd1-ceaa-4e14-9d59-e71b95f08ae7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b83810-96d75b4fd-6rmvx"
Sep 10 19:35:06.778599 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:06.778576    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d78eafd1-ceaa-4e14-9d59-e71b95f08ae7-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-b83810-96d75b4fd-6rmvx\" (UID: \"d78eafd1-ceaa-4e14-9d59-e71b95f08ae7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b83810-96d75b4fd-6rmvx"
Sep 10 19:35:06.789181 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:06.789154    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdqbm\" (UniqueName: \"kubernetes.io/projected/d78eafd1-ceaa-4e14-9d59-e71b95f08ae7-kube-api-access-vdqbm\") pod \"payload-pre-processing-e2e-shared-b-w3-b83810-96d75b4fd-6rmvx\" (UID: \"d78eafd1-ceaa-4e14-9d59-e71b95f08ae7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b83810-96d75b4fd-6rmvx"
Sep 10 19:35:06.927845 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:06.927755    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b83810-96d75b4fd-6rmvx"
Sep 10 19:35:07.216981 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:07.216948    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b83810-96d75b4fd-6rmvx"]
Sep 10 19:35:07.218733 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:35:07.218703    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd78eafd1_ceaa_4e14_9d59_e71b95f08ae7.slice/crio-11f06a06b18b46df9e16be47aa89c85c1007911c1a09f58c07a0ca9afd37d13b WatchSource:0}: Error finding container 11f06a06b18b46df9e16be47aa89c85c1007911c1a09f58c07a0ca9afd37d13b: Status 404 returned error can't find the container with id 11f06a06b18b46df9e16be47aa89c85c1007911c1a09f58c07a0ca9afd37d13b
Sep 10 19:35:07.973346 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:07.973296    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b83810-96d75b4fd-6rmvx" event={"ID":"d78eafd1-ceaa-4e14-9d59-e71b95f08ae7","Type":"ContainerStarted","Data":"f5e28604804ffbe7e03edac3813eaff2e902e5f5b273ea709b08dc24b88afacb"}
Sep 10 19:35:07.973882 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:07.973373    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b83810-96d75b4fd-6rmvx" event={"ID":"d78eafd1-ceaa-4e14-9d59-e71b95f08ae7","Type":"ContainerStarted","Data":"11f06a06b18b46df9e16be47aa89c85c1007911c1a09f58c07a0ca9afd37d13b"}
Sep 10 19:35:07.974351 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:07.974326    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b83810-96d75b4fd-6rmvx"
Sep 10 19:35:07.996597 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:07.996544    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b83810-96d75b4fd-6rmvx" podStartSLOduration=1.99652576 podStartE2EDuration="1.99652576s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:35:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:35:07.994824439 +0000 UTC m=+1163.595731687" watchObservedRunningTime="2026-09-10 19:35:07.99652576 +0000 UTC m=+1163.597433000"
Sep 10 19:35:12.495968 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:12.495932    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:12.941691 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:12.941661    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-32ce4f-8896f4555-89kkl"
Sep 10 19:35:13.946901 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:13.946872    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-32ce4f-649996998-6pzhs"
Sep 10 19:35:16.025664 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:16.025615    2564 generic.go:356] "Generic (PLEG): container finished" podID="e1301503-9300-43b1-913e-589ea3d31f71" containerID="62cb85ae6a53dc79bef7ef86877292824e937d96b1869593a5092bb38c578bcf" exitCode=0
Sep 10 19:35:16.025664 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:16.025649    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98" event={"ID":"e1301503-9300-43b1-913e-589ea3d31f71","Type":"ContainerDied","Data":"62cb85ae6a53dc79bef7ef86877292824e937d96b1869593a5092bb38c578bcf"}
Sep 10 19:35:16.026102 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:16.026037    2564 scope.go:117] "RemoveContainer" containerID="62cb85ae6a53dc79bef7ef86877292824e937d96b1869593a5092bb38c578bcf"
Sep 10 19:35:16.027145 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:16.027119    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:35:16.027212 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:16.027184    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:35:16.027254 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:16.027220    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:35:17.032904 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:17.032871    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98" event={"ID":"e1301503-9300-43b1-913e-589ea3d31f71","Type":"ContainerStarted","Data":"d7e7fbdb5f09486c263183d281e571fc0dfee4002f6acb332605460a061cf76c"}
Sep 10 19:35:17.354879 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:17.354779    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98"
Sep 10 19:35:17.355060 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:17.355007    2564 patch_prober.go:28] interesting pod/e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Sep 10 19:35:17.355136 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:17.355067    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98" podUID="e1301503-9300-43b1-913e-589ea3d31f71" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Sep 10 19:35:18.355146 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:18.355110    2564 patch_prober.go:28] interesting pod/e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Sep 10 19:35:18.355698 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:18.355162    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98" podUID="e1301503-9300-43b1-913e-589ea3d31f71" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Sep 10 19:35:19.355865 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:19.355824    2564 patch_prober.go:28] interesting pod/e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Sep 10 19:35:19.356251 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:19.355886    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98" podUID="e1301503-9300-43b1-913e-589ea3d31f71" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Sep 10 19:35:19.990699 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:19.990664    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b83810-96d75b4fd-6rmvx"
Sep 10 19:35:20.355464 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:20.355426    2564 patch_prober.go:28] interesting pod/e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Sep 10 19:35:20.355668 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:20.355523    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98" podUID="e1301503-9300-43b1-913e-589ea3d31f71" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Sep 10 19:35:21.355482 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:21.355447    2564 patch_prober.go:28] interesting pod/e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Sep 10 19:35:21.355899 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:21.355508    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98" podUID="e1301503-9300-43b1-913e-589ea3d31f71" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Sep 10 19:35:21.526416 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:21.526376    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:22.444098 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:22.444059    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98"
Sep 10 19:35:22.444587 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:22.444452    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98"
Sep 10 19:35:22.446288 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:22.446267    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98"
Sep 10 19:35:23.926208 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:23.926158    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:24.259644 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:24.259591    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:24.558611 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:24.558523    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46"]
Sep 10 19:35:24.562948 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:24.562920    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46"
Sep 10 19:35:24.566262 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:24.566238    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-7bfb5d-openshift-default-dockercfg-lbt7n\""
Sep 10 19:35:24.574943 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:24.574918    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46"]
Sep 10 19:35:24.586325 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:24.586288    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:24.671715 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:24.671679    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4e00663-6fca-498b-926e-716be61dff37-credential-socket\") pod \"e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46\" (UID: \"b4e00663-6fca-498b-926e-716be61dff37\") " pod="openshift-ingress/e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46"
Sep 10 19:35:24.671996 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:24.671947    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4e00663-6fca-498b-926e-716be61dff37-workload-socket\") pod \"e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46\" (UID: \"b4e00663-6fca-498b-926e-716be61dff37\") " pod="openshift-ingress/e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46"
Sep 10 19:35:24.672144 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:24.672022    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4e00663-6fca-498b-926e-716be61dff37-istio-envoy\") pod \"e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46\" (UID: \"b4e00663-6fca-498b-926e-716be61dff37\") " pod="openshift-ingress/e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46"
Sep 10 19:35:24.672144 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:24.672061    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdwc2\" (UniqueName: \"kubernetes.io/projected/b4e00663-6fca-498b-926e-716be61dff37-kube-api-access-vdwc2\") pod \"e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46\" (UID: \"b4e00663-6fca-498b-926e-716be61dff37\") " pod="openshift-ingress/e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46"
Sep 10 19:35:24.672144 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:24.672083    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4e00663-6fca-498b-926e-716be61dff37-istio-data\") pod \"e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46\" (UID: \"b4e00663-6fca-498b-926e-716be61dff37\") " pod="openshift-ingress/e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46"
Sep 10 19:35:24.672328 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:24.672151    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4e00663-6fca-498b-926e-716be61dff37-istio-podinfo\") pod \"e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46\" (UID: \"b4e00663-6fca-498b-926e-716be61dff37\") " pod="openshift-ingress/e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46"
Sep 10 19:35:24.672328 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:24.672197    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4e00663-6fca-498b-926e-716be61dff37-istio-token\") pod \"e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46\" (UID: \"b4e00663-6fca-498b-926e-716be61dff37\") " pod="openshift-ingress/e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46"
Sep 10 19:35:24.672328 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:24.672286    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4e00663-6fca-498b-926e-716be61dff37-istiod-ca-cert\") pod \"e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46\" (UID: \"b4e00663-6fca-498b-926e-716be61dff37\") " pod="openshift-ingress/e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46"
Sep 10 19:35:24.672328 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:24.672322    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4e00663-6fca-498b-926e-716be61dff37-workload-certs\") pod \"e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46\" (UID: \"b4e00663-6fca-498b-926e-716be61dff37\") " pod="openshift-ingress/e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46"
Sep 10 19:35:24.722595 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:24.722557    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:24.773911 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:24.773869    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4e00663-6fca-498b-926e-716be61dff37-istiod-ca-cert\") pod \"e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46\" (UID: \"b4e00663-6fca-498b-926e-716be61dff37\") " pod="openshift-ingress/e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46"
Sep 10 19:35:24.774101 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:24.773919    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4e00663-6fca-498b-926e-716be61dff37-workload-certs\") pod \"e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46\" (UID: \"b4e00663-6fca-498b-926e-716be61dff37\") " pod="openshift-ingress/e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46"
Sep 10 19:35:24.774101 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:24.773972    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4e00663-6fca-498b-926e-716be61dff37-credential-socket\") pod \"e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46\" (UID: \"b4e00663-6fca-498b-926e-716be61dff37\") " pod="openshift-ingress/e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46"
Sep 10 19:35:24.774101 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:24.773997    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4e00663-6fca-498b-926e-716be61dff37-workload-socket\") pod \"e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46\" (UID: \"b4e00663-6fca-498b-926e-716be61dff37\") " pod="openshift-ingress/e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46"
Sep 10 19:35:24.774259 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:24.774134    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4e00663-6fca-498b-926e-716be61dff37-istio-envoy\") pod \"e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46\" (UID: \"b4e00663-6fca-498b-926e-716be61dff37\") " pod="openshift-ingress/e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46"
Sep 10 19:35:24.774259 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:24.774188    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdwc2\" (UniqueName: \"kubernetes.io/projected/b4e00663-6fca-498b-926e-716be61dff37-kube-api-access-vdwc2\") pod \"e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46\" (UID: \"b4e00663-6fca-498b-926e-716be61dff37\") " pod="openshift-ingress/e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46"
Sep 10 19:35:24.774259 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:24.774219    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4e00663-6fca-498b-926e-716be61dff37-istio-data\") pod \"e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46\" (UID: \"b4e00663-6fca-498b-926e-716be61dff37\") " pod="openshift-ingress/e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46"
Sep 10 19:35:24.774414 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:24.774266    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4e00663-6fca-498b-926e-716be61dff37-istio-podinfo\") pod \"e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46\" (UID: \"b4e00663-6fca-498b-926e-716be61dff37\") " pod="openshift-ingress/e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46"
Sep 10 19:35:24.774414 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:24.774304    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4e00663-6fca-498b-926e-716be61dff37-istio-token\") pod \"e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46\" (UID: \"b4e00663-6fca-498b-926e-716be61dff37\") " pod="openshift-ingress/e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46"
Sep 10 19:35:24.774667 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:24.774642    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4e00663-6fca-498b-926e-716be61dff37-workload-certs\") pod \"e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46\" (UID: \"b4e00663-6fca-498b-926e-716be61dff37\") " pod="openshift-ingress/e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46"
Sep 10 19:35:24.774849 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:24.774826    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4e00663-6fca-498b-926e-716be61dff37-credential-socket\") pod \"e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46\" (UID: \"b4e00663-6fca-498b-926e-716be61dff37\") " pod="openshift-ingress/e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46"
Sep 10 19:35:24.775045 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:24.774748    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4e00663-6fca-498b-926e-716be61dff37-workload-socket\") pod \"e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46\" (UID: \"b4e00663-6fca-498b-926e-716be61dff37\") " pod="openshift-ingress/e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46"
Sep 10 19:35:24.775045 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:24.774741    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4e00663-6fca-498b-926e-716be61dff37-istiod-ca-cert\") pod \"e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46\" (UID: \"b4e00663-6fca-498b-926e-716be61dff37\") " pod="openshift-ingress/e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46"
Sep 10 19:35:24.775045 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:24.774909    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4e00663-6fca-498b-926e-716be61dff37-istio-data\") pod \"e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46\" (UID: \"b4e00663-6fca-498b-926e-716be61dff37\") " pod="openshift-ingress/e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46"
Sep 10 19:35:24.776998 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:24.776960    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4e00663-6fca-498b-926e-716be61dff37-istio-envoy\") pod \"e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46\" (UID: \"b4e00663-6fca-498b-926e-716be61dff37\") " pod="openshift-ingress/e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46"
Sep 10 19:35:24.777687 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:24.777664    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4e00663-6fca-498b-926e-716be61dff37-istio-podinfo\") pod \"e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46\" (UID: \"b4e00663-6fca-498b-926e-716be61dff37\") " pod="openshift-ingress/e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46"
Sep 10 19:35:24.783684 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:24.783614    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4e00663-6fca-498b-926e-716be61dff37-istio-token\") pod \"e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46\" (UID: \"b4e00663-6fca-498b-926e-716be61dff37\") " pod="openshift-ingress/e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46"
Sep 10 19:35:24.783820 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:24.783756    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdwc2\" (UniqueName: \"kubernetes.io/projected/b4e00663-6fca-498b-926e-716be61dff37-kube-api-access-vdwc2\") pod \"e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46\" (UID: \"b4e00663-6fca-498b-926e-716be61dff37\") " pod="openshift-ingress/e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46"
Sep 10 19:35:24.874899 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:24.874805    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46"
Sep 10 19:35:25.027455 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:25.027425    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:25.163884 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:25.163789    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46"]
Sep 10 19:35:25.166146 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:35:25.166112    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4e00663_6fca_498b_926e_716be61dff37.slice/crio-b52bb21f81d7a4353e55bfed0e62d22b20438816d6c56726a5686fc9403fcad6 WatchSource:0}: Error finding container b52bb21f81d7a4353e55bfed0e62d22b20438816d6c56726a5686fc9403fcad6: Status 404 returned error can't find the container with id b52bb21f81d7a4353e55bfed0e62d22b20438816d6c56726a5686fc9403fcad6
Sep 10 19:35:25.168695 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:25.168651    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:35:25.168799 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:25.168743    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:35:25.168799 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:25.168783    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:35:26.085346 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:26.085314    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46" event={"ID":"b4e00663-6fca-498b-926e-716be61dff37","Type":"ContainerStarted","Data":"a427ec5de8d06f85ca8c4d8bfc565986c0d431c6499242b2fde078b4232e6c4e"}
Sep 10 19:35:26.085754 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:26.085363    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46" event={"ID":"b4e00663-6fca-498b-926e-716be61dff37","Type":"ContainerStarted","Data":"b52bb21f81d7a4353e55bfed0e62d22b20438816d6c56726a5686fc9403fcad6"}
Sep 10 19:35:26.110872 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:26.110813    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46" podStartSLOduration=2.110797844 podStartE2EDuration="2.110797844s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:35:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:35:26.109727558 +0000 UTC m=+1181.710634800" watchObservedRunningTime="2026-09-10 19:35:26.110797844 +0000 UTC m=+1181.711705083"
Sep 10 19:35:26.876032 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:26.875988    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46"
Sep 10 19:35:26.881184 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:26.881159    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46"
Sep 10 19:35:27.051468 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:27.051435    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:27.090910 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:27.090879    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46"
Sep 10 19:35:27.092006 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:27.091988    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46"
Sep 10 19:35:28.098725 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:28.098686    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:28.191154 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:28.191111    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:29.303898 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:29.303853    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:29.360093 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:29.360054    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:29.626137 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:29.626045    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:29.727426 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:29.727376    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:29.909934 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:29.909833    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:30.006798 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:30.006764    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:40.776647 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:40.761143    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:46.202455 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:46.202406    2564 generic.go:356] "Generic (PLEG): container finished" podID="b4e00663-6fca-498b-926e-716be61dff37" containerID="a427ec5de8d06f85ca8c4d8bfc565986c0d431c6499242b2fde078b4232e6c4e" exitCode=0
Sep 10 19:35:46.202919 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:46.202481    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46" event={"ID":"b4e00663-6fca-498b-926e-716be61dff37","Type":"ContainerDied","Data":"a427ec5de8d06f85ca8c4d8bfc565986c0d431c6499242b2fde078b4232e6c4e"}
Sep 10 19:35:46.203002 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:46.202988    2564 scope.go:117] "RemoveContainer" containerID="a427ec5de8d06f85ca8c4d8bfc565986c0d431c6499242b2fde078b4232e6c4e"
Sep 10 19:35:46.204220 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:46.204189    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:35:46.204345 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:46.204258    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:35:46.204345 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:46.204292    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:35:47.210195 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:47.210159    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46" event={"ID":"b4e00663-6fca-498b-926e-716be61dff37","Type":"ContainerStarted","Data":"e984a042c1f2460152cf4e1a77f41af578f86cded44cc4f353ec087ba863aad1"}
Sep 10 19:35:47.875700 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:47.875660    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46"
Sep 10 19:35:47.875949 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:47.875923    2564 patch_prober.go:28] interesting pod/e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Sep 10 19:35:47.876124 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:47.876097    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46" podUID="b4e00663-6fca-498b-926e-716be61dff37" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Sep 10 19:35:48.875581 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:48.875539    2564 patch_prober.go:28] interesting pod/e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Sep 10 19:35:48.876609 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:48.876199    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46" podUID="b4e00663-6fca-498b-926e-716be61dff37" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Sep 10 19:35:49.876217 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:49.876169    2564 patch_prober.go:28] interesting pod/e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Sep 10 19:35:49.876739 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:49.876256    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46" podUID="b4e00663-6fca-498b-926e-716be61dff37" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Sep 10 19:35:50.876100 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:50.876051    2564 patch_prober.go:28] interesting pod/e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Sep 10 19:35:50.876438 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:50.876404    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46" podUID="b4e00663-6fca-498b-926e-716be61dff37" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Sep 10 19:35:51.876020 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:51.875979    2564 patch_prober.go:28] interesting pod/e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Sep 10 19:35:51.876330 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:51.876301    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46" podUID="b4e00663-6fca-498b-926e-716be61dff37" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Sep 10 19:35:52.879556 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:52.879527    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46"
Sep 10 19:35:52.879950 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:52.879930    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46"
Sep 10 19:35:52.880604 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:52.880581    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46"
Sep 10 19:35:53.082335 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:35:53.082299    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:01.912785 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:01.912745    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:02.197530 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:02.197445    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj"]
Sep 10 19:36:02.202808 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:02.202781    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj"
Sep 10 19:36:02.205484 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:02.205454    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-26a424d6-openshift-default-dockercfg-rpp49\""
Sep 10 19:36:02.211608 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:02.211578    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj"]
Sep 10 19:36:02.225108 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:02.225077    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:02.356507 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:02.356471    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/874928dd-bd01-4675-8aa8-84af5a92f21d-istio-token\") pod \"e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj\" (UID: \"874928dd-bd01-4675-8aa8-84af5a92f21d\") " pod="openshift-ingress/e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj"
Sep 10 19:36:02.356729 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:02.356519    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/874928dd-bd01-4675-8aa8-84af5a92f21d-workload-certs\") pod \"e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj\" (UID: \"874928dd-bd01-4675-8aa8-84af5a92f21d\") " pod="openshift-ingress/e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj"
Sep 10 19:36:02.356729 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:02.356587    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxfv7\" (UniqueName: \"kubernetes.io/projected/874928dd-bd01-4675-8aa8-84af5a92f21d-kube-api-access-dxfv7\") pod \"e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj\" (UID: \"874928dd-bd01-4675-8aa8-84af5a92f21d\") " pod="openshift-ingress/e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj"
Sep 10 19:36:02.356729 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:02.356666    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/874928dd-bd01-4675-8aa8-84af5a92f21d-istio-data\") pod \"e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj\" (UID: \"874928dd-bd01-4675-8aa8-84af5a92f21d\") " pod="openshift-ingress/e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj"
Sep 10 19:36:02.356729 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:02.356708    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/874928dd-bd01-4675-8aa8-84af5a92f21d-istio-envoy\") pod \"e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj\" (UID: \"874928dd-bd01-4675-8aa8-84af5a92f21d\") " pod="openshift-ingress/e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj"
Sep 10 19:36:02.356729 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:02.356723    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/874928dd-bd01-4675-8aa8-84af5a92f21d-istio-podinfo\") pod \"e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj\" (UID: \"874928dd-bd01-4675-8aa8-84af5a92f21d\") " pod="openshift-ingress/e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj"
Sep 10 19:36:02.356930 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:02.356801    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/874928dd-bd01-4675-8aa8-84af5a92f21d-workload-socket\") pod \"e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj\" (UID: \"874928dd-bd01-4675-8aa8-84af5a92f21d\") " pod="openshift-ingress/e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj"
Sep 10 19:36:02.356930 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:02.356853    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/874928dd-bd01-4675-8aa8-84af5a92f21d-credential-socket\") pod \"e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj\" (UID: \"874928dd-bd01-4675-8aa8-84af5a92f21d\") " pod="openshift-ingress/e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj"
Sep 10 19:36:02.356930 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:02.356895    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/874928dd-bd01-4675-8aa8-84af5a92f21d-istiod-ca-cert\") pod \"e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj\" (UID: \"874928dd-bd01-4675-8aa8-84af5a92f21d\") " pod="openshift-ingress/e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj"
Sep 10 19:36:02.458008 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:02.457915    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/874928dd-bd01-4675-8aa8-84af5a92f21d-istio-podinfo\") pod \"e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj\" (UID: \"874928dd-bd01-4675-8aa8-84af5a92f21d\") " pod="openshift-ingress/e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj"
Sep 10 19:36:02.458008 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:02.457985    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/874928dd-bd01-4675-8aa8-84af5a92f21d-workload-socket\") pod \"e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj\" (UID: \"874928dd-bd01-4675-8aa8-84af5a92f21d\") " pod="openshift-ingress/e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj"
Sep 10 19:36:02.458244 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:02.458015    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/874928dd-bd01-4675-8aa8-84af5a92f21d-credential-socket\") pod \"e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj\" (UID: \"874928dd-bd01-4675-8aa8-84af5a92f21d\") " pod="openshift-ingress/e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj"
Sep 10 19:36:02.458244 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:02.458056    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/874928dd-bd01-4675-8aa8-84af5a92f21d-istiod-ca-cert\") pod \"e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj\" (UID: \"874928dd-bd01-4675-8aa8-84af5a92f21d\") " pod="openshift-ingress/e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj"
Sep 10 19:36:02.458244 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:02.458092    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/874928dd-bd01-4675-8aa8-84af5a92f21d-istio-token\") pod \"e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj\" (UID: \"874928dd-bd01-4675-8aa8-84af5a92f21d\") " pod="openshift-ingress/e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj"
Sep 10 19:36:02.458244 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:02.458127    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/874928dd-bd01-4675-8aa8-84af5a92f21d-workload-certs\") pod \"e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj\" (UID: \"874928dd-bd01-4675-8aa8-84af5a92f21d\") " pod="openshift-ingress/e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj"
Sep 10 19:36:02.458244 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:02.458159    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxfv7\" (UniqueName: \"kubernetes.io/projected/874928dd-bd01-4675-8aa8-84af5a92f21d-kube-api-access-dxfv7\") pod \"e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj\" (UID: \"874928dd-bd01-4675-8aa8-84af5a92f21d\") " pod="openshift-ingress/e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj"
Sep 10 19:36:02.458244 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:02.458208    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/874928dd-bd01-4675-8aa8-84af5a92f21d-istio-data\") pod \"e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj\" (UID: \"874928dd-bd01-4675-8aa8-84af5a92f21d\") " pod="openshift-ingress/e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj"
Sep 10 19:36:02.458694 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:02.458263    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/874928dd-bd01-4675-8aa8-84af5a92f21d-istio-envoy\") pod \"e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj\" (UID: \"874928dd-bd01-4675-8aa8-84af5a92f21d\") " pod="openshift-ingress/e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj"
Sep 10 19:36:02.458694 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:02.458453    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/874928dd-bd01-4675-8aa8-84af5a92f21d-workload-socket\") pod \"e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj\" (UID: \"874928dd-bd01-4675-8aa8-84af5a92f21d\") " pod="openshift-ingress/e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj"
Sep 10 19:36:02.458694 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:02.458507    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/874928dd-bd01-4675-8aa8-84af5a92f21d-credential-socket\") pod \"e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj\" (UID: \"874928dd-bd01-4675-8aa8-84af5a92f21d\") " pod="openshift-ingress/e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj"
Sep 10 19:36:02.458694 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:02.458517    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/874928dd-bd01-4675-8aa8-84af5a92f21d-workload-certs\") pod \"e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj\" (UID: \"874928dd-bd01-4675-8aa8-84af5a92f21d\") " pod="openshift-ingress/e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj"
Sep 10 19:36:02.458879 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:02.458728    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/874928dd-bd01-4675-8aa8-84af5a92f21d-istio-data\") pod \"e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj\" (UID: \"874928dd-bd01-4675-8aa8-84af5a92f21d\") " pod="openshift-ingress/e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj"
Sep 10 19:36:02.459002 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:02.458984    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/874928dd-bd01-4675-8aa8-84af5a92f21d-istiod-ca-cert\") pod \"e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj\" (UID: \"874928dd-bd01-4675-8aa8-84af5a92f21d\") " pod="openshift-ingress/e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj"
Sep 10 19:36:02.460572 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:02.460544    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/874928dd-bd01-4675-8aa8-84af5a92f21d-istio-envoy\") pod \"e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj\" (UID: \"874928dd-bd01-4675-8aa8-84af5a92f21d\") " pod="openshift-ingress/e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj"
Sep 10 19:36:02.460939 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:02.460917    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/874928dd-bd01-4675-8aa8-84af5a92f21d-istio-podinfo\") pod \"e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj\" (UID: \"874928dd-bd01-4675-8aa8-84af5a92f21d\") " pod="openshift-ingress/e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj"
Sep 10 19:36:02.464241 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:02.464214    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:02.468090 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:02.468065    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/874928dd-bd01-4675-8aa8-84af5a92f21d-istio-token\") pod \"e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj\" (UID: \"874928dd-bd01-4675-8aa8-84af5a92f21d\") " pod="openshift-ingress/e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj"
Sep 10 19:36:02.468382 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:02.468358    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxfv7\" (UniqueName: \"kubernetes.io/projected/874928dd-bd01-4675-8aa8-84af5a92f21d-kube-api-access-dxfv7\") pod \"e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj\" (UID: \"874928dd-bd01-4675-8aa8-84af5a92f21d\") " pod="openshift-ingress/e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj"
Sep 10 19:36:02.515974 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:02.515934    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj"
Sep 10 19:36:02.793363 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:02.793335    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj"]
Sep 10 19:36:02.795245 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:36:02.795208    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod874928dd_bd01_4675_8aa8_84af5a92f21d.slice/crio-7b3b186f643fc581b6487f4b8154d8bec4c8efbe35fedc54951b6f04f8404fc9 WatchSource:0}: Error finding container 7b3b186f643fc581b6487f4b8154d8bec4c8efbe35fedc54951b6f04f8404fc9: Status 404 returned error can't find the container with id 7b3b186f643fc581b6487f4b8154d8bec4c8efbe35fedc54951b6f04f8404fc9
Sep 10 19:36:02.797559 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:02.797522    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:36:02.797680 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:02.797593    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:36:02.797680 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:02.797653    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:36:03.306552 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:03.306519    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj" event={"ID":"874928dd-bd01-4675-8aa8-84af5a92f21d","Type":"ContainerStarted","Data":"a869210946f8988b78ca94d02840d6047be1cc2b358568e3b5f47a3c83568077"}
Sep 10 19:36:03.306976 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:03.306571    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj" event={"ID":"874928dd-bd01-4675-8aa8-84af5a92f21d","Type":"ContainerStarted","Data":"7b3b186f643fc581b6487f4b8154d8bec4c8efbe35fedc54951b6f04f8404fc9"}
Sep 10 19:36:03.330089 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:03.330029    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj" podStartSLOduration=1.330014128 podStartE2EDuration="1.330014128s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:36:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:36:03.327279994 +0000 UTC m=+1218.928187234" watchObservedRunningTime="2026-09-10 19:36:03.330014128 +0000 UTC m=+1218.930921388"
Sep 10 19:36:03.516579 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:03.516542    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj"
Sep 10 19:36:03.521838 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:03.521813    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj"
Sep 10 19:36:04.312892 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:04.312855    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj"
Sep 10 19:36:04.314345 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:04.314324    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj"
Sep 10 19:36:05.259442 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:05.259395    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:05.355862 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:05.355825    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:06.439672 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:06.439598    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:06.539767 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:06.539730    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:06.961957 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:06.961919    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:07.058081 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:07.058044    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:07.106770 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:07.106729    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-26a424d6-7ffc8667cc-xs54z"]
Sep 10 19:36:07.110563 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:07.110542    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-26a424d6-7ffc8667cc-xs54z"
Sep 10 19:36:07.113103 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:07.113082    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-26a424d6-dockercfg-2s5k6\""
Sep 10 19:36:07.113103 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:07.113097    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-26a424d6\""
Sep 10 19:36:07.121553 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:07.121529    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-26a424d6-7ffc8667cc-xs54z"]
Sep 10 19:36:07.208458 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:07.208407    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77qk2\" (UniqueName: \"kubernetes.io/projected/081388c4-c5b0-4670-b8d4-cfcf011be55a-kube-api-access-77qk2\") pod \"payload-processing-e2e-mt-26a424d6-7ffc8667cc-xs54z\" (UID: \"081388c4-c5b0-4670-b8d4-cfcf011be55a\") " pod="openshift-ingress/payload-processing-e2e-mt-26a424d6-7ffc8667cc-xs54z"
Sep 10 19:36:07.208779 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:07.208741    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/081388c4-c5b0-4670-b8d4-cfcf011be55a-plugins-config-volume\") pod \"payload-processing-e2e-mt-26a424d6-7ffc8667cc-xs54z\" (UID: \"081388c4-c5b0-4670-b8d4-cfcf011be55a\") " pod="openshift-ingress/payload-processing-e2e-mt-26a424d6-7ffc8667cc-xs54z"
Sep 10 19:36:07.310512 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:07.310469    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77qk2\" (UniqueName: \"kubernetes.io/projected/081388c4-c5b0-4670-b8d4-cfcf011be55a-kube-api-access-77qk2\") pod \"payload-processing-e2e-mt-26a424d6-7ffc8667cc-xs54z\" (UID: \"081388c4-c5b0-4670-b8d4-cfcf011be55a\") " pod="openshift-ingress/payload-processing-e2e-mt-26a424d6-7ffc8667cc-xs54z"
Sep 10 19:36:07.310512 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:07.310526    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/081388c4-c5b0-4670-b8d4-cfcf011be55a-plugins-config-volume\") pod \"payload-processing-e2e-mt-26a424d6-7ffc8667cc-xs54z\" (UID: \"081388c4-c5b0-4670-b8d4-cfcf011be55a\") " pod="openshift-ingress/payload-processing-e2e-mt-26a424d6-7ffc8667cc-xs54z"
Sep 10 19:36:07.311316 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:07.311287    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/081388c4-c5b0-4670-b8d4-cfcf011be55a-plugins-config-volume\") pod \"payload-processing-e2e-mt-26a424d6-7ffc8667cc-xs54z\" (UID: \"081388c4-c5b0-4670-b8d4-cfcf011be55a\") " pod="openshift-ingress/payload-processing-e2e-mt-26a424d6-7ffc8667cc-xs54z"
Sep 10 19:36:07.319258 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:07.319229    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-77qk2\" (UniqueName: \"kubernetes.io/projected/081388c4-c5b0-4670-b8d4-cfcf011be55a-kube-api-access-77qk2\") pod \"payload-processing-e2e-mt-26a424d6-7ffc8667cc-xs54z\" (UID: \"081388c4-c5b0-4670-b8d4-cfcf011be55a\") " pod="openshift-ingress/payload-processing-e2e-mt-26a424d6-7ffc8667cc-xs54z"
Sep 10 19:36:07.421768 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:07.421730    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-26a424d6-7ffc8667cc-xs54z"
Sep 10 19:36:07.696547 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:07.696524    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-26a424d6-7ffc8667cc-xs54z"]
Sep 10 19:36:07.698696 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:36:07.698664    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod081388c4_c5b0_4670_b8d4_cfcf011be55a.slice/crio-7486341c9f2f6113b05cb7c4100e0f5423e61f06fc29db50f6befeed8c06ffbf WatchSource:0}: Error finding container 7486341c9f2f6113b05cb7c4100e0f5423e61f06fc29db50f6befeed8c06ffbf: Status 404 returned error can't find the container with id 7486341c9f2f6113b05cb7c4100e0f5423e61f06fc29db50f6befeed8c06ffbf
Sep 10 19:36:08.336515 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:08.336477    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-26a424d6-7ffc8667cc-xs54z" event={"ID":"081388c4-c5b0-4670-b8d4-cfcf011be55a","Type":"ContainerStarted","Data":"e05f620f05043cdc00125810ed7e1560f99deb16f7ef2e679425cb1aca8e8c7e"}
Sep 10 19:36:08.336708 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:08.336530    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-26a424d6-7ffc8667cc-xs54z" event={"ID":"081388c4-c5b0-4670-b8d4-cfcf011be55a","Type":"ContainerStarted","Data":"7486341c9f2f6113b05cb7c4100e0f5423e61f06fc29db50f6befeed8c06ffbf"}
Sep 10 19:36:08.336708 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:08.336556    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-26a424d6-7ffc8667cc-xs54z"
Sep 10 19:36:08.359535 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:08.359484    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-26a424d6-7ffc8667cc-xs54z" podStartSLOduration=1.359467241 podStartE2EDuration="1.359467241s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:36:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:36:08.357100714 +0000 UTC m=+1223.958007953" watchObservedRunningTime="2026-09-10 19:36:08.359467241 +0000 UTC m=+1223.960374480"
Sep 10 19:36:19.343797 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:19.343765    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-26a424d6-7ffc8667cc-xs54z"
Sep 10 19:36:25.446117 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:25.446073    2564 generic.go:356] "Generic (PLEG): container finished" podID="874928dd-bd01-4675-8aa8-84af5a92f21d" containerID="a869210946f8988b78ca94d02840d6047be1cc2b358568e3b5f47a3c83568077" exitCode=0
Sep 10 19:36:25.446546 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:25.446147    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj" event={"ID":"874928dd-bd01-4675-8aa8-84af5a92f21d","Type":"ContainerDied","Data":"a869210946f8988b78ca94d02840d6047be1cc2b358568e3b5f47a3c83568077"}
Sep 10 19:36:25.446604 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:25.446589    2564 scope.go:117] "RemoveContainer" containerID="a869210946f8988b78ca94d02840d6047be1cc2b358568e3b5f47a3c83568077"
Sep 10 19:36:25.447751 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:25.447722    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:36:25.447819 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:25.447788    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:36:25.447856 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:25.447817    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:36:26.453514 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:26.453480    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj" event={"ID":"874928dd-bd01-4675-8aa8-84af5a92f21d","Type":"ContainerStarted","Data":"4effff54104d2cd96fe6411b35ca4627a03fa828c911939e59caae9301876e14"}
Sep 10 19:36:26.516449 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:26.516409    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj"
Sep 10 19:36:26.516727 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:26.516706    2564 patch_prober.go:28] interesting pod/e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Sep 10 19:36:26.516778 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:26.516752    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj" podUID="874928dd-bd01-4675-8aa8-84af5a92f21d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Sep 10 19:36:27.516768 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:27.516733    2564 patch_prober.go:28] interesting pod/e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Sep 10 19:36:27.517150 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:27.516788    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj" podUID="874928dd-bd01-4675-8aa8-84af5a92f21d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Sep 10 19:36:28.448440 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:28.448407    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:28.517023 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:28.516950    2564 patch_prober.go:28] interesting pod/e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Sep 10 19:36:28.517489 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:28.517032    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj" podUID="874928dd-bd01-4675-8aa8-84af5a92f21d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Sep 10 19:36:29.517180 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:29.517143    2564 patch_prober.go:28] interesting pod/e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Sep 10 19:36:29.517573 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:29.517196    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj" podUID="874928dd-bd01-4675-8aa8-84af5a92f21d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Sep 10 19:36:29.903308 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:29.903213    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:30.256633 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:30.256584    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:30.357174 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:30.357136    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:30.517203 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:30.517098    2564 patch_prober.go:28] interesting pod/e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Sep 10 19:36:30.517203 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:30.517164    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj" podUID="874928dd-bd01-4675-8aa8-84af5a92f21d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Sep 10 19:36:31.516461 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:31.516425    2564 patch_prober.go:28] interesting pod/e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Sep 10 19:36:31.516667 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:31.516517    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj" podUID="874928dd-bd01-4675-8aa8-84af5a92f21d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Sep 10 19:36:32.591917 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:32.591879    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj"
Sep 10 19:36:32.592419 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:32.592223    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj"
Sep 10 19:36:32.594842 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:32.594783    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj"
Sep 10 19:36:45.120243 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:45.120167    2564 scope.go:117] "RemoveContainer" containerID="df8bd430fdd5a8eea7dc9046407c8450016f1dedb02624ec6439545d988683ed"
Sep 10 19:36:45.128649 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:36:45.128617    2564 scope.go:117] "RemoveContainer" containerID="bbac273b662039a55e874dc17dd0b9a44b88a61bf06ae7f5b5b17dfeb9bc9d0f"
Sep 10 19:37:08.849055 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:37:08.849006    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:06.655728 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:38:06.655694    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-26a424d6-vbg9z"]
Sep 10 19:38:06.659527 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:38:06.659509    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-26a424d6-vbg9z"
Sep 10 19:38:06.661985 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:38:06.661960    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tjmqn\""
Sep 10 19:38:06.662146 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:38:06.662004    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 19:38:06.667943 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:38:06.667423    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-26a424d6-vbg9z"]
Sep 10 19:38:06.713674 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:38:06.713611    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd6a5354-c246-4d2d-a2dc-cf67bca654d8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-26a424d6-vbg9z\" (UID: \"cd6a5354-c246-4d2d-a2dc-cf67bca654d8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-26a424d6-vbg9z"
Sep 10 19:38:06.814514 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:38:06.814470    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd6a5354-c246-4d2d-a2dc-cf67bca654d8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-26a424d6-vbg9z\" (UID: \"cd6a5354-c246-4d2d-a2dc-cf67bca654d8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-26a424d6-vbg9z"
Sep 10 19:38:06.815150 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:38:06.815128    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd6a5354-c246-4d2d-a2dc-cf67bca654d8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-26a424d6-vbg9z\" (UID: \"cd6a5354-c246-4d2d-a2dc-cf67bca654d8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-26a424d6-vbg9z"
Sep 10 19:38:06.973014 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:38:06.972934    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-26a424d6-vbg9z"
Sep 10 19:38:07.246752 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:38:07.246720    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-26a424d6-vbg9z"]
Sep 10 19:38:07.249296 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:38:07.249269    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd6a5354_c246_4d2d_a2dc_cf67bca654d8.slice/crio-737ea4d37359b8dbc6b3dc8d51ea5ecc382b15dec90b2e98bc1100fa5c522323 WatchSource:0}: Error finding container 737ea4d37359b8dbc6b3dc8d51ea5ecc382b15dec90b2e98bc1100fa5c522323: Status 404 returned error can't find the container with id 737ea4d37359b8dbc6b3dc8d51ea5ecc382b15dec90b2e98bc1100fa5c522323
Sep 10 19:38:08.048401 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:38:08.048361    2564 generic.go:356] "Generic (PLEG): container finished" podID="cd6a5354-c246-4d2d-a2dc-cf67bca654d8" containerID="34f727a62c4874b5ff5b7f84c97bec6f8b713673f07705d4df685405611b5add" exitCode=0
Sep 10 19:38:08.048794 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:38:08.048441    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-26a424d6-vbg9z" event={"ID":"cd6a5354-c246-4d2d-a2dc-cf67bca654d8","Type":"ContainerDied","Data":"34f727a62c4874b5ff5b7f84c97bec6f8b713673f07705d4df685405611b5add"}
Sep 10 19:38:08.048794 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:38:08.048490    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-26a424d6-vbg9z" event={"ID":"cd6a5354-c246-4d2d-a2dc-cf67bca654d8","Type":"ContainerStarted","Data":"737ea4d37359b8dbc6b3dc8d51ea5ecc382b15dec90b2e98bc1100fa5c522323"}
Sep 10 19:38:09.196308 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:38:09.196281    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-26a424d6-vbg9z"
Sep 10 19:38:09.237106 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:38:09.237064    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd6a5354-c246-4d2d-a2dc-cf67bca654d8-maas-api-service-ca\") pod \"cd6a5354-c246-4d2d-a2dc-cf67bca654d8\" (UID: \"cd6a5354-c246-4d2d-a2dc-cf67bca654d8\") "
Sep 10 19:38:09.237480 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:38:09.237451    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd6a5354-c246-4d2d-a2dc-cf67bca654d8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "cd6a5354-c246-4d2d-a2dc-cf67bca654d8" (UID: "cd6a5354-c246-4d2d-a2dc-cf67bca654d8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:38:09.339022 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:38:09.338925    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd6a5354-c246-4d2d-a2dc-cf67bca654d8-maas-api-service-ca\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:38:10.061400 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:38:10.061373    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-26a424d6-vbg9z"
Sep 10 19:38:10.061400 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:38:10.061390    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-26a424d6-vbg9z" event={"ID":"cd6a5354-c246-4d2d-a2dc-cf67bca654d8","Type":"ContainerDied","Data":"737ea4d37359b8dbc6b3dc8d51ea5ecc382b15dec90b2e98bc1100fa5c522323"}
Sep 10 19:38:10.061610 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:38:10.061445    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="737ea4d37359b8dbc6b3dc8d51ea5ecc382b15dec90b2e98bc1100fa5c522323"
Sep 10 19:38:22.288023 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:38:22.287977    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:22.316516 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:38:22.316475    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-26a424d6-7ffc8667cc-xs54z"]
Sep 10 19:38:22.316820 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:38:22.316785    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-26a424d6-7ffc8667cc-xs54z" podUID="081388c4-c5b0-4670-b8d4-cfcf011be55a" containerName="payload-processing" containerID="cri-o://e05f620f05043cdc00125810ed7e1560f99deb16f7ef2e679425cb1aca8e8c7e" gracePeriod=30
Sep 10 19:38:22.388484 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:38:22.388447    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:22.496655 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:38:22.496575    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:22.589961 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:38:22.589920    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:22.606351 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:38:22.606328    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-26a424d6-vbg9z"]
Sep 10 19:38:22.609407 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:38:22.609386    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-26a424d6-vbg9z"]
Sep 10 19:38:22.612684 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:38:22.612655    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-26a424d6-7ffc8667cc-xs54z"
Sep 10 19:38:22.654514 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:38:22.654478    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:22.669840 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:38:22.668598    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-77qk2\" (UniqueName: \"kubernetes.io/projected/081388c4-c5b0-4670-b8d4-cfcf011be55a-kube-api-access-77qk2\") pod \"081388c4-c5b0-4670-b8d4-cfcf011be55a\" (UID: \"081388c4-c5b0-4670-b8d4-cfcf011be55a\") "
Sep 10 19:38:22.669840 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:38:22.668764    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/081388c4-c5b0-4670-b8d4-cfcf011be55a-plugins-config-volume\") pod \"081388c4-c5b0-4670-b8d4-cfcf011be55a\" (UID: \"081388c4-c5b0-4670-b8d4-cfcf011be55a\") "
Sep 10 19:38:22.669840 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:38:22.669390    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/081388c4-c5b0-4670-b8d4-cfcf011be55a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "081388c4-c5b0-4670-b8d4-cfcf011be55a" (UID: "081388c4-c5b0-4670-b8d4-cfcf011be55a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:38:22.672039 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:38:22.672010    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/081388c4-c5b0-4670-b8d4-cfcf011be55a-kube-api-access-77qk2" (OuterVolumeSpecName: "kube-api-access-77qk2") pod "081388c4-c5b0-4670-b8d4-cfcf011be55a" (UID: "081388c4-c5b0-4670-b8d4-cfcf011be55a"). InnerVolumeSpecName "kube-api-access-77qk2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:38:22.770612 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:38:22.770574    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/081388c4-c5b0-4670-b8d4-cfcf011be55a-plugins-config-volume\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:38:22.770612 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:38:22.770605    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-77qk2\" (UniqueName: \"kubernetes.io/projected/081388c4-c5b0-4670-b8d4-cfcf011be55a-kube-api-access-77qk2\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:38:22.950311 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:38:22.950214    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd6a5354-c246-4d2d-a2dc-cf67bca654d8" path="/var/lib/kubelet/pods/cd6a5354-c246-4d2d-a2dc-cf67bca654d8/volumes"
Sep 10 19:38:23.138013 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:38:23.137971    2564 generic.go:356] "Generic (PLEG): container finished" podID="081388c4-c5b0-4670-b8d4-cfcf011be55a" containerID="e05f620f05043cdc00125810ed7e1560f99deb16f7ef2e679425cb1aca8e8c7e" exitCode=0
Sep 10 19:38:23.138013 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:38:23.138018    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-26a424d6-7ffc8667cc-xs54z"
Sep 10 19:38:23.138280 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:38:23.138063    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-26a424d6-7ffc8667cc-xs54z" event={"ID":"081388c4-c5b0-4670-b8d4-cfcf011be55a","Type":"ContainerDied","Data":"e05f620f05043cdc00125810ed7e1560f99deb16f7ef2e679425cb1aca8e8c7e"}
Sep 10 19:38:23.138280 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:38:23.138128    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-26a424d6-7ffc8667cc-xs54z" event={"ID":"081388c4-c5b0-4670-b8d4-cfcf011be55a","Type":"ContainerDied","Data":"7486341c9f2f6113b05cb7c4100e0f5423e61f06fc29db50f6befeed8c06ffbf"}
Sep 10 19:38:23.138280 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:38:23.138160    2564 scope.go:117] "RemoveContainer" containerID="e05f620f05043cdc00125810ed7e1560f99deb16f7ef2e679425cb1aca8e8c7e"
Sep 10 19:38:23.147316 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:38:23.147299    2564 scope.go:117] "RemoveContainer" containerID="e05f620f05043cdc00125810ed7e1560f99deb16f7ef2e679425cb1aca8e8c7e"
Sep 10 19:38:23.147656 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:38:23.147616    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e05f620f05043cdc00125810ed7e1560f99deb16f7ef2e679425cb1aca8e8c7e\": container with ID starting with e05f620f05043cdc00125810ed7e1560f99deb16f7ef2e679425cb1aca8e8c7e not found: ID does not exist" containerID="e05f620f05043cdc00125810ed7e1560f99deb16f7ef2e679425cb1aca8e8c7e"
Sep 10 19:38:23.147762 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:38:23.147665    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e05f620f05043cdc00125810ed7e1560f99deb16f7ef2e679425cb1aca8e8c7e"} err="failed to get container status \"e05f620f05043cdc00125810ed7e1560f99deb16f7ef2e679425cb1aca8e8c7e\": rpc error: code = NotFound desc = could not find container \"e05f620f05043cdc00125810ed7e1560f99deb16f7ef2e679425cb1aca8e8c7e\": container with ID starting with e05f620f05043cdc00125810ed7e1560f99deb16f7ef2e679425cb1aca8e8c7e not found: ID does not exist"
Sep 10 19:38:23.158864 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:38:23.158837    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-26a424d6-7ffc8667cc-xs54z"]
Sep 10 19:38:23.165412 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:38:23.165382    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-26a424d6-7ffc8667cc-xs54z"]
Sep 10 19:38:24.949719 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:38:24.949678    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="081388c4-c5b0-4670-b8d4-cfcf011be55a" path="/var/lib/kubelet/pods/081388c4-c5b0-4670-b8d4-cfcf011be55a/volumes"
Sep 10 19:38:25.871659 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:38:25.871611    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:30.950845 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:38:30.950807    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:43.232946 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:38:43.232908    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:54.835031 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:38:54.834993    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:54.935317 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:38:54.935271    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:07.083187 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:07.083147    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:08.554062 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:08.554017    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:14.370299 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:14.370257    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:14.424039 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:14.424002    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:14.640369 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:14.640274    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj"]
Sep 10 19:39:14.640670 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:14.640605    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj" podUID="874928dd-bd01-4675-8aa8-84af5a92f21d" containerName="istio-proxy" containerID="cri-o://4effff54104d2cd96fe6411b35ca4627a03fa828c911939e59caae9301876e14" gracePeriod=30
Sep 10 19:39:14.651270 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:14.651237    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:15.874226 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:15.874192    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:15.969082 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:15.969050    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:16.082649 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:16.082598    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:16.175416 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:16.175322    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:16.276318 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:16.276281    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:16.692643 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:16.692577    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:16.785926 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:16.785893    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:16.889024 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:16.888995    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:17.594794 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:17.594751    2564 patch_prober.go:28] interesting pod/e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 19:39:17.594972 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:17.594822    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj" podUID="874928dd-bd01-4675-8aa8-84af5a92f21d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 19:39:18.195771 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:18.195732    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:19.901996 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:19.901930    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj"
Sep 10 19:39:19.967544 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:19.967510    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/874928dd-bd01-4675-8aa8-84af5a92f21d-istio-podinfo\") pod \"874928dd-bd01-4675-8aa8-84af5a92f21d\" (UID: \"874928dd-bd01-4675-8aa8-84af5a92f21d\") "
Sep 10 19:39:19.967755 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:19.967559    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/874928dd-bd01-4675-8aa8-84af5a92f21d-istio-token\") pod \"874928dd-bd01-4675-8aa8-84af5a92f21d\" (UID: \"874928dd-bd01-4675-8aa8-84af5a92f21d\") "
Sep 10 19:39:19.967755 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:19.967603    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/874928dd-bd01-4675-8aa8-84af5a92f21d-istio-data\") pod \"874928dd-bd01-4675-8aa8-84af5a92f21d\" (UID: \"874928dd-bd01-4675-8aa8-84af5a92f21d\") "
Sep 10 19:39:19.967755 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:19.967642    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/874928dd-bd01-4675-8aa8-84af5a92f21d-workload-certs\") pod \"874928dd-bd01-4675-8aa8-84af5a92f21d\" (UID: \"874928dd-bd01-4675-8aa8-84af5a92f21d\") "
Sep 10 19:39:19.967755 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:19.967660    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dxfv7\" (UniqueName: \"kubernetes.io/projected/874928dd-bd01-4675-8aa8-84af5a92f21d-kube-api-access-dxfv7\") pod \"874928dd-bd01-4675-8aa8-84af5a92f21d\" (UID: \"874928dd-bd01-4675-8aa8-84af5a92f21d\") "
Sep 10 19:39:19.967755 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:19.967724    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/874928dd-bd01-4675-8aa8-84af5a92f21d-istio-envoy\") pod \"874928dd-bd01-4675-8aa8-84af5a92f21d\" (UID: \"874928dd-bd01-4675-8aa8-84af5a92f21d\") "
Sep 10 19:39:19.968030 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:19.967774    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/874928dd-bd01-4675-8aa8-84af5a92f21d-credential-socket\") pod \"874928dd-bd01-4675-8aa8-84af5a92f21d\" (UID: \"874928dd-bd01-4675-8aa8-84af5a92f21d\") "
Sep 10 19:39:19.968030 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:19.967810    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/874928dd-bd01-4675-8aa8-84af5a92f21d-istiod-ca-cert\") pod \"874928dd-bd01-4675-8aa8-84af5a92f21d\" (UID: \"874928dd-bd01-4675-8aa8-84af5a92f21d\") "
Sep 10 19:39:19.968030 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:19.967886    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/874928dd-bd01-4675-8aa8-84af5a92f21d-workload-socket\") pod \"874928dd-bd01-4675-8aa8-84af5a92f21d\" (UID: \"874928dd-bd01-4675-8aa8-84af5a92f21d\") "
Sep 10 19:39:19.968030 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:19.967924    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/874928dd-bd01-4675-8aa8-84af5a92f21d-istio-data" (OuterVolumeSpecName: "istio-data") pod "874928dd-bd01-4675-8aa8-84af5a92f21d" (UID: "874928dd-bd01-4675-8aa8-84af5a92f21d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:39:19.968030 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:19.967969    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/874928dd-bd01-4675-8aa8-84af5a92f21d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "874928dd-bd01-4675-8aa8-84af5a92f21d" (UID: "874928dd-bd01-4675-8aa8-84af5a92f21d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:39:19.968277 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:19.968177    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/874928dd-bd01-4675-8aa8-84af5a92f21d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "874928dd-bd01-4675-8aa8-84af5a92f21d" (UID: "874928dd-bd01-4675-8aa8-84af5a92f21d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:39:19.968336 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:19.968294    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/874928dd-bd01-4675-8aa8-84af5a92f21d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "874928dd-bd01-4675-8aa8-84af5a92f21d" (UID: "874928dd-bd01-4675-8aa8-84af5a92f21d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:39:19.968400 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:19.968328    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/874928dd-bd01-4675-8aa8-84af5a92f21d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "874928dd-bd01-4675-8aa8-84af5a92f21d" (UID: "874928dd-bd01-4675-8aa8-84af5a92f21d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:39:19.968400 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:19.968360    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/874928dd-bd01-4675-8aa8-84af5a92f21d-workload-socket\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:39:19.968400 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:19.968384    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/874928dd-bd01-4675-8aa8-84af5a92f21d-istio-data\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:39:19.968400 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:19.968398    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/874928dd-bd01-4675-8aa8-84af5a92f21d-workload-certs\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:39:19.970032 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:19.970008    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/874928dd-bd01-4675-8aa8-84af5a92f21d-kube-api-access-dxfv7" (OuterVolumeSpecName: "kube-api-access-dxfv7") pod "874928dd-bd01-4675-8aa8-84af5a92f21d" (UID: "874928dd-bd01-4675-8aa8-84af5a92f21d"). InnerVolumeSpecName "kube-api-access-dxfv7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:39:19.970188 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:19.970174    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/874928dd-bd01-4675-8aa8-84af5a92f21d-istio-token" (OuterVolumeSpecName: "istio-token") pod "874928dd-bd01-4675-8aa8-84af5a92f21d" (UID: "874928dd-bd01-4675-8aa8-84af5a92f21d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:39:19.970251 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:19.970219    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/874928dd-bd01-4675-8aa8-84af5a92f21d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "874928dd-bd01-4675-8aa8-84af5a92f21d" (UID: "874928dd-bd01-4675-8aa8-84af5a92f21d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 19:39:19.970335 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:19.970319    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/874928dd-bd01-4675-8aa8-84af5a92f21d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "874928dd-bd01-4675-8aa8-84af5a92f21d" (UID: "874928dd-bd01-4675-8aa8-84af5a92f21d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:39:20.069302 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:20.069269    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/874928dd-bd01-4675-8aa8-84af5a92f21d-istio-envoy\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:39:20.069302 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:20.069297    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/874928dd-bd01-4675-8aa8-84af5a92f21d-credential-socket\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:39:20.069302 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:20.069308    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/874928dd-bd01-4675-8aa8-84af5a92f21d-istiod-ca-cert\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:39:20.069549 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:20.069317    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/874928dd-bd01-4675-8aa8-84af5a92f21d-istio-podinfo\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:39:20.069549 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:20.069325    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/874928dd-bd01-4675-8aa8-84af5a92f21d-istio-token\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:39:20.069549 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:20.069333    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dxfv7\" (UniqueName: \"kubernetes.io/projected/874928dd-bd01-4675-8aa8-84af5a92f21d-kube-api-access-dxfv7\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:39:20.476138 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:20.476097    2564 generic.go:356] "Generic (PLEG): container finished" podID="874928dd-bd01-4675-8aa8-84af5a92f21d" containerID="4effff54104d2cd96fe6411b35ca4627a03fa828c911939e59caae9301876e14" exitCode=0
Sep 10 19:39:20.476318 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:20.476150    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj"
Sep 10 19:39:20.476318 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:20.476151    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj" event={"ID":"874928dd-bd01-4675-8aa8-84af5a92f21d","Type":"ContainerDied","Data":"4effff54104d2cd96fe6411b35ca4627a03fa828c911939e59caae9301876e14"}
Sep 10 19:39:20.476318 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:20.476212    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj" event={"ID":"874928dd-bd01-4675-8aa8-84af5a92f21d","Type":"ContainerDied","Data":"7b3b186f643fc581b6487f4b8154d8bec4c8efbe35fedc54951b6f04f8404fc9"}
Sep 10 19:39:20.476318 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:20.476237    2564 scope.go:117] "RemoveContainer" containerID="4effff54104d2cd96fe6411b35ca4627a03fa828c911939e59caae9301876e14"
Sep 10 19:39:20.485981 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:20.485961    2564 scope.go:117] "RemoveContainer" containerID="a869210946f8988b78ca94d02840d6047be1cc2b358568e3b5f47a3c83568077"
Sep 10 19:39:20.494238 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:20.494223    2564 scope.go:117] "RemoveContainer" containerID="4effff54104d2cd96fe6411b35ca4627a03fa828c911939e59caae9301876e14"
Sep 10 19:39:20.494491 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:39:20.494472    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4effff54104d2cd96fe6411b35ca4627a03fa828c911939e59caae9301876e14\": container with ID starting with 4effff54104d2cd96fe6411b35ca4627a03fa828c911939e59caae9301876e14 not found: ID does not exist" containerID="4effff54104d2cd96fe6411b35ca4627a03fa828c911939e59caae9301876e14"
Sep 10 19:39:20.494537 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:20.494500    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4effff54104d2cd96fe6411b35ca4627a03fa828c911939e59caae9301876e14"} err="failed to get container status \"4effff54104d2cd96fe6411b35ca4627a03fa828c911939e59caae9301876e14\": rpc error: code = NotFound desc = could not find container \"4effff54104d2cd96fe6411b35ca4627a03fa828c911939e59caae9301876e14\": container with ID starting with 4effff54104d2cd96fe6411b35ca4627a03fa828c911939e59caae9301876e14 not found: ID does not exist"
Sep 10 19:39:20.494537 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:20.494519    2564 scope.go:117] "RemoveContainer" containerID="a869210946f8988b78ca94d02840d6047be1cc2b358568e3b5f47a3c83568077"
Sep 10 19:39:20.494875 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:39:20.494858    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a869210946f8988b78ca94d02840d6047be1cc2b358568e3b5f47a3c83568077\": container with ID starting with a869210946f8988b78ca94d02840d6047be1cc2b358568e3b5f47a3c83568077 not found: ID does not exist" containerID="a869210946f8988b78ca94d02840d6047be1cc2b358568e3b5f47a3c83568077"
Sep 10 19:39:20.494967 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:20.494878    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a869210946f8988b78ca94d02840d6047be1cc2b358568e3b5f47a3c83568077"} err="failed to get container status \"a869210946f8988b78ca94d02840d6047be1cc2b358568e3b5f47a3c83568077\": rpc error: code = NotFound desc = could not find container \"a869210946f8988b78ca94d02840d6047be1cc2b358568e3b5f47a3c83568077\": container with ID starting with a869210946f8988b78ca94d02840d6047be1cc2b358568e3b5f47a3c83568077 not found: ID does not exist"
Sep 10 19:39:20.500476 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:20.500455    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj"]
Sep 10 19:39:20.504971 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:20.504948    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-26a424d6-openshift-default-5d6bb7bb96-dswgj"]
Sep 10 19:39:20.950760 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:20.950719    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="874928dd-bd01-4675-8aa8-84af5a92f21d" path="/var/lib/kubelet/pods/874928dd-bd01-4675-8aa8-84af5a92f21d/volumes"
Sep 10 19:39:22.121616 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:22.121582    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:22.221122 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:22.221086    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:24.306259 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:24.306220    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:24.370639 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:24.370588    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:25.046907 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:25.046868    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:31.215299 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:31.215257    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:31.317729 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:31.317693    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:38.077294 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:38.077220    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:39.320753 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:39.320719    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:41.835083 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:41.835040    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:42.072351 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:42.072312    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:45.820674 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:45.820617    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:59.450231 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:39:59.450187    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:03.063385 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:03.063342    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:06.819881 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:06.819846    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:09.592598 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:09.592563    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:09.644912 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:09.644863    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:09.955808 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:09.955718    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:10.265544 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:10.265509    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-846240-openshift-default-5f9c644d-kgh6j"]
Sep 10 19:40:10.266203 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:10.266182    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd6a5354-c246-4d2d-a2dc-cf67bca654d8" containerName="revoke-keys"
Sep 10 19:40:10.266287 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:10.266206    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd6a5354-c246-4d2d-a2dc-cf67bca654d8" containerName="revoke-keys"
Sep 10 19:40:10.266287 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:10.266243    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="874928dd-bd01-4675-8aa8-84af5a92f21d" containerName="istio-proxy"
Sep 10 19:40:10.266287 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:10.266251    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="874928dd-bd01-4675-8aa8-84af5a92f21d" containerName="istio-proxy"
Sep 10 19:40:10.266287 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:10.266262    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="081388c4-c5b0-4670-b8d4-cfcf011be55a" containerName="payload-processing"
Sep 10 19:40:10.266287 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:10.266271    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="081388c4-c5b0-4670-b8d4-cfcf011be55a" containerName="payload-processing"
Sep 10 19:40:10.266532 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:10.266378    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="874928dd-bd01-4675-8aa8-84af5a92f21d" containerName="istio-proxy"
Sep 10 19:40:10.266532 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:10.266389    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd6a5354-c246-4d2d-a2dc-cf67bca654d8" containerName="revoke-keys"
Sep 10 19:40:10.266532 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:10.266401    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="874928dd-bd01-4675-8aa8-84af5a92f21d" containerName="istio-proxy"
Sep 10 19:40:10.266532 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:10.266413    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="081388c4-c5b0-4670-b8d4-cfcf011be55a" containerName="payload-processing"
Sep 10 19:40:10.271397 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:10.271375    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-846240-openshift-default-5f9c644d-kgh6j"
Sep 10 19:40:10.274611 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:10.274589    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-846240-openshift-default-dockercfg-9rrk7\""
Sep 10 19:40:10.281237 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:10.281210    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-846240-openshift-default-5f9c644d-kgh6j"]
Sep 10 19:40:10.284956 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:10.284921    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:10.375551 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:10.375515    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7777610-d77a-42c5-b6ed-60cabe303d70-istiod-ca-cert\") pod \"e2e-api-a-846240-openshift-default-5f9c644d-kgh6j\" (UID: \"b7777610-d77a-42c5-b6ed-60cabe303d70\") " pod="openshift-ingress/e2e-api-a-846240-openshift-default-5f9c644d-kgh6j"
Sep 10 19:40:10.375551 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:10.375571    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7777610-d77a-42c5-b6ed-60cabe303d70-credential-socket\") pod \"e2e-api-a-846240-openshift-default-5f9c644d-kgh6j\" (UID: \"b7777610-d77a-42c5-b6ed-60cabe303d70\") " pod="openshift-ingress/e2e-api-a-846240-openshift-default-5f9c644d-kgh6j"
Sep 10 19:40:10.375879 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:10.375684    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldthl\" (UniqueName: \"kubernetes.io/projected/b7777610-d77a-42c5-b6ed-60cabe303d70-kube-api-access-ldthl\") pod \"e2e-api-a-846240-openshift-default-5f9c644d-kgh6j\" (UID: \"b7777610-d77a-42c5-b6ed-60cabe303d70\") " pod="openshift-ingress/e2e-api-a-846240-openshift-default-5f9c644d-kgh6j"
Sep 10 19:40:10.375879 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:10.375730    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7777610-d77a-42c5-b6ed-60cabe303d70-istio-token\") pod \"e2e-api-a-846240-openshift-default-5f9c644d-kgh6j\" (UID: \"b7777610-d77a-42c5-b6ed-60cabe303d70\") " pod="openshift-ingress/e2e-api-a-846240-openshift-default-5f9c644d-kgh6j"
Sep 10 19:40:10.375879 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:10.375774    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7777610-d77a-42c5-b6ed-60cabe303d70-istio-envoy\") pod \"e2e-api-a-846240-openshift-default-5f9c644d-kgh6j\" (UID: \"b7777610-d77a-42c5-b6ed-60cabe303d70\") " pod="openshift-ingress/e2e-api-a-846240-openshift-default-5f9c644d-kgh6j"
Sep 10 19:40:10.375879 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:10.375789    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7777610-d77a-42c5-b6ed-60cabe303d70-istio-podinfo\") pod \"e2e-api-a-846240-openshift-default-5f9c644d-kgh6j\" (UID: \"b7777610-d77a-42c5-b6ed-60cabe303d70\") " pod="openshift-ingress/e2e-api-a-846240-openshift-default-5f9c644d-kgh6j"
Sep 10 19:40:10.376130 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:10.375895    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7777610-d77a-42c5-b6ed-60cabe303d70-workload-certs\") pod \"e2e-api-a-846240-openshift-default-5f9c644d-kgh6j\" (UID: \"b7777610-d77a-42c5-b6ed-60cabe303d70\") " pod="openshift-ingress/e2e-api-a-846240-openshift-default-5f9c644d-kgh6j"
Sep 10 19:40:10.376130 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:10.375920    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7777610-d77a-42c5-b6ed-60cabe303d70-istio-data\") pod \"e2e-api-a-846240-openshift-default-5f9c644d-kgh6j\" (UID: \"b7777610-d77a-42c5-b6ed-60cabe303d70\") " pod="openshift-ingress/e2e-api-a-846240-openshift-default-5f9c644d-kgh6j"
Sep 10 19:40:10.376130 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:10.375942    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7777610-d77a-42c5-b6ed-60cabe303d70-workload-socket\") pod \"e2e-api-a-846240-openshift-default-5f9c644d-kgh6j\" (UID: \"b7777610-d77a-42c5-b6ed-60cabe303d70\") " pod="openshift-ingress/e2e-api-a-846240-openshift-default-5f9c644d-kgh6j"
Sep 10 19:40:10.476917 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:10.476887    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7777610-d77a-42c5-b6ed-60cabe303d70-credential-socket\") pod \"e2e-api-a-846240-openshift-default-5f9c644d-kgh6j\" (UID: \"b7777610-d77a-42c5-b6ed-60cabe303d70\") " pod="openshift-ingress/e2e-api-a-846240-openshift-default-5f9c644d-kgh6j"
Sep 10 19:40:10.477111 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:10.476940    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldthl\" (UniqueName: \"kubernetes.io/projected/b7777610-d77a-42c5-b6ed-60cabe303d70-kube-api-access-ldthl\") pod \"e2e-api-a-846240-openshift-default-5f9c644d-kgh6j\" (UID: \"b7777610-d77a-42c5-b6ed-60cabe303d70\") " pod="openshift-ingress/e2e-api-a-846240-openshift-default-5f9c644d-kgh6j"
Sep 10 19:40:10.477111 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:10.476964    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7777610-d77a-42c5-b6ed-60cabe303d70-istio-token\") pod \"e2e-api-a-846240-openshift-default-5f9c644d-kgh6j\" (UID: \"b7777610-d77a-42c5-b6ed-60cabe303d70\") " pod="openshift-ingress/e2e-api-a-846240-openshift-default-5f9c644d-kgh6j"
Sep 10 19:40:10.477111 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:10.476994    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7777610-d77a-42c5-b6ed-60cabe303d70-istio-envoy\") pod \"e2e-api-a-846240-openshift-default-5f9c644d-kgh6j\" (UID: \"b7777610-d77a-42c5-b6ed-60cabe303d70\") " pod="openshift-ingress/e2e-api-a-846240-openshift-default-5f9c644d-kgh6j"
Sep 10 19:40:10.477111 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:10.477009    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7777610-d77a-42c5-b6ed-60cabe303d70-istio-podinfo\") pod \"e2e-api-a-846240-openshift-default-5f9c644d-kgh6j\" (UID: \"b7777610-d77a-42c5-b6ed-60cabe303d70\") " pod="openshift-ingress/e2e-api-a-846240-openshift-default-5f9c644d-kgh6j"
Sep 10 19:40:10.477111 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:10.477052    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7777610-d77a-42c5-b6ed-60cabe303d70-workload-certs\") pod \"e2e-api-a-846240-openshift-default-5f9c644d-kgh6j\" (UID: \"b7777610-d77a-42c5-b6ed-60cabe303d70\") " pod="openshift-ingress/e2e-api-a-846240-openshift-default-5f9c644d-kgh6j"
Sep 10 19:40:10.477111 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:10.477075    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7777610-d77a-42c5-b6ed-60cabe303d70-istio-data\") pod \"e2e-api-a-846240-openshift-default-5f9c644d-kgh6j\" (UID: \"b7777610-d77a-42c5-b6ed-60cabe303d70\") " pod="openshift-ingress/e2e-api-a-846240-openshift-default-5f9c644d-kgh6j"
Sep 10 19:40:10.477111 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:10.477096    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7777610-d77a-42c5-b6ed-60cabe303d70-workload-socket\") pod \"e2e-api-a-846240-openshift-default-5f9c644d-kgh6j\" (UID: \"b7777610-d77a-42c5-b6ed-60cabe303d70\") " pod="openshift-ingress/e2e-api-a-846240-openshift-default-5f9c644d-kgh6j"
Sep 10 19:40:10.477495 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:10.477157    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7777610-d77a-42c5-b6ed-60cabe303d70-istiod-ca-cert\") pod \"e2e-api-a-846240-openshift-default-5f9c644d-kgh6j\" (UID: \"b7777610-d77a-42c5-b6ed-60cabe303d70\") " pod="openshift-ingress/e2e-api-a-846240-openshift-default-5f9c644d-kgh6j"
Sep 10 19:40:10.477551 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:10.477491    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7777610-d77a-42c5-b6ed-60cabe303d70-workload-socket\") pod \"e2e-api-a-846240-openshift-default-5f9c644d-kgh6j\" (UID: \"b7777610-d77a-42c5-b6ed-60cabe303d70\") " pod="openshift-ingress/e2e-api-a-846240-openshift-default-5f9c644d-kgh6j"
Sep 10 19:40:10.477604 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:10.477545    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7777610-d77a-42c5-b6ed-60cabe303d70-credential-socket\") pod \"e2e-api-a-846240-openshift-default-5f9c644d-kgh6j\" (UID: \"b7777610-d77a-42c5-b6ed-60cabe303d70\") " pod="openshift-ingress/e2e-api-a-846240-openshift-default-5f9c644d-kgh6j"
Sep 10 19:40:10.477689 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:10.477616    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7777610-d77a-42c5-b6ed-60cabe303d70-istio-data\") pod \"e2e-api-a-846240-openshift-default-5f9c644d-kgh6j\" (UID: \"b7777610-d77a-42c5-b6ed-60cabe303d70\") " pod="openshift-ingress/e2e-api-a-846240-openshift-default-5f9c644d-kgh6j"
Sep 10 19:40:10.477689 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:10.477658    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7777610-d77a-42c5-b6ed-60cabe303d70-workload-certs\") pod \"e2e-api-a-846240-openshift-default-5f9c644d-kgh6j\" (UID: \"b7777610-d77a-42c5-b6ed-60cabe303d70\") " pod="openshift-ingress/e2e-api-a-846240-openshift-default-5f9c644d-kgh6j"
Sep 10 19:40:10.477883 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:10.477864    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7777610-d77a-42c5-b6ed-60cabe303d70-istiod-ca-cert\") pod \"e2e-api-a-846240-openshift-default-5f9c644d-kgh6j\" (UID: \"b7777610-d77a-42c5-b6ed-60cabe303d70\") " pod="openshift-ingress/e2e-api-a-846240-openshift-default-5f9c644d-kgh6j"
Sep 10 19:40:10.479263 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:10.479240    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7777610-d77a-42c5-b6ed-60cabe303d70-istio-envoy\") pod \"e2e-api-a-846240-openshift-default-5f9c644d-kgh6j\" (UID: \"b7777610-d77a-42c5-b6ed-60cabe303d70\") " pod="openshift-ingress/e2e-api-a-846240-openshift-default-5f9c644d-kgh6j"
Sep 10 19:40:10.479526 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:10.479508    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7777610-d77a-42c5-b6ed-60cabe303d70-istio-podinfo\") pod \"e2e-api-a-846240-openshift-default-5f9c644d-kgh6j\" (UID: \"b7777610-d77a-42c5-b6ed-60cabe303d70\") " pod="openshift-ingress/e2e-api-a-846240-openshift-default-5f9c644d-kgh6j"
Sep 10 19:40:10.485939 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:10.485920    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldthl\" (UniqueName: \"kubernetes.io/projected/b7777610-d77a-42c5-b6ed-60cabe303d70-kube-api-access-ldthl\") pod \"e2e-api-a-846240-openshift-default-5f9c644d-kgh6j\" (UID: \"b7777610-d77a-42c5-b6ed-60cabe303d70\") " pod="openshift-ingress/e2e-api-a-846240-openshift-default-5f9c644d-kgh6j"
Sep 10 19:40:10.486031 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:10.486002    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7777610-d77a-42c5-b6ed-60cabe303d70-istio-token\") pod \"e2e-api-a-846240-openshift-default-5f9c644d-kgh6j\" (UID: \"b7777610-d77a-42c5-b6ed-60cabe303d70\") " pod="openshift-ingress/e2e-api-a-846240-openshift-default-5f9c644d-kgh6j"
Sep 10 19:40:10.582273 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:10.582176    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-846240-openshift-default-5f9c644d-kgh6j"
Sep 10 19:40:10.860474 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:10.860438    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-846240-openshift-default-5f9c644d-kgh6j"]
Sep 10 19:40:10.863028 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:40:10.862992    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7777610_d77a_42c5_b6ed_60cabe303d70.slice/crio-9b7c928863abd9f77523e2650d53ae4c89a056fa630b48d706b35250ef55fa2e WatchSource:0}: Error finding container 9b7c928863abd9f77523e2650d53ae4c89a056fa630b48d706b35250ef55fa2e: Status 404 returned error can't find the container with id 9b7c928863abd9f77523e2650d53ae4c89a056fa630b48d706b35250ef55fa2e
Sep 10 19:40:10.865686 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:10.865645    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:40:10.865803 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:10.865744    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:40:10.865803 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:10.865783    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:40:11.049363 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:11.049322    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:11.781636 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:11.781587    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-846240-openshift-default-5f9c644d-kgh6j" event={"ID":"b7777610-d77a-42c5-b6ed-60cabe303d70","Type":"ContainerStarted","Data":"5c2194f26685fc7d337fbe48a3e08cf04780c70d7e4d948d441ae74511b5dd9e"}
Sep 10 19:40:11.781840 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:11.781667    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-846240-openshift-default-5f9c644d-kgh6j" event={"ID":"b7777610-d77a-42c5-b6ed-60cabe303d70","Type":"ContainerStarted","Data":"9b7c928863abd9f77523e2650d53ae4c89a056fa630b48d706b35250ef55fa2e"}
Sep 10 19:40:11.804046 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:11.803987    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-846240-openshift-default-5f9c644d-kgh6j" podStartSLOduration=1.8039663030000002 podStartE2EDuration="1.803966303s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:40:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:40:11.80382615 +0000 UTC m=+1467.404733391" watchObservedRunningTime="2026-09-10 19:40:11.803966303 +0000 UTC m=+1467.404873543"
Sep 10 19:40:12.583053 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:12.583019    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-846240-openshift-default-5f9c644d-kgh6j"
Sep 10 19:40:12.588783 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:12.588756    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-846240-openshift-default-5f9c644d-kgh6j"
Sep 10 19:40:12.787728 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:12.787701    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-846240-openshift-default-5f9c644d-kgh6j"
Sep 10 19:40:12.788833 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:12.788812    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-846240-openshift-default-5f9c644d-kgh6j"
Sep 10 19:40:13.753312 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:13.753264    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:13.845857 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:13.845828    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:14.926746 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:14.926707    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:15.027749 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:15.027704    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:15.271138 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:15.271100    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:15.433100 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:15.433060    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:15.528361 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:15.528255    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:15.529895 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:15.529874    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-846240-bf54c4c4b-247nf"]
Sep 10 19:40:15.530314 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:15.530302    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="874928dd-bd01-4675-8aa8-84af5a92f21d" containerName="istio-proxy"
Sep 10 19:40:15.530364 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:15.530315    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="874928dd-bd01-4675-8aa8-84af5a92f21d" containerName="istio-proxy"
Sep 10 19:40:15.533550 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:15.533533    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-846240-bf54c4c4b-247nf"
Sep 10 19:40:15.535585 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:15.535565    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-846240\""
Sep 10 19:40:15.544431 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:15.544401    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-846240-bf54c4c4b-247nf"]
Sep 10 19:40:15.633073 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:15.633028    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jw4js\" (UniqueName: \"kubernetes.io/projected/901e511b-f3d5-44f1-ba9b-29f7372f4bb5-kube-api-access-jw4js\") pod \"payload-pre-processing-e2e-api-a-846240-bf54c4c4b-247nf\" (UID: \"901e511b-f3d5-44f1-ba9b-29f7372f4bb5\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-846240-bf54c4c4b-247nf"
Sep 10 19:40:15.633073 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:15.633076    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/901e511b-f3d5-44f1-ba9b-29f7372f4bb5-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-846240-bf54c4c4b-247nf\" (UID: \"901e511b-f3d5-44f1-ba9b-29f7372f4bb5\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-846240-bf54c4c4b-247nf"
Sep 10 19:40:15.733720 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:15.733676    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jw4js\" (UniqueName: \"kubernetes.io/projected/901e511b-f3d5-44f1-ba9b-29f7372f4bb5-kube-api-access-jw4js\") pod \"payload-pre-processing-e2e-api-a-846240-bf54c4c4b-247nf\" (UID: \"901e511b-f3d5-44f1-ba9b-29f7372f4bb5\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-846240-bf54c4c4b-247nf"
Sep 10 19:40:15.733720 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:15.733726    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/901e511b-f3d5-44f1-ba9b-29f7372f4bb5-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-846240-bf54c4c4b-247nf\" (UID: \"901e511b-f3d5-44f1-ba9b-29f7372f4bb5\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-846240-bf54c4c4b-247nf"
Sep 10 19:40:15.734439 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:15.734419    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/901e511b-f3d5-44f1-ba9b-29f7372f4bb5-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-846240-bf54c4c4b-247nf\" (UID: \"901e511b-f3d5-44f1-ba9b-29f7372f4bb5\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-846240-bf54c4c4b-247nf"
Sep 10 19:40:15.742741 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:15.742716    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jw4js\" (UniqueName: \"kubernetes.io/projected/901e511b-f3d5-44f1-ba9b-29f7372f4bb5-kube-api-access-jw4js\") pod \"payload-pre-processing-e2e-api-a-846240-bf54c4c4b-247nf\" (UID: \"901e511b-f3d5-44f1-ba9b-29f7372f4bb5\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-846240-bf54c4c4b-247nf"
Sep 10 19:40:15.846339 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:15.846247    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-846240-bf54c4c4b-247nf"
Sep 10 19:40:16.120313 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:16.120275    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-846240-bf54c4c4b-247nf"]
Sep 10 19:40:16.121516 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:40:16.121491    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod901e511b_f3d5_44f1_ba9b_29f7372f4bb5.slice/crio-4d8dc36bc01f748549ffb935444808b57a8b6c323c9e285237b7c553816ff5c7 WatchSource:0}: Error finding container 4d8dc36bc01f748549ffb935444808b57a8b6c323c9e285237b7c553816ff5c7: Status 404 returned error can't find the container with id 4d8dc36bc01f748549ffb935444808b57a8b6c323c9e285237b7c553816ff5c7
Sep 10 19:40:16.812410 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:16.812377    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-846240-bf54c4c4b-247nf" event={"ID":"901e511b-f3d5-44f1-ba9b-29f7372f4bb5","Type":"ContainerStarted","Data":"9a45d0004731c2e23cc10558cdaf6db986c571975ab718ac3c177c51cab6ec1a"}
Sep 10 19:40:16.812952 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:16.812425    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-846240-bf54c4c4b-247nf" event={"ID":"901e511b-f3d5-44f1-ba9b-29f7372f4bb5","Type":"ContainerStarted","Data":"4d8dc36bc01f748549ffb935444808b57a8b6c323c9e285237b7c553816ff5c7"}
Sep 10 19:40:16.812952 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:16.812457    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-846240-bf54c4c4b-247nf"
Sep 10 19:40:16.833587 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:16.833536    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-846240-bf54c4c4b-247nf" podStartSLOduration=1.833520474 podStartE2EDuration="1.833520474s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:40:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:40:16.831298478 +0000 UTC m=+1472.432205718" watchObservedRunningTime="2026-09-10 19:40:16.833520474 +0000 UTC m=+1472.434427714"
Sep 10 19:40:22.292535 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:22.292488    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:27.820567 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:27.820534    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-846240-bf54c4c4b-247nf"
Sep 10 19:40:27.958138 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:27.958093    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:28.829854 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:28.829808    2564 patch_prober.go:28] interesting pod/e2e-api-a-846240-openshift-default-5f9c644d-kgh6j container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 10 19:40:28.830255 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:28.829879    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-846240-openshift-default-5f9c644d-kgh6j" podUID="b7777610-d77a-42c5-b6ed-60cabe303d70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 10 19:40:30.913928 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:30.913895    2564 generic.go:356] "Generic (PLEG): container finished" podID="b7777610-d77a-42c5-b6ed-60cabe303d70" containerID="5c2194f26685fc7d337fbe48a3e08cf04780c70d7e4d948d441ae74511b5dd9e" exitCode=0
Sep 10 19:40:30.914358 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:30.913969    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-846240-openshift-default-5f9c644d-kgh6j" event={"ID":"b7777610-d77a-42c5-b6ed-60cabe303d70","Type":"ContainerDied","Data":"5c2194f26685fc7d337fbe48a3e08cf04780c70d7e4d948d441ae74511b5dd9e"}
Sep 10 19:40:30.914419 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:30.914391    2564 scope.go:117] "RemoveContainer" containerID="5c2194f26685fc7d337fbe48a3e08cf04780c70d7e4d948d441ae74511b5dd9e"
Sep 10 19:40:30.915566 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:30.915534    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:40:30.915688 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:30.915617    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:40:30.915760 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:30.915700    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:40:31.921230 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:31.921190    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-846240-openshift-default-5f9c644d-kgh6j" event={"ID":"b7777610-d77a-42c5-b6ed-60cabe303d70","Type":"ContainerStarted","Data":"184d41ac33942f961d74285760a037a7fb689d5fd28e1ba0a98870a170d4f18a"}
Sep 10 19:40:32.583442 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:32.583378    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-846240-openshift-default-5f9c644d-kgh6j"
Sep 10 19:40:32.583795 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:32.583766    2564 patch_prober.go:28] interesting pod/e2e-api-a-846240-openshift-default-5f9c644d-kgh6j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Sep 10 19:40:32.583903 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:32.583835    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-846240-openshift-default-5f9c644d-kgh6j" podUID="b7777610-d77a-42c5-b6ed-60cabe303d70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Sep 10 19:40:33.583106 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:33.583058    2564 patch_prober.go:28] interesting pod/e2e-api-a-846240-openshift-default-5f9c644d-kgh6j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Sep 10 19:40:33.583655 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:33.583130    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-846240-openshift-default-5f9c644d-kgh6j" podUID="b7777610-d77a-42c5-b6ed-60cabe303d70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Sep 10 19:40:34.583299 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:34.582820    2564 patch_prober.go:28] interesting pod/e2e-api-a-846240-openshift-default-5f9c644d-kgh6j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Sep 10 19:40:34.583299 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:34.582902    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-846240-openshift-default-5f9c644d-kgh6j" podUID="b7777610-d77a-42c5-b6ed-60cabe303d70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Sep 10 19:40:35.582991 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:35.582941    2564 patch_prober.go:28] interesting pod/e2e-api-a-846240-openshift-default-5f9c644d-kgh6j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Sep 10 19:40:35.583209 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:35.583026    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-846240-openshift-default-5f9c644d-kgh6j" podUID="b7777610-d77a-42c5-b6ed-60cabe303d70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Sep 10 19:40:35.842780 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:35.842685    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:36.583564 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:36.583523    2564 patch_prober.go:28] interesting pod/e2e-api-a-846240-openshift-default-5f9c644d-kgh6j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Sep 10 19:40:36.583890 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:36.583859    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-846240-openshift-default-5f9c644d-kgh6j" podUID="b7777610-d77a-42c5-b6ed-60cabe303d70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Sep 10 19:40:37.586978 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:37.586942    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-846240-openshift-default-5f9c644d-kgh6j"
Sep 10 19:40:37.587445 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:37.587332    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-846240-openshift-default-5f9c644d-kgh6j"
Sep 10 19:40:37.588057 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:37.588040    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-846240-openshift-default-5f9c644d-kgh6j"
Sep 10 19:40:38.781690 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:38.781650    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:39.252262 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:39.252231    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:39.545503 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:39.545418    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h"]
Sep 10 19:40:39.549249 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:39.549224    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h"
Sep 10 19:40:39.551866 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:39.551839    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-ed39fa-openshift-default-dockercfg-z8pwl\""
Sep 10 19:40:39.560092 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:39.559732    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h"]
Sep 10 19:40:39.573321 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:39.573282    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:39.575387 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:39.575361    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/650b189f-bc2e-41a8-bc9d-d4b0cee41294-istiod-ca-cert\") pod \"e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h\" (UID: \"650b189f-bc2e-41a8-bc9d-d4b0cee41294\") " pod="openshift-ingress/e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h"
Sep 10 19:40:39.575524 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:39.575419    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/650b189f-bc2e-41a8-bc9d-d4b0cee41294-credential-socket\") pod \"e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h\" (UID: \"650b189f-bc2e-41a8-bc9d-d4b0cee41294\") " pod="openshift-ingress/e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h"
Sep 10 19:40:39.575524 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:39.575444    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/650b189f-bc2e-41a8-bc9d-d4b0cee41294-istio-podinfo\") pod \"e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h\" (UID: \"650b189f-bc2e-41a8-bc9d-d4b0cee41294\") " pod="openshift-ingress/e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h"
Sep 10 19:40:39.575524 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:39.575463    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8j8dx\" (UniqueName: \"kubernetes.io/projected/650b189f-bc2e-41a8-bc9d-d4b0cee41294-kube-api-access-8j8dx\") pod \"e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h\" (UID: \"650b189f-bc2e-41a8-bc9d-d4b0cee41294\") " pod="openshift-ingress/e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h"
Sep 10 19:40:39.575524 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:39.575486    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/650b189f-bc2e-41a8-bc9d-d4b0cee41294-workload-certs\") pod \"e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h\" (UID: \"650b189f-bc2e-41a8-bc9d-d4b0cee41294\") " pod="openshift-ingress/e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h"
Sep 10 19:40:39.575769 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:39.575538    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/650b189f-bc2e-41a8-bc9d-d4b0cee41294-workload-socket\") pod \"e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h\" (UID: \"650b189f-bc2e-41a8-bc9d-d4b0cee41294\") " pod="openshift-ingress/e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h"
Sep 10 19:40:39.575769 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:39.575599    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/650b189f-bc2e-41a8-bc9d-d4b0cee41294-istio-data\") pod \"e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h\" (UID: \"650b189f-bc2e-41a8-bc9d-d4b0cee41294\") " pod="openshift-ingress/e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h"
Sep 10 19:40:39.575769 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:39.575695    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/650b189f-bc2e-41a8-bc9d-d4b0cee41294-istio-envoy\") pod \"e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h\" (UID: \"650b189f-bc2e-41a8-bc9d-d4b0cee41294\") " pod="openshift-ingress/e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h"
Sep 10 19:40:39.575769 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:39.575720    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/650b189f-bc2e-41a8-bc9d-d4b0cee41294-istio-token\") pod \"e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h\" (UID: \"650b189f-bc2e-41a8-bc9d-d4b0cee41294\") " pod="openshift-ingress/e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h"
Sep 10 19:40:39.677469 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:39.677420    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/650b189f-bc2e-41a8-bc9d-d4b0cee41294-istiod-ca-cert\") pod \"e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h\" (UID: \"650b189f-bc2e-41a8-bc9d-d4b0cee41294\") " pod="openshift-ingress/e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h"
Sep 10 19:40:39.677696 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:39.677517    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/650b189f-bc2e-41a8-bc9d-d4b0cee41294-credential-socket\") pod \"e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h\" (UID: \"650b189f-bc2e-41a8-bc9d-d4b0cee41294\") " pod="openshift-ingress/e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h"
Sep 10 19:40:39.677696 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:39.677544    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/650b189f-bc2e-41a8-bc9d-d4b0cee41294-istio-podinfo\") pod \"e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h\" (UID: \"650b189f-bc2e-41a8-bc9d-d4b0cee41294\") " pod="openshift-ingress/e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h"
Sep 10 19:40:39.677696 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:39.677573    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8j8dx\" (UniqueName: \"kubernetes.io/projected/650b189f-bc2e-41a8-bc9d-d4b0cee41294-kube-api-access-8j8dx\") pod \"e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h\" (UID: \"650b189f-bc2e-41a8-bc9d-d4b0cee41294\") " pod="openshift-ingress/e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h"
Sep 10 19:40:39.677696 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:39.677612    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/650b189f-bc2e-41a8-bc9d-d4b0cee41294-workload-certs\") pod \"e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h\" (UID: \"650b189f-bc2e-41a8-bc9d-d4b0cee41294\") " pod="openshift-ingress/e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h"
Sep 10 19:40:39.677696 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:39.677665    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/650b189f-bc2e-41a8-bc9d-d4b0cee41294-workload-socket\") pod \"e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h\" (UID: \"650b189f-bc2e-41a8-bc9d-d4b0cee41294\") " pod="openshift-ingress/e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h"
Sep 10 19:40:39.677999 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:39.677868    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/650b189f-bc2e-41a8-bc9d-d4b0cee41294-istio-data\") pod \"e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h\" (UID: \"650b189f-bc2e-41a8-bc9d-d4b0cee41294\") " pod="openshift-ingress/e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h"
Sep 10 19:40:39.678063 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:39.678015    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/650b189f-bc2e-41a8-bc9d-d4b0cee41294-istio-envoy\") pod \"e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h\" (UID: \"650b189f-bc2e-41a8-bc9d-d4b0cee41294\") " pod="openshift-ingress/e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h"
Sep 10 19:40:39.678203 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:39.678101    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/650b189f-bc2e-41a8-bc9d-d4b0cee41294-istio-token\") pod \"e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h\" (UID: \"650b189f-bc2e-41a8-bc9d-d4b0cee41294\") " pod="openshift-ingress/e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h"
Sep 10 19:40:39.678396 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:39.678365    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/650b189f-bc2e-41a8-bc9d-d4b0cee41294-istiod-ca-cert\") pod \"e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h\" (UID: \"650b189f-bc2e-41a8-bc9d-d4b0cee41294\") " pod="openshift-ingress/e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h"
Sep 10 19:40:39.678726 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:39.678702    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/650b189f-bc2e-41a8-bc9d-d4b0cee41294-workload-certs\") pod \"e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h\" (UID: \"650b189f-bc2e-41a8-bc9d-d4b0cee41294\") " pod="openshift-ingress/e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h"
Sep 10 19:40:39.679033 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:39.678987    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/650b189f-bc2e-41a8-bc9d-d4b0cee41294-workload-socket\") pod \"e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h\" (UID: \"650b189f-bc2e-41a8-bc9d-d4b0cee41294\") " pod="openshift-ingress/e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h"
Sep 10 19:40:39.679198 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:39.679167    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/650b189f-bc2e-41a8-bc9d-d4b0cee41294-istio-data\") pod \"e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h\" (UID: \"650b189f-bc2e-41a8-bc9d-d4b0cee41294\") " pod="openshift-ingress/e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h"
Sep 10 19:40:39.679295 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:39.679279    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/650b189f-bc2e-41a8-bc9d-d4b0cee41294-credential-socket\") pod \"e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h\" (UID: \"650b189f-bc2e-41a8-bc9d-d4b0cee41294\") " pod="openshift-ingress/e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h"
Sep 10 19:40:39.681577 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:39.681550    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/650b189f-bc2e-41a8-bc9d-d4b0cee41294-istio-envoy\") pod \"e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h\" (UID: \"650b189f-bc2e-41a8-bc9d-d4b0cee41294\") " pod="openshift-ingress/e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h"
Sep 10 19:40:39.681823 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:39.681804    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/650b189f-bc2e-41a8-bc9d-d4b0cee41294-istio-podinfo\") pod \"e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h\" (UID: \"650b189f-bc2e-41a8-bc9d-d4b0cee41294\") " pod="openshift-ingress/e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h"
Sep 10 19:40:39.689595 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:39.689569    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/650b189f-bc2e-41a8-bc9d-d4b0cee41294-istio-token\") pod \"e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h\" (UID: \"650b189f-bc2e-41a8-bc9d-d4b0cee41294\") " pod="openshift-ingress/e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h"
Sep 10 19:40:39.689893 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:39.689871    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8j8dx\" (UniqueName: \"kubernetes.io/projected/650b189f-bc2e-41a8-bc9d-d4b0cee41294-kube-api-access-8j8dx\") pod \"e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h\" (UID: \"650b189f-bc2e-41a8-bc9d-d4b0cee41294\") " pod="openshift-ingress/e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h"
Sep 10 19:40:39.861571 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:39.861483    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h"
Sep 10 19:40:40.108870 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:40.108840    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:40.140539 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:40.140509    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h"]
Sep 10 19:40:40.142333 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:40:40.142296    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod650b189f_bc2e_41a8_bc9d_d4b0cee41294.slice/crio-51b6e6e0780c7b7ef21aa6a60e0a27757107d2fa7354d725446e434ca5ed9175 WatchSource:0}: Error finding container 51b6e6e0780c7b7ef21aa6a60e0a27757107d2fa7354d725446e434ca5ed9175: Status 404 returned error can't find the container with id 51b6e6e0780c7b7ef21aa6a60e0a27757107d2fa7354d725446e434ca5ed9175
Sep 10 19:40:40.144670 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:40.144638    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:40:40.144747 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:40.144702    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:40:40.144747 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:40.144733    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:40:40.975727 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:40.975687    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h" event={"ID":"650b189f-bc2e-41a8-bc9d-d4b0cee41294","Type":"ContainerStarted","Data":"f690ede07373401b6be14c489e97f609f7b2e980094424b267614ec72fa28d03"}
Sep 10 19:40:40.976127 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:40.975751    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h" event={"ID":"650b189f-bc2e-41a8-bc9d-d4b0cee41294","Type":"ContainerStarted","Data":"51b6e6e0780c7b7ef21aa6a60e0a27757107d2fa7354d725446e434ca5ed9175"}
Sep 10 19:40:40.998911 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:40.998759    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h" podStartSLOduration=1.998739381 podStartE2EDuration="1.998739381s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:40:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:40:40.996596368 +0000 UTC m=+1496.597503608" watchObservedRunningTime="2026-09-10 19:40:40.998739381 +0000 UTC m=+1496.599646622"
Sep 10 19:40:41.862362 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:41.862321    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h"
Sep 10 19:40:41.868489 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:41.868461    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h"
Sep 10 19:40:41.981954 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:41.981920    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h"
Sep 10 19:40:41.982945 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:41.982927    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h"
Sep 10 19:40:42.962464 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:42.962422    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:43.061361 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:43.061323    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:44.254993 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:44.254954    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:44.355393 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:44.355354    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:44.495782 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:44.495746    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:44.764086 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:44.764041    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:44.827160 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:44.827114    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:44.879352 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:44.879319    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-ed39fa-697974657d-46thd"]
Sep 10 19:40:44.883409 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:44.883388    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-ed39fa-697974657d-46thd"
Sep 10 19:40:44.885892 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:44.885868    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-ed39fa\""
Sep 10 19:40:44.892516 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:44.892493    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-ed39fa-697974657d-46thd"]
Sep 10 19:40:44.933105 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:44.933067    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xd4sv\" (UniqueName: \"kubernetes.io/projected/77a9a1f5-9f6d-4aea-8f04-52bfb7785214-kube-api-access-xd4sv\") pod \"payload-pre-processing-e2e-api-b-ed39fa-697974657d-46thd\" (UID: \"77a9a1f5-9f6d-4aea-8f04-52bfb7785214\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-ed39fa-697974657d-46thd"
Sep 10 19:40:44.933249 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:44.933155    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/77a9a1f5-9f6d-4aea-8f04-52bfb7785214-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-ed39fa-697974657d-46thd\" (UID: \"77a9a1f5-9f6d-4aea-8f04-52bfb7785214\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-ed39fa-697974657d-46thd"
Sep 10 19:40:45.034360 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:45.034258    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xd4sv\" (UniqueName: \"kubernetes.io/projected/77a9a1f5-9f6d-4aea-8f04-52bfb7785214-kube-api-access-xd4sv\") pod \"payload-pre-processing-e2e-api-b-ed39fa-697974657d-46thd\" (UID: \"77a9a1f5-9f6d-4aea-8f04-52bfb7785214\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-ed39fa-697974657d-46thd"
Sep 10 19:40:45.034549 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:45.034381    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/77a9a1f5-9f6d-4aea-8f04-52bfb7785214-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-ed39fa-697974657d-46thd\" (UID: \"77a9a1f5-9f6d-4aea-8f04-52bfb7785214\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-ed39fa-697974657d-46thd"
Sep 10 19:40:45.035303 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:45.035276    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/77a9a1f5-9f6d-4aea-8f04-52bfb7785214-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-ed39fa-697974657d-46thd\" (UID: \"77a9a1f5-9f6d-4aea-8f04-52bfb7785214\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-ed39fa-697974657d-46thd"
Sep 10 19:40:45.045887 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:45.045856    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xd4sv\" (UniqueName: \"kubernetes.io/projected/77a9a1f5-9f6d-4aea-8f04-52bfb7785214-kube-api-access-xd4sv\") pod \"payload-pre-processing-e2e-api-b-ed39fa-697974657d-46thd\" (UID: \"77a9a1f5-9f6d-4aea-8f04-52bfb7785214\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-ed39fa-697974657d-46thd"
Sep 10 19:40:45.196407 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:45.196374    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-ed39fa-697974657d-46thd"
Sep 10 19:40:45.469949 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:45.469920    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-ed39fa-697974657d-46thd"]
Sep 10 19:40:45.471817 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:40:45.471788    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77a9a1f5_9f6d_4aea_8f04_52bfb7785214.slice/crio-6f3814bb826a2dd2c911beb9a3f225be3f4b03d0ba824f8656c4b50b039c39b2 WatchSource:0}: Error finding container 6f3814bb826a2dd2c911beb9a3f225be3f4b03d0ba824f8656c4b50b039c39b2: Status 404 returned error can't find the container with id 6f3814bb826a2dd2c911beb9a3f225be3f4b03d0ba824f8656c4b50b039c39b2
Sep 10 19:40:46.021875 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:46.021842    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-ed39fa-697974657d-46thd" event={"ID":"77a9a1f5-9f6d-4aea-8f04-52bfb7785214","Type":"ContainerStarted","Data":"af554b8b683484378b29a7002cc76284f2c231768252a27617aec1e99084de65"}
Sep 10 19:40:46.022048 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:46.021899    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-ed39fa-697974657d-46thd" event={"ID":"77a9a1f5-9f6d-4aea-8f04-52bfb7785214","Type":"ContainerStarted","Data":"6f3814bb826a2dd2c911beb9a3f225be3f4b03d0ba824f8656c4b50b039c39b2"}
Sep 10 19:40:46.022048 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:46.021985    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-ed39fa-697974657d-46thd"
Sep 10 19:40:46.044080 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:46.044023    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-ed39fa-697974657d-46thd" podStartSLOduration=2.044008635 podStartE2EDuration="2.044008635s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:40:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:40:46.041309403 +0000 UTC m=+1501.642216645" watchObservedRunningTime="2026-09-10 19:40:46.044008635 +0000 UTC m=+1501.644915909"
Sep 10 19:40:50.953180 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:50.953142    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:57.030796 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:40:57.030765    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-ed39fa-697974657d-46thd"
Sep 10 19:41:04.145164 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:04.145127    2564 generic.go:356] "Generic (PLEG): container finished" podID="650b189f-bc2e-41a8-bc9d-d4b0cee41294" containerID="f690ede07373401b6be14c489e97f609f7b2e980094424b267614ec72fa28d03" exitCode=0
Sep 10 19:41:04.145591 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:04.145202    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h" event={"ID":"650b189f-bc2e-41a8-bc9d-d4b0cee41294","Type":"ContainerDied","Data":"f690ede07373401b6be14c489e97f609f7b2e980094424b267614ec72fa28d03"}
Sep 10 19:41:04.145694 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:04.145679    2564 scope.go:117] "RemoveContainer" containerID="f690ede07373401b6be14c489e97f609f7b2e980094424b267614ec72fa28d03"
Sep 10 19:41:04.146878 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:04.146848    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:41:04.146969 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:04.146917    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:41:04.146969 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:04.146949    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:41:05.154431 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:05.154396    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h" event={"ID":"650b189f-bc2e-41a8-bc9d-d4b0cee41294","Type":"ContainerStarted","Data":"d5aa7d96bbeca19ca36c5e51e982bf8e4c0d77caf52e8d383f48f598d09d3838"}
Sep 10 19:41:05.356003 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:05.355968    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:41:05.863136 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:05.863097    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h"
Sep 10 19:41:05.863353 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:05.863216    2564 patch_prober.go:28] interesting pod/e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Sep 10 19:41:05.863353 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:05.863270    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h" podUID="650b189f-bc2e-41a8-bc9d-d4b0cee41294" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Sep 10 19:41:06.862325 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:06.862274    2564 patch_prober.go:28] interesting pod/e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Sep 10 19:41:06.862857 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:06.862349    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h" podUID="650b189f-bc2e-41a8-bc9d-d4b0cee41294" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Sep 10 19:41:07.862670 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:07.862601    2564 patch_prober.go:28] interesting pod/e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Sep 10 19:41:07.863171 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:07.862697    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h" podUID="650b189f-bc2e-41a8-bc9d-d4b0cee41294" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Sep 10 19:41:07.942241 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:07.942204    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:41:08.862522 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:08.862473    2564 patch_prober.go:28] interesting pod/e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Sep 10 19:41:08.862778 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:08.862548    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h" podUID="650b189f-bc2e-41a8-bc9d-d4b0cee41294" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Sep 10 19:41:09.861999 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:09.861957    2564 patch_prober.go:28] interesting pod/e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Sep 10 19:41:09.862296 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:09.862266    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h" podUID="650b189f-bc2e-41a8-bc9d-d4b0cee41294" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Sep 10 19:41:10.866761 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:10.866730    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h"
Sep 10 19:41:10.867260 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:10.867031    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h"
Sep 10 19:41:10.867773 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:10.867750    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h"
Sep 10 19:41:10.963229 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:10.963197    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-846240-kmnrf"]
Sep 10 19:41:10.966797 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:10.966778    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-846240-kmnrf"
Sep 10 19:41:10.969436 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:10.969415    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 19:41:10.969603 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:10.969586    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tjmqn\""
Sep 10 19:41:10.975681 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:10.975650    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-846240-kmnrf"]
Sep 10 19:41:11.038565 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:11.038529    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ce66642-8a7f-4f6e-9f46-9e403257f238-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-846240-kmnrf\" (UID: \"9ce66642-8a7f-4f6e-9f46-9e403257f238\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-846240-kmnrf"
Sep 10 19:41:11.072297 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:11.072258    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:41:11.139985 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:11.139858    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ce66642-8a7f-4f6e-9f46-9e403257f238-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-846240-kmnrf\" (UID: \"9ce66642-8a7f-4f6e-9f46-9e403257f238\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-846240-kmnrf"
Sep 10 19:41:11.140913 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:11.140884    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ce66642-8a7f-4f6e-9f46-9e403257f238-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-846240-kmnrf\" (UID: \"9ce66642-8a7f-4f6e-9f46-9e403257f238\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-846240-kmnrf"
Sep 10 19:41:11.279601 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:11.279566    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-846240-kmnrf"
Sep 10 19:41:11.552684 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:11.552650    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-846240-kmnrf"]
Sep 10 19:41:11.555935 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:41:11.555907    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ce66642_8a7f_4f6e_9f46_9e403257f238.slice/crio-980db121cec2cafba4ac7626b921844a14bb624146f601957d2da336e6e23879 WatchSource:0}: Error finding container 980db121cec2cafba4ac7626b921844a14bb624146f601957d2da336e6e23879: Status 404 returned error can't find the container with id 980db121cec2cafba4ac7626b921844a14bb624146f601957d2da336e6e23879
Sep 10 19:41:12.210345 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:12.210255    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-846240-kmnrf" event={"ID":"9ce66642-8a7f-4f6e-9f46-9e403257f238","Type":"ContainerStarted","Data":"e838751250a4a3d11b7ad526631bf0a92fad7fa212ca0a4e3ab7ef4e3c7e78a3"}
Sep 10 19:41:12.210345 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:12.210307    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-846240-kmnrf" event={"ID":"9ce66642-8a7f-4f6e-9f46-9e403257f238","Type":"ContainerStarted","Data":"980db121cec2cafba4ac7626b921844a14bb624146f601957d2da336e6e23879"}
Sep 10 19:41:12.229708 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:12.229654    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-846240-kmnrf" podStartSLOduration=2.229636974 podStartE2EDuration="2.229636974s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:41:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:41:12.2284074 +0000 UTC m=+1527.829314640" watchObservedRunningTime="2026-09-10 19:41:12.229636974 +0000 UTC m=+1527.830544208"
Sep 10 19:41:13.217111 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:13.217073    2564 generic.go:356] "Generic (PLEG): container finished" podID="9ce66642-8a7f-4f6e-9f46-9e403257f238" containerID="e838751250a4a3d11b7ad526631bf0a92fad7fa212ca0a4e3ab7ef4e3c7e78a3" exitCode=0
Sep 10 19:41:13.217564 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:13.217146    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-846240-kmnrf" event={"ID":"9ce66642-8a7f-4f6e-9f46-9e403257f238","Type":"ContainerDied","Data":"e838751250a4a3d11b7ad526631bf0a92fad7fa212ca0a4e3ab7ef4e3c7e78a3"}
Sep 10 19:41:14.225031 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:14.225000    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:41:14.377514 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:14.377490    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-846240-kmnrf"
Sep 10 19:41:14.472889 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:14.472853    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ce66642-8a7f-4f6e-9f46-9e403257f238-maas-api-service-ca\") pod \"9ce66642-8a7f-4f6e-9f46-9e403257f238\" (UID: \"9ce66642-8a7f-4f6e-9f46-9e403257f238\") "
Sep 10 19:41:14.473266 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:14.473206    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ce66642-8a7f-4f6e-9f46-9e403257f238-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9ce66642-8a7f-4f6e-9f46-9e403257f238" (UID: "9ce66642-8a7f-4f6e-9f46-9e403257f238"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:41:14.574088 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:14.574050    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ce66642-8a7f-4f6e-9f46-9e403257f238-maas-api-service-ca\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:41:15.230180 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:15.230091    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-846240-kmnrf"
Sep 10 19:41:15.230180 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:15.230130    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-846240-kmnrf" event={"ID":"9ce66642-8a7f-4f6e-9f46-9e403257f238","Type":"ContainerDied","Data":"980db121cec2cafba4ac7626b921844a14bb624146f601957d2da336e6e23879"}
Sep 10 19:41:15.230583 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:15.230183    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="980db121cec2cafba4ac7626b921844a14bb624146f601957d2da336e6e23879"
Sep 10 19:41:15.298656 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:15.298608    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:41:18.946059 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:18.945979    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:41:26.547658 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:26.547602    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:41:26.587560 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:26.587518    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-846240-bf54c4c4b-247nf"]
Sep 10 19:41:26.587826 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:26.587796    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-846240-bf54c4c4b-247nf" podUID="901e511b-f3d5-44f1-ba9b-29f7372f4bb5" containerName="payload-pre-processing" containerID="cri-o://9a45d0004731c2e23cc10558cdaf6db986c571975ab718ac3c177c51cab6ec1a" gracePeriod=30
Sep 10 19:41:26.648281 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:26.648221    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:41:26.747407 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:26.747370    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:41:26.772401 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:26.772327    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-846240-kmnrf"]
Sep 10 19:41:26.776019 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:26.775988    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-846240-kmnrf"]
Sep 10 19:41:26.800502 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:26.800409    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:41:26.881195 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:26.879742    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-846240-bf54c4c4b-247nf"
Sep 10 19:41:26.951611 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:26.951577    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ce66642-8a7f-4f6e-9f46-9e403257f238" path="/var/lib/kubelet/pods/9ce66642-8a7f-4f6e-9f46-9e403257f238/volumes"
Sep 10 19:41:27.012802 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:27.012768    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jw4js\" (UniqueName: \"kubernetes.io/projected/901e511b-f3d5-44f1-ba9b-29f7372f4bb5-kube-api-access-jw4js\") pod \"901e511b-f3d5-44f1-ba9b-29f7372f4bb5\" (UID: \"901e511b-f3d5-44f1-ba9b-29f7372f4bb5\") "
Sep 10 19:41:27.012987 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:27.012871    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/901e511b-f3d5-44f1-ba9b-29f7372f4bb5-plugins-config-volume\") pod \"901e511b-f3d5-44f1-ba9b-29f7372f4bb5\" (UID: \"901e511b-f3d5-44f1-ba9b-29f7372f4bb5\") "
Sep 10 19:41:27.013175 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:27.013154    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/901e511b-f3d5-44f1-ba9b-29f7372f4bb5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "901e511b-f3d5-44f1-ba9b-29f7372f4bb5" (UID: "901e511b-f3d5-44f1-ba9b-29f7372f4bb5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:41:27.014945 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:27.014917    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/901e511b-f3d5-44f1-ba9b-29f7372f4bb5-kube-api-access-jw4js" (OuterVolumeSpecName: "kube-api-access-jw4js") pod "901e511b-f3d5-44f1-ba9b-29f7372f4bb5" (UID: "901e511b-f3d5-44f1-ba9b-29f7372f4bb5"). InnerVolumeSpecName "kube-api-access-jw4js". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:41:27.114258 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:27.114178    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jw4js\" (UniqueName: \"kubernetes.io/projected/901e511b-f3d5-44f1-ba9b-29f7372f4bb5-kube-api-access-jw4js\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:41:27.114258 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:27.114206    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/901e511b-f3d5-44f1-ba9b-29f7372f4bb5-plugins-config-volume\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:41:27.162011 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:27.161959    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:41:27.306136 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:27.306097    2564 generic.go:356] "Generic (PLEG): container finished" podID="901e511b-f3d5-44f1-ba9b-29f7372f4bb5" containerID="9a45d0004731c2e23cc10558cdaf6db986c571975ab718ac3c177c51cab6ec1a" exitCode=0
Sep 10 19:41:27.306309 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:27.306143    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-846240-bf54c4c4b-247nf"
Sep 10 19:41:27.306309 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:27.306156    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-846240-bf54c4c4b-247nf" event={"ID":"901e511b-f3d5-44f1-ba9b-29f7372f4bb5","Type":"ContainerDied","Data":"9a45d0004731c2e23cc10558cdaf6db986c571975ab718ac3c177c51cab6ec1a"}
Sep 10 19:41:27.306309 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:27.306203    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-846240-bf54c4c4b-247nf" event={"ID":"901e511b-f3d5-44f1-ba9b-29f7372f4bb5","Type":"ContainerDied","Data":"4d8dc36bc01f748549ffb935444808b57a8b6c323c9e285237b7c553816ff5c7"}
Sep 10 19:41:27.306309 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:27.306229    2564 scope.go:117] "RemoveContainer" containerID="9a45d0004731c2e23cc10558cdaf6db986c571975ab718ac3c177c51cab6ec1a"
Sep 10 19:41:27.318615 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:27.318592    2564 scope.go:117] "RemoveContainer" containerID="9a45d0004731c2e23cc10558cdaf6db986c571975ab718ac3c177c51cab6ec1a"
Sep 10 19:41:27.318948 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:41:27.318926    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a45d0004731c2e23cc10558cdaf6db986c571975ab718ac3c177c51cab6ec1a\": container with ID starting with 9a45d0004731c2e23cc10558cdaf6db986c571975ab718ac3c177c51cab6ec1a not found: ID does not exist" containerID="9a45d0004731c2e23cc10558cdaf6db986c571975ab718ac3c177c51cab6ec1a"
Sep 10 19:41:27.318998 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:27.318956    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a45d0004731c2e23cc10558cdaf6db986c571975ab718ac3c177c51cab6ec1a"} err="failed to get container status \"9a45d0004731c2e23cc10558cdaf6db986c571975ab718ac3c177c51cab6ec1a\": rpc error: code = NotFound desc = could not find container \"9a45d0004731c2e23cc10558cdaf6db986c571975ab718ac3c177c51cab6ec1a\": container with ID starting with 9a45d0004731c2e23cc10558cdaf6db986c571975ab718ac3c177c51cab6ec1a not found: ID does not exist"
Sep 10 19:41:27.331967 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:27.331937    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-846240-bf54c4c4b-247nf"]
Sep 10 19:41:27.336316 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:27.336292    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-846240-bf54c4c4b-247nf"]
Sep 10 19:41:28.950210 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:28.950180    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="901e511b-f3d5-44f1-ba9b-29f7372f4bb5" path="/var/lib/kubelet/pods/901e511b-f3d5-44f1-ba9b-29f7372f4bb5/volumes"
Sep 10 19:41:29.265591 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:29.265554    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:41:33.315006 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:33.314974    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:41:33.585807 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:33.585705    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-846240-openshift-default-5f9c644d-kgh6j"]
Sep 10 19:41:33.585974 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:33.585952    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-846240-openshift-default-5f9c644d-kgh6j" podUID="b7777610-d77a-42c5-b6ed-60cabe303d70" containerName="istio-proxy" containerID="cri-o://184d41ac33942f961d74285760a037a7fb689d5fd28e1ba0a98870a170d4f18a" gracePeriod=30
Sep 10 19:41:33.963452 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:33.963358    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-ed39fa-fgm2t"]
Sep 10 19:41:33.964086 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:33.964065    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ce66642-8a7f-4f6e-9f46-9e403257f238" containerName="revoke-keys"
Sep 10 19:41:33.964086 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:33.964086    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ce66642-8a7f-4f6e-9f46-9e403257f238" containerName="revoke-keys"
Sep 10 19:41:33.964258 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:33.964111    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="901e511b-f3d5-44f1-ba9b-29f7372f4bb5" containerName="payload-pre-processing"
Sep 10 19:41:33.964258 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:33.964123    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="901e511b-f3d5-44f1-ba9b-29f7372f4bb5" containerName="payload-pre-processing"
Sep 10 19:41:33.964258 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:33.964242    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="901e511b-f3d5-44f1-ba9b-29f7372f4bb5" containerName="payload-pre-processing"
Sep 10 19:41:33.964258 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:33.964252    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ce66642-8a7f-4f6e-9f46-9e403257f238" containerName="revoke-keys"
Sep 10 19:41:33.968833 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:33.968812    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-ed39fa-fgm2t"
Sep 10 19:41:33.971276 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:33.971252    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 19:41:33.971841 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:33.971676    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tjmqn\""
Sep 10 19:41:33.974613 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:33.974591    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-ed39fa-fgm2t"]
Sep 10 19:41:33.992935 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:33.992906    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/15711f0a-cd76-4f89-87b8-b1ad4a47c1ce-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-ed39fa-fgm2t\" (UID: \"15711f0a-cd76-4f89-87b8-b1ad4a47c1ce\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-ed39fa-fgm2t"
Sep 10 19:41:34.094609 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:34.094569    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/15711f0a-cd76-4f89-87b8-b1ad4a47c1ce-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-ed39fa-fgm2t\" (UID: \"15711f0a-cd76-4f89-87b8-b1ad4a47c1ce\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-ed39fa-fgm2t"
Sep 10 19:41:34.095371 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:34.095348    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/15711f0a-cd76-4f89-87b8-b1ad4a47c1ce-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-ed39fa-fgm2t\" (UID: \"15711f0a-cd76-4f89-87b8-b1ad4a47c1ce\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-ed39fa-fgm2t"
Sep 10 19:41:34.281259 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:34.281226    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-ed39fa-fgm2t"
Sep 10 19:41:34.553273 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:34.553248    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-ed39fa-fgm2t"]
Sep 10 19:41:34.555154 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:41:34.555124    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15711f0a_cd76_4f89_87b8_b1ad4a47c1ce.slice/crio-e92521fbd1447b6d43a76f5d55f50c628bb38f59a32475e5bd9d6018a4d137af WatchSource:0}: Error finding container e92521fbd1447b6d43a76f5d55f50c628bb38f59a32475e5bd9d6018a4d137af: Status 404 returned error can't find the container with id e92521fbd1447b6d43a76f5d55f50c628bb38f59a32475e5bd9d6018a4d137af
Sep 10 19:41:34.853521 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:34.853408    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:41:35.358410 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:35.358363    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-ed39fa-fgm2t" event={"ID":"15711f0a-cd76-4f89-87b8-b1ad4a47c1ce","Type":"ContainerStarted","Data":"83f0295658e020999a0e12059df69d4883c5256620f2fffaf050e364f8a7233a"}
Sep 10 19:41:35.358569 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:35.358432    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-ed39fa-fgm2t" event={"ID":"15711f0a-cd76-4f89-87b8-b1ad4a47c1ce","Type":"ContainerStarted","Data":"e92521fbd1447b6d43a76f5d55f50c628bb38f59a32475e5bd9d6018a4d137af"}
Sep 10 19:41:35.376523 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:35.376472    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-ed39fa-fgm2t" podStartSLOduration=2.376454779 podStartE2EDuration="2.376454779s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:41:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:41:35.375036474 +0000 UTC m=+1550.975943714" watchObservedRunningTime="2026-09-10 19:41:35.376454779 +0000 UTC m=+1550.977362037"
Sep 10 19:41:36.368078 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:36.368031    2564 generic.go:356] "Generic (PLEG): container finished" podID="15711f0a-cd76-4f89-87b8-b1ad4a47c1ce" containerID="83f0295658e020999a0e12059df69d4883c5256620f2fffaf050e364f8a7233a" exitCode=0
Sep 10 19:41:36.368535 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:36.368120    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-ed39fa-fgm2t" event={"ID":"15711f0a-cd76-4f89-87b8-b1ad4a47c1ce","Type":"ContainerDied","Data":"83f0295658e020999a0e12059df69d4883c5256620f2fffaf050e364f8a7233a"}
Sep 10 19:41:37.518566 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:37.518540    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-ed39fa-fgm2t"
Sep 10 19:41:37.535456 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:37.535415    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/15711f0a-cd76-4f89-87b8-b1ad4a47c1ce-maas-api-service-ca\") pod \"15711f0a-cd76-4f89-87b8-b1ad4a47c1ce\" (UID: \"15711f0a-cd76-4f89-87b8-b1ad4a47c1ce\") "
Sep 10 19:41:37.535851 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:37.535822    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15711f0a-cd76-4f89-87b8-b1ad4a47c1ce-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "15711f0a-cd76-4f89-87b8-b1ad4a47c1ce" (UID: "15711f0a-cd76-4f89-87b8-b1ad4a47c1ce"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:41:37.536004 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:37.535988    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/15711f0a-cd76-4f89-87b8-b1ad4a47c1ce-maas-api-service-ca\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:41:37.590412 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:37.590374    2564 patch_prober.go:28] interesting pod/e2e-api-a-846240-openshift-default-5f9c644d-kgh6j container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 19:41:37.590572 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:37.590429    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-846240-openshift-default-5f9c644d-kgh6j" podUID="b7777610-d77a-42c5-b6ed-60cabe303d70" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 19:41:38.381357 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:38.381329    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-ed39fa-fgm2t"
Sep 10 19:41:38.381525 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:38.381328    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-ed39fa-fgm2t" event={"ID":"15711f0a-cd76-4f89-87b8-b1ad4a47c1ce","Type":"ContainerDied","Data":"e92521fbd1447b6d43a76f5d55f50c628bb38f59a32475e5bd9d6018a4d137af"}
Sep 10 19:41:38.381525 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:38.381448    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e92521fbd1447b6d43a76f5d55f50c628bb38f59a32475e5bd9d6018a4d137af"
Sep 10 19:41:38.654328 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:38.654233    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:41:38.873045 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:38.873019    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-846240-openshift-default-5f9c644d-kgh6j"
Sep 10 19:41:38.951439 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:38.951362    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7777610-d77a-42c5-b6ed-60cabe303d70-credential-socket\") pod \"b7777610-d77a-42c5-b6ed-60cabe303d70\" (UID: \"b7777610-d77a-42c5-b6ed-60cabe303d70\") "
Sep 10 19:41:38.951439 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:38.951420    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7777610-d77a-42c5-b6ed-60cabe303d70-workload-socket\") pod \"b7777610-d77a-42c5-b6ed-60cabe303d70\" (UID: \"b7777610-d77a-42c5-b6ed-60cabe303d70\") "
Sep 10 19:41:38.951697 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:38.951450    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7777610-d77a-42c5-b6ed-60cabe303d70-istio-token\") pod \"b7777610-d77a-42c5-b6ed-60cabe303d70\" (UID: \"b7777610-d77a-42c5-b6ed-60cabe303d70\") "
Sep 10 19:41:38.951697 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:38.951480    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ldthl\" (UniqueName: \"kubernetes.io/projected/b7777610-d77a-42c5-b6ed-60cabe303d70-kube-api-access-ldthl\") pod \"b7777610-d77a-42c5-b6ed-60cabe303d70\" (UID: \"b7777610-d77a-42c5-b6ed-60cabe303d70\") "
Sep 10 19:41:38.951697 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:38.951514    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7777610-d77a-42c5-b6ed-60cabe303d70-istiod-ca-cert\") pod \"b7777610-d77a-42c5-b6ed-60cabe303d70\" (UID: \"b7777610-d77a-42c5-b6ed-60cabe303d70\") "
Sep 10 19:41:38.951697 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:38.951650    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7777610-d77a-42c5-b6ed-60cabe303d70-istio-envoy\") pod \"b7777610-d77a-42c5-b6ed-60cabe303d70\" (UID: \"b7777610-d77a-42c5-b6ed-60cabe303d70\") "
Sep 10 19:41:38.951697 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:38.951668    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7777610-d77a-42c5-b6ed-60cabe303d70-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b7777610-d77a-42c5-b6ed-60cabe303d70" (UID: "b7777610-d77a-42c5-b6ed-60cabe303d70"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:41:38.951955 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:38.951748    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7777610-d77a-42c5-b6ed-60cabe303d70-workload-certs\") pod \"b7777610-d77a-42c5-b6ed-60cabe303d70\" (UID: \"b7777610-d77a-42c5-b6ed-60cabe303d70\") "
Sep 10 19:41:38.951955 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:38.951801    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7777610-d77a-42c5-b6ed-60cabe303d70-istio-podinfo\") pod \"b7777610-d77a-42c5-b6ed-60cabe303d70\" (UID: \"b7777610-d77a-42c5-b6ed-60cabe303d70\") "
Sep 10 19:41:38.951955 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:38.951824    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7777610-d77a-42c5-b6ed-60cabe303d70-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b7777610-d77a-42c5-b6ed-60cabe303d70" (UID: "b7777610-d77a-42c5-b6ed-60cabe303d70"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:41:38.951955 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:38.951889    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7777610-d77a-42c5-b6ed-60cabe303d70-istio-data\") pod \"b7777610-d77a-42c5-b6ed-60cabe303d70\" (UID: \"b7777610-d77a-42c5-b6ed-60cabe303d70\") "
Sep 10 19:41:38.951955 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:38.951930    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7777610-d77a-42c5-b6ed-60cabe303d70-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b7777610-d77a-42c5-b6ed-60cabe303d70" (UID: "b7777610-d77a-42c5-b6ed-60cabe303d70"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:41:38.952173 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:38.952046    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7777610-d77a-42c5-b6ed-60cabe303d70-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b7777610-d77a-42c5-b6ed-60cabe303d70" (UID: "b7777610-d77a-42c5-b6ed-60cabe303d70"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:41:38.952212 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:38.952184    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7777610-d77a-42c5-b6ed-60cabe303d70-istio-data" (OuterVolumeSpecName: "istio-data") pod "b7777610-d77a-42c5-b6ed-60cabe303d70" (UID: "b7777610-d77a-42c5-b6ed-60cabe303d70"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:41:38.952356 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:38.952333    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7777610-d77a-42c5-b6ed-60cabe303d70-workload-certs\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:41:38.952495 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:38.952392    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7777610-d77a-42c5-b6ed-60cabe303d70-istio-data\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:41:38.952495 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:38.952406    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7777610-d77a-42c5-b6ed-60cabe303d70-credential-socket\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:41:38.952495 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:38.952415    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7777610-d77a-42c5-b6ed-60cabe303d70-workload-socket\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:41:38.952495 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:38.952423    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7777610-d77a-42c5-b6ed-60cabe303d70-istiod-ca-cert\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:41:38.953888 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:38.953868    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7777610-d77a-42c5-b6ed-60cabe303d70-kube-api-access-ldthl" (OuterVolumeSpecName: "kube-api-access-ldthl") pod "b7777610-d77a-42c5-b6ed-60cabe303d70" (UID: "b7777610-d77a-42c5-b6ed-60cabe303d70"). InnerVolumeSpecName "kube-api-access-ldthl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:41:38.953980 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:38.953901    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7777610-d77a-42c5-b6ed-60cabe303d70-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b7777610-d77a-42c5-b6ed-60cabe303d70" (UID: "b7777610-d77a-42c5-b6ed-60cabe303d70"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:41:38.954032 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:38.954016    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7777610-d77a-42c5-b6ed-60cabe303d70-istio-token" (OuterVolumeSpecName: "istio-token") pod "b7777610-d77a-42c5-b6ed-60cabe303d70" (UID: "b7777610-d77a-42c5-b6ed-60cabe303d70"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:41:38.954089 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:38.954069    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b7777610-d77a-42c5-b6ed-60cabe303d70-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b7777610-d77a-42c5-b6ed-60cabe303d70" (UID: "b7777610-d77a-42c5-b6ed-60cabe303d70"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 19:41:39.053046 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:39.053003    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7777610-d77a-42c5-b6ed-60cabe303d70-istio-podinfo\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:41:39.053046 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:39.053039    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7777610-d77a-42c5-b6ed-60cabe303d70-istio-token\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:41:39.053046 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:39.053053    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ldthl\" (UniqueName: \"kubernetes.io/projected/b7777610-d77a-42c5-b6ed-60cabe303d70-kube-api-access-ldthl\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:41:39.053289 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:39.053066    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7777610-d77a-42c5-b6ed-60cabe303d70-istio-envoy\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:41:39.388576 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:39.388538    2564 generic.go:356] "Generic (PLEG): container finished" podID="b7777610-d77a-42c5-b6ed-60cabe303d70" containerID="184d41ac33942f961d74285760a037a7fb689d5fd28e1ba0a98870a170d4f18a" exitCode=0
Sep 10 19:41:39.388869 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:39.388601    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-846240-openshift-default-5f9c644d-kgh6j"
Sep 10 19:41:39.388869 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:39.388635    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-846240-openshift-default-5f9c644d-kgh6j" event={"ID":"b7777610-d77a-42c5-b6ed-60cabe303d70","Type":"ContainerDied","Data":"184d41ac33942f961d74285760a037a7fb689d5fd28e1ba0a98870a170d4f18a"}
Sep 10 19:41:39.388869 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:39.388720    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-846240-openshift-default-5f9c644d-kgh6j" event={"ID":"b7777610-d77a-42c5-b6ed-60cabe303d70","Type":"ContainerDied","Data":"9b7c928863abd9f77523e2650d53ae4c89a056fa630b48d706b35250ef55fa2e"}
Sep 10 19:41:39.388869 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:39.388748    2564 scope.go:117] "RemoveContainer" containerID="184d41ac33942f961d74285760a037a7fb689d5fd28e1ba0a98870a170d4f18a"
Sep 10 19:41:39.401450 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:39.400789    2564 scope.go:117] "RemoveContainer" containerID="5c2194f26685fc7d337fbe48a3e08cf04780c70d7e4d948d441ae74511b5dd9e"
Sep 10 19:41:39.410313 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:39.410292    2564 scope.go:117] "RemoveContainer" containerID="184d41ac33942f961d74285760a037a7fb689d5fd28e1ba0a98870a170d4f18a"
Sep 10 19:41:39.410594 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:41:39.410576    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"184d41ac33942f961d74285760a037a7fb689d5fd28e1ba0a98870a170d4f18a\": container with ID starting with 184d41ac33942f961d74285760a037a7fb689d5fd28e1ba0a98870a170d4f18a not found: ID does not exist" containerID="184d41ac33942f961d74285760a037a7fb689d5fd28e1ba0a98870a170d4f18a"
Sep 10 19:41:39.410672 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:39.410603    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"184d41ac33942f961d74285760a037a7fb689d5fd28e1ba0a98870a170d4f18a"} err="failed to get container status \"184d41ac33942f961d74285760a037a7fb689d5fd28e1ba0a98870a170d4f18a\": rpc error: code = NotFound desc = could not find container \"184d41ac33942f961d74285760a037a7fb689d5fd28e1ba0a98870a170d4f18a\": container with ID starting with 184d41ac33942f961d74285760a037a7fb689d5fd28e1ba0a98870a170d4f18a not found: ID does not exist"
Sep 10 19:41:39.410672 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:39.410633    2564 scope.go:117] "RemoveContainer" containerID="5c2194f26685fc7d337fbe48a3e08cf04780c70d7e4d948d441ae74511b5dd9e"
Sep 10 19:41:39.411026 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:41:39.410995    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c2194f26685fc7d337fbe48a3e08cf04780c70d7e4d948d441ae74511b5dd9e\": container with ID starting with 5c2194f26685fc7d337fbe48a3e08cf04780c70d7e4d948d441ae74511b5dd9e not found: ID does not exist" containerID="5c2194f26685fc7d337fbe48a3e08cf04780c70d7e4d948d441ae74511b5dd9e"
Sep 10 19:41:39.411147 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:39.411032    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c2194f26685fc7d337fbe48a3e08cf04780c70d7e4d948d441ae74511b5dd9e"} err="failed to get container status \"5c2194f26685fc7d337fbe48a3e08cf04780c70d7e4d948d441ae74511b5dd9e\": rpc error: code = NotFound desc = could not find container \"5c2194f26685fc7d337fbe48a3e08cf04780c70d7e4d948d441ae74511b5dd9e\": container with ID starting with 5c2194f26685fc7d337fbe48a3e08cf04780c70d7e4d948d441ae74511b5dd9e not found: ID does not exist"
Sep 10 19:41:39.413949 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:39.413928    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-846240-openshift-default-5f9c644d-kgh6j"]
Sep 10 19:41:39.421355 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:39.421330    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-846240-openshift-default-5f9c644d-kgh6j"]
Sep 10 19:41:40.949519 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:40.949485    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7777610-d77a-42c5-b6ed-60cabe303d70" path="/var/lib/kubelet/pods/b7777610-d77a-42c5-b6ed-60cabe303d70/volumes"
Sep 10 19:41:49.721321 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:49.721283    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:41:49.758761 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:49.758728    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-ed39fa-697974657d-46thd"]
Sep 10 19:41:49.759010 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:49.758985    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-ed39fa-697974657d-46thd" podUID="77a9a1f5-9f6d-4aea-8f04-52bfb7785214" containerName="payload-pre-processing" containerID="cri-o://af554b8b683484378b29a7002cc76284f2c231768252a27617aec1e99084de65" gracePeriod=30
Sep 10 19:41:49.784240 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:49.784196    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:41:49.838686 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:49.838647    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:41:49.940032 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:49.939989    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:41:49.940204 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:49.940125    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-ed39fa-fgm2t"]
Sep 10 19:41:49.942462 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:49.942436    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-ed39fa-fgm2t"]
Sep 10 19:41:50.025360 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:50.025322    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:41:50.030509 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:50.030483    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-ed39fa-697974657d-46thd"
Sep 10 19:41:50.175469 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:50.175430    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/77a9a1f5-9f6d-4aea-8f04-52bfb7785214-plugins-config-volume\") pod \"77a9a1f5-9f6d-4aea-8f04-52bfb7785214\" (UID: \"77a9a1f5-9f6d-4aea-8f04-52bfb7785214\") "
Sep 10 19:41:50.175675 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:50.175582    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xd4sv\" (UniqueName: \"kubernetes.io/projected/77a9a1f5-9f6d-4aea-8f04-52bfb7785214-kube-api-access-xd4sv\") pod \"77a9a1f5-9f6d-4aea-8f04-52bfb7785214\" (UID: \"77a9a1f5-9f6d-4aea-8f04-52bfb7785214\") "
Sep 10 19:41:50.175882 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:50.175853    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77a9a1f5-9f6d-4aea-8f04-52bfb7785214-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "77a9a1f5-9f6d-4aea-8f04-52bfb7785214" (UID: "77a9a1f5-9f6d-4aea-8f04-52bfb7785214"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:41:50.175995 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:50.175977    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/77a9a1f5-9f6d-4aea-8f04-52bfb7785214-plugins-config-volume\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:41:50.177707 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:50.177682    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77a9a1f5-9f6d-4aea-8f04-52bfb7785214-kube-api-access-xd4sv" (OuterVolumeSpecName: "kube-api-access-xd4sv") pod "77a9a1f5-9f6d-4aea-8f04-52bfb7785214" (UID: "77a9a1f5-9f6d-4aea-8f04-52bfb7785214"). InnerVolumeSpecName "kube-api-access-xd4sv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:41:50.277453 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:50.277363    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xd4sv\" (UniqueName: \"kubernetes.io/projected/77a9a1f5-9f6d-4aea-8f04-52bfb7785214-kube-api-access-xd4sv\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:41:50.319922 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:50.319881    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:41:50.454633 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:50.454586    2564 generic.go:356] "Generic (PLEG): container finished" podID="77a9a1f5-9f6d-4aea-8f04-52bfb7785214" containerID="af554b8b683484378b29a7002cc76284f2c231768252a27617aec1e99084de65" exitCode=0
Sep 10 19:41:50.454834 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:50.454652    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-ed39fa-697974657d-46thd"
Sep 10 19:41:50.454834 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:50.454650    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-ed39fa-697974657d-46thd" event={"ID":"77a9a1f5-9f6d-4aea-8f04-52bfb7785214","Type":"ContainerDied","Data":"af554b8b683484378b29a7002cc76284f2c231768252a27617aec1e99084de65"}
Sep 10 19:41:50.454834 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:50.454768    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-ed39fa-697974657d-46thd" event={"ID":"77a9a1f5-9f6d-4aea-8f04-52bfb7785214","Type":"ContainerDied","Data":"6f3814bb826a2dd2c911beb9a3f225be3f4b03d0ba824f8656c4b50b039c39b2"}
Sep 10 19:41:50.454834 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:50.454792    2564 scope.go:117] "RemoveContainer" containerID="af554b8b683484378b29a7002cc76284f2c231768252a27617aec1e99084de65"
Sep 10 19:41:50.464527 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:50.464507    2564 scope.go:117] "RemoveContainer" containerID="af554b8b683484378b29a7002cc76284f2c231768252a27617aec1e99084de65"
Sep 10 19:41:50.464852 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:41:50.464833    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af554b8b683484378b29a7002cc76284f2c231768252a27617aec1e99084de65\": container with ID starting with af554b8b683484378b29a7002cc76284f2c231768252a27617aec1e99084de65 not found: ID does not exist" containerID="af554b8b683484378b29a7002cc76284f2c231768252a27617aec1e99084de65"
Sep 10 19:41:50.464937 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:50.464863    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af554b8b683484378b29a7002cc76284f2c231768252a27617aec1e99084de65"} err="failed to get container status \"af554b8b683484378b29a7002cc76284f2c231768252a27617aec1e99084de65\": rpc error: code = NotFound desc = could not find container \"af554b8b683484378b29a7002cc76284f2c231768252a27617aec1e99084de65\": container with ID starting with af554b8b683484378b29a7002cc76284f2c231768252a27617aec1e99084de65 not found: ID does not exist"
Sep 10 19:41:50.478030 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:50.477995    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-ed39fa-697974657d-46thd"]
Sep 10 19:41:50.482212 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:50.482189    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-ed39fa-697974657d-46thd"]
Sep 10 19:41:50.950017 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:50.949977    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15711f0a-cd76-4f89-87b8-b1ad4a47c1ce" path="/var/lib/kubelet/pods/15711f0a-cd76-4f89-87b8-b1ad4a47c1ce/volumes"
Sep 10 19:41:50.950719 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:50.950697    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77a9a1f5-9f6d-4aea-8f04-52bfb7785214" path="/var/lib/kubelet/pods/77a9a1f5-9f6d-4aea-8f04-52bfb7785214/volumes"
Sep 10 19:41:56.719595 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:56.719559    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:41:56.777453 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:56.777413    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:41:56.990121 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:56.990036    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h"]
Sep 10 19:41:56.990335 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:56.990308    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h" podUID="650b189f-bc2e-41a8-bc9d-d4b0cee41294" containerName="istio-proxy" containerID="cri-o://d5aa7d96bbeca19ca36c5e51e982bf8e4c0d77caf52e8d383f48f598d09d3838" gracePeriod=30
Sep 10 19:41:57.005591 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:57.005557    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:41:57.369614 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:57.369580    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-32ce4f-brwmq"]
Sep 10 19:41:57.370077 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:57.370063    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15711f0a-cd76-4f89-87b8-b1ad4a47c1ce" containerName="revoke-keys"
Sep 10 19:41:57.370123 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:57.370078    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="15711f0a-cd76-4f89-87b8-b1ad4a47c1ce" containerName="revoke-keys"
Sep 10 19:41:57.370123 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:57.370095    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7777610-d77a-42c5-b6ed-60cabe303d70" containerName="istio-proxy"
Sep 10 19:41:57.370123 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:57.370101    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7777610-d77a-42c5-b6ed-60cabe303d70" containerName="istio-proxy"
Sep 10 19:41:57.370123 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:57.370109    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77a9a1f5-9f6d-4aea-8f04-52bfb7785214" containerName="payload-pre-processing"
Sep 10 19:41:57.370123 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:57.370115    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="77a9a1f5-9f6d-4aea-8f04-52bfb7785214" containerName="payload-pre-processing"
Sep 10 19:41:57.370281 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:57.370128    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7777610-d77a-42c5-b6ed-60cabe303d70" containerName="istio-proxy"
Sep 10 19:41:57.370281 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:57.370134    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7777610-d77a-42c5-b6ed-60cabe303d70" containerName="istio-proxy"
Sep 10 19:41:57.370281 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:57.370198    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7777610-d77a-42c5-b6ed-60cabe303d70" containerName="istio-proxy"
Sep 10 19:41:57.370281 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:57.370205    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="15711f0a-cd76-4f89-87b8-b1ad4a47c1ce" containerName="revoke-keys"
Sep 10 19:41:57.370281 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:57.370211    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="77a9a1f5-9f6d-4aea-8f04-52bfb7785214" containerName="payload-pre-processing"
Sep 10 19:41:57.370281 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:57.370220    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7777610-d77a-42c5-b6ed-60cabe303d70" containerName="istio-proxy"
Sep 10 19:41:57.374670 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:57.374649    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-32ce4f-brwmq"
Sep 10 19:41:57.377092 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:57.377074    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 19:41:57.377325 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:57.377309    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tjmqn\""
Sep 10 19:41:57.381922 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:57.381902    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-32ce4f-brwmq"]
Sep 10 19:41:57.473172 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:57.473135    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:41:57.547336 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:57.547304    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7066f8b2-dc9f-4269-8d86-73d6353ad6da-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-32ce4f-brwmq\" (UID: \"7066f8b2-dc9f-4269-8d86-73d6353ad6da\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-32ce4f-brwmq"
Sep 10 19:41:57.648539 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:57.648430    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7066f8b2-dc9f-4269-8d86-73d6353ad6da-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-32ce4f-brwmq\" (UID: \"7066f8b2-dc9f-4269-8d86-73d6353ad6da\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-32ce4f-brwmq"
Sep 10 19:41:57.649156 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:57.649131    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7066f8b2-dc9f-4269-8d86-73d6353ad6da-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-32ce4f-brwmq\" (UID: \"7066f8b2-dc9f-4269-8d86-73d6353ad6da\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-32ce4f-brwmq"
Sep 10 19:41:57.686437 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:57.686403    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-32ce4f-brwmq"
Sep 10 19:41:57.959433 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:57.959407    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-32ce4f-brwmq"]
Sep 10 19:41:57.962206 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:41:57.962178    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7066f8b2_dc9f_4269_8d86_73d6353ad6da.slice/crio-e8d66f7a55407b598100d3c69aa6128e46cab419f5be644a598d8f2f74c183ec WatchSource:0}: Error finding container e8d66f7a55407b598100d3c69aa6128e46cab419f5be644a598d8f2f74c183ec: Status 404 returned error can't find the container with id e8d66f7a55407b598100d3c69aa6128e46cab419f5be644a598d8f2f74c183ec
Sep 10 19:41:58.502234 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:58.502195    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-32ce4f-brwmq" event={"ID":"7066f8b2-dc9f-4269-8d86-73d6353ad6da","Type":"ContainerStarted","Data":"d153c970861a2a5e59a5a196613776669640cf2cbde2f29bb0854aa0a2f630c9"}
Sep 10 19:41:58.502412 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:58.502250    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-32ce4f-brwmq" event={"ID":"7066f8b2-dc9f-4269-8d86-73d6353ad6da","Type":"ContainerStarted","Data":"e8d66f7a55407b598100d3c69aa6128e46cab419f5be644a598d8f2f74c183ec"}
Sep 10 19:41:58.518940 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:58.518843    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-32ce4f-brwmq" podStartSLOduration=1.518825413 podStartE2EDuration="1.518825413s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:41:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:41:58.516500243 +0000 UTC m=+1574.117407485" watchObservedRunningTime="2026-09-10 19:41:58.518825413 +0000 UTC m=+1574.119732653"
Sep 10 19:41:59.509660 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:59.509607    2564 generic.go:356] "Generic (PLEG): container finished" podID="7066f8b2-dc9f-4269-8d86-73d6353ad6da" containerID="d153c970861a2a5e59a5a196613776669640cf2cbde2f29bb0854aa0a2f630c9" exitCode=0
Sep 10 19:41:59.510073 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:41:59.509698    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-32ce4f-brwmq" event={"ID":"7066f8b2-dc9f-4269-8d86-73d6353ad6da","Type":"ContainerDied","Data":"d153c970861a2a5e59a5a196613776669640cf2cbde2f29bb0854aa0a2f630c9"}
Sep 10 19:42:00.661678 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:00.661650    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-32ce4f-brwmq"
Sep 10 19:42:00.778883 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:00.778771    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7066f8b2-dc9f-4269-8d86-73d6353ad6da-maas-api-service-ca\") pod \"7066f8b2-dc9f-4269-8d86-73d6353ad6da\" (UID: \"7066f8b2-dc9f-4269-8d86-73d6353ad6da\") "
Sep 10 19:42:00.779135 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:00.779108    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7066f8b2-dc9f-4269-8d86-73d6353ad6da-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7066f8b2-dc9f-4269-8d86-73d6353ad6da" (UID: "7066f8b2-dc9f-4269-8d86-73d6353ad6da"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:42:00.880371 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:00.880319    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7066f8b2-dc9f-4269-8d86-73d6353ad6da-maas-api-service-ca\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:42:01.521698 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:01.521666    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-32ce4f-brwmq"
Sep 10 19:42:01.521894 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:01.521729    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-32ce4f-brwmq" event={"ID":"7066f8b2-dc9f-4269-8d86-73d6353ad6da","Type":"ContainerDied","Data":"e8d66f7a55407b598100d3c69aa6128e46cab419f5be644a598d8f2f74c183ec"}
Sep 10 19:42:01.521894 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:01.521794    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e8d66f7a55407b598100d3c69aa6128e46cab419f5be644a598d8f2f74c183ec"
Sep 10 19:42:02.260548 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:02.260479    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h"
Sep 10 19:42:02.395323 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:02.395286    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/650b189f-bc2e-41a8-bc9d-d4b0cee41294-istiod-ca-cert\") pod \"650b189f-bc2e-41a8-bc9d-d4b0cee41294\" (UID: \"650b189f-bc2e-41a8-bc9d-d4b0cee41294\") "
Sep 10 19:42:02.395323 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:02.395328    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/650b189f-bc2e-41a8-bc9d-d4b0cee41294-istio-data\") pod \"650b189f-bc2e-41a8-bc9d-d4b0cee41294\" (UID: \"650b189f-bc2e-41a8-bc9d-d4b0cee41294\") "
Sep 10 19:42:02.395562 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:02.395353    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/650b189f-bc2e-41a8-bc9d-d4b0cee41294-credential-socket\") pod \"650b189f-bc2e-41a8-bc9d-d4b0cee41294\" (UID: \"650b189f-bc2e-41a8-bc9d-d4b0cee41294\") "
Sep 10 19:42:02.395562 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:02.395396    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/650b189f-bc2e-41a8-bc9d-d4b0cee41294-istio-envoy\") pod \"650b189f-bc2e-41a8-bc9d-d4b0cee41294\" (UID: \"650b189f-bc2e-41a8-bc9d-d4b0cee41294\") "
Sep 10 19:42:02.395562 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:02.395499    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/650b189f-bc2e-41a8-bc9d-d4b0cee41294-workload-certs\") pod \"650b189f-bc2e-41a8-bc9d-d4b0cee41294\" (UID: \"650b189f-bc2e-41a8-bc9d-d4b0cee41294\") "
Sep 10 19:42:02.395562 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:02.395524    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/650b189f-bc2e-41a8-bc9d-d4b0cee41294-istio-token\") pod \"650b189f-bc2e-41a8-bc9d-d4b0cee41294\" (UID: \"650b189f-bc2e-41a8-bc9d-d4b0cee41294\") "
Sep 10 19:42:02.395562 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:02.395547    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/650b189f-bc2e-41a8-bc9d-d4b0cee41294-workload-socket\") pod \"650b189f-bc2e-41a8-bc9d-d4b0cee41294\" (UID: \"650b189f-bc2e-41a8-bc9d-d4b0cee41294\") "
Sep 10 19:42:02.395858 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:02.395713    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/650b189f-bc2e-41a8-bc9d-d4b0cee41294-istio-data" (OuterVolumeSpecName: "istio-data") pod "650b189f-bc2e-41a8-bc9d-d4b0cee41294" (UID: "650b189f-bc2e-41a8-bc9d-d4b0cee41294"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:42:02.395858 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:02.395826    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/650b189f-bc2e-41a8-bc9d-d4b0cee41294-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "650b189f-bc2e-41a8-bc9d-d4b0cee41294" (UID: "650b189f-bc2e-41a8-bc9d-d4b0cee41294"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:42:02.395858 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:02.395819    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/650b189f-bc2e-41a8-bc9d-d4b0cee41294-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "650b189f-bc2e-41a8-bc9d-d4b0cee41294" (UID: "650b189f-bc2e-41a8-bc9d-d4b0cee41294"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:42:02.395858 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:02.395845    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8j8dx\" (UniqueName: \"kubernetes.io/projected/650b189f-bc2e-41a8-bc9d-d4b0cee41294-kube-api-access-8j8dx\") pod \"650b189f-bc2e-41a8-bc9d-d4b0cee41294\" (UID: \"650b189f-bc2e-41a8-bc9d-d4b0cee41294\") "
Sep 10 19:42:02.395858 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:02.395845    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/650b189f-bc2e-41a8-bc9d-d4b0cee41294-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "650b189f-bc2e-41a8-bc9d-d4b0cee41294" (UID: "650b189f-bc2e-41a8-bc9d-d4b0cee41294"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:42:02.396050 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:02.395856    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/650b189f-bc2e-41a8-bc9d-d4b0cee41294-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "650b189f-bc2e-41a8-bc9d-d4b0cee41294" (UID: "650b189f-bc2e-41a8-bc9d-d4b0cee41294"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:42:02.396050 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:02.395922    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/650b189f-bc2e-41a8-bc9d-d4b0cee41294-istio-podinfo\") pod \"650b189f-bc2e-41a8-bc9d-d4b0cee41294\" (UID: \"650b189f-bc2e-41a8-bc9d-d4b0cee41294\") "
Sep 10 19:42:02.396268 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:02.396251    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/650b189f-bc2e-41a8-bc9d-d4b0cee41294-workload-certs\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:42:02.396339 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:02.396269    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/650b189f-bc2e-41a8-bc9d-d4b0cee41294-workload-socket\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:42:02.396339 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:02.396278    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/650b189f-bc2e-41a8-bc9d-d4b0cee41294-istiod-ca-cert\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:42:02.396339 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:02.396288    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/650b189f-bc2e-41a8-bc9d-d4b0cee41294-istio-data\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:42:02.396339 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:02.396297    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/650b189f-bc2e-41a8-bc9d-d4b0cee41294-credential-socket\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:42:02.397974 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:02.397942    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/650b189f-bc2e-41a8-bc9d-d4b0cee41294-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "650b189f-bc2e-41a8-bc9d-d4b0cee41294" (UID: "650b189f-bc2e-41a8-bc9d-d4b0cee41294"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 19:42:02.398098 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:02.397968    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/650b189f-bc2e-41a8-bc9d-d4b0cee41294-kube-api-access-8j8dx" (OuterVolumeSpecName: "kube-api-access-8j8dx") pod "650b189f-bc2e-41a8-bc9d-d4b0cee41294" (UID: "650b189f-bc2e-41a8-bc9d-d4b0cee41294"). InnerVolumeSpecName "kube-api-access-8j8dx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:42:02.398159 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:02.398131    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/650b189f-bc2e-41a8-bc9d-d4b0cee41294-istio-token" (OuterVolumeSpecName: "istio-token") pod "650b189f-bc2e-41a8-bc9d-d4b0cee41294" (UID: "650b189f-bc2e-41a8-bc9d-d4b0cee41294"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:42:02.398222 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:02.398202    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/650b189f-bc2e-41a8-bc9d-d4b0cee41294-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "650b189f-bc2e-41a8-bc9d-d4b0cee41294" (UID: "650b189f-bc2e-41a8-bc9d-d4b0cee41294"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:42:02.497747 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:02.497702    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8j8dx\" (UniqueName: \"kubernetes.io/projected/650b189f-bc2e-41a8-bc9d-d4b0cee41294-kube-api-access-8j8dx\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:42:02.497747 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:02.497739    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/650b189f-bc2e-41a8-bc9d-d4b0cee41294-istio-podinfo\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:42:02.497747 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:02.497752    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/650b189f-bc2e-41a8-bc9d-d4b0cee41294-istio-envoy\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:42:02.498056 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:02.497761    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/650b189f-bc2e-41a8-bc9d-d4b0cee41294-istio-token\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:42:02.529558 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:02.529468    2564 generic.go:356] "Generic (PLEG): container finished" podID="650b189f-bc2e-41a8-bc9d-d4b0cee41294" containerID="d5aa7d96bbeca19ca36c5e51e982bf8e4c0d77caf52e8d383f48f598d09d3838" exitCode=0
Sep 10 19:42:02.529558 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:02.529529    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h"
Sep 10 19:42:02.529775 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:02.529552    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h" event={"ID":"650b189f-bc2e-41a8-bc9d-d4b0cee41294","Type":"ContainerDied","Data":"d5aa7d96bbeca19ca36c5e51e982bf8e4c0d77caf52e8d383f48f598d09d3838"}
Sep 10 19:42:02.529775 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:02.529606    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h" event={"ID":"650b189f-bc2e-41a8-bc9d-d4b0cee41294","Type":"ContainerDied","Data":"51b6e6e0780c7b7ef21aa6a60e0a27757107d2fa7354d725446e434ca5ed9175"}
Sep 10 19:42:02.529775 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:02.529659    2564 scope.go:117] "RemoveContainer" containerID="d5aa7d96bbeca19ca36c5e51e982bf8e4c0d77caf52e8d383f48f598d09d3838"
Sep 10 19:42:02.540158 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:02.540136    2564 scope.go:117] "RemoveContainer" containerID="f690ede07373401b6be14c489e97f609f7b2e980094424b267614ec72fa28d03"
Sep 10 19:42:02.548839 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:02.548819    2564 scope.go:117] "RemoveContainer" containerID="d5aa7d96bbeca19ca36c5e51e982bf8e4c0d77caf52e8d383f48f598d09d3838"
Sep 10 19:42:02.549130 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:42:02.549110    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5aa7d96bbeca19ca36c5e51e982bf8e4c0d77caf52e8d383f48f598d09d3838\": container with ID starting with d5aa7d96bbeca19ca36c5e51e982bf8e4c0d77caf52e8d383f48f598d09d3838 not found: ID does not exist" containerID="d5aa7d96bbeca19ca36c5e51e982bf8e4c0d77caf52e8d383f48f598d09d3838"
Sep 10 19:42:02.549176 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:02.549140    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5aa7d96bbeca19ca36c5e51e982bf8e4c0d77caf52e8d383f48f598d09d3838"} err="failed to get container status \"d5aa7d96bbeca19ca36c5e51e982bf8e4c0d77caf52e8d383f48f598d09d3838\": rpc error: code = NotFound desc = could not find container \"d5aa7d96bbeca19ca36c5e51e982bf8e4c0d77caf52e8d383f48f598d09d3838\": container with ID starting with d5aa7d96bbeca19ca36c5e51e982bf8e4c0d77caf52e8d383f48f598d09d3838 not found: ID does not exist"
Sep 10 19:42:02.549176 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:02.549159    2564 scope.go:117] "RemoveContainer" containerID="f690ede07373401b6be14c489e97f609f7b2e980094424b267614ec72fa28d03"
Sep 10 19:42:02.549431 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:42:02.549413    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f690ede07373401b6be14c489e97f609f7b2e980094424b267614ec72fa28d03\": container with ID starting with f690ede07373401b6be14c489e97f609f7b2e980094424b267614ec72fa28d03 not found: ID does not exist" containerID="f690ede07373401b6be14c489e97f609f7b2e980094424b267614ec72fa28d03"
Sep 10 19:42:02.549475 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:02.549438    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f690ede07373401b6be14c489e97f609f7b2e980094424b267614ec72fa28d03"} err="failed to get container status \"f690ede07373401b6be14c489e97f609f7b2e980094424b267614ec72fa28d03\": rpc error: code = NotFound desc = could not find container \"f690ede07373401b6be14c489e97f609f7b2e980094424b267614ec72fa28d03\": container with ID starting with f690ede07373401b6be14c489e97f609f7b2e980094424b267614ec72fa28d03 not found: ID does not exist"
Sep 10 19:42:02.553866 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:02.553839    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h"]
Sep 10 19:42:02.558464 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:02.558438    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-ed39fa-openshift-default-69f8b89b58-9w28h"]
Sep 10 19:42:02.956178 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:02.956095    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="650b189f-bc2e-41a8-bc9d-d4b0cee41294" path="/var/lib/kubelet/pods/650b189f-bc2e-41a8-bc9d-d4b0cee41294/volumes"
Sep 10 19:42:07.876365 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:07.876325    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:42:07.905263 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:07.905225    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-32ce4f-8896f4555-89kkl"]
Sep 10 19:42:07.905530 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:07.905498    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-32ce4f-8896f4555-89kkl" podUID="731b5f5c-45a1-466d-b0b8-143680a1733d" containerName="payload-processing" containerID="cri-o://4266291e64b680947830fdbb09fe222e1f27e244918efdf108ab0c00200bcab9" gracePeriod=30
Sep 10 19:42:07.913986 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:07.913960    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-32ce4f-649996998-6pzhs"]
Sep 10 19:42:07.914212 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:07.914188    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-32ce4f-649996998-6pzhs" podUID="649c71f4-a31f-4934-a004-a0f41ddd57cc" containerName="payload-pre-processing" containerID="cri-o://6c0a40ba9e95c80c0c433aa21ebcec5bc706e70b056d262a9ed48cd0a8ffb128" gracePeriod=30
Sep 10 19:42:07.945611 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:07.945573    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:42:08.001514 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:08.001480    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:42:08.079326 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:08.078549    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-32ce4f-brwmq"]
Sep 10 19:42:08.080815 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:08.080786    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:42:08.081653 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:08.081615    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-32ce4f-brwmq"]
Sep 10 19:42:08.135017 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:08.134837    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:42:08.196513 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:08.196487    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-32ce4f-8896f4555-89kkl"
Sep 10 19:42:08.199354 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:08.199330    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-32ce4f-649996998-6pzhs"
Sep 10 19:42:08.353617 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:08.353577    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/731b5f5c-45a1-466d-b0b8-143680a1733d-plugins-config-volume\") pod \"731b5f5c-45a1-466d-b0b8-143680a1733d\" (UID: \"731b5f5c-45a1-466d-b0b8-143680a1733d\") "
Sep 10 19:42:08.353829 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:08.353642    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/649c71f4-a31f-4934-a004-a0f41ddd57cc-plugins-config-volume\") pod \"649c71f4-a31f-4934-a004-a0f41ddd57cc\" (UID: \"649c71f4-a31f-4934-a004-a0f41ddd57cc\") "
Sep 10 19:42:08.353829 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:08.353700    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zn65m\" (UniqueName: \"kubernetes.io/projected/731b5f5c-45a1-466d-b0b8-143680a1733d-kube-api-access-zn65m\") pod \"731b5f5c-45a1-466d-b0b8-143680a1733d\" (UID: \"731b5f5c-45a1-466d-b0b8-143680a1733d\") "
Sep 10 19:42:08.353829 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:08.353731    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m4rgb\" (UniqueName: \"kubernetes.io/projected/649c71f4-a31f-4934-a004-a0f41ddd57cc-kube-api-access-m4rgb\") pod \"649c71f4-a31f-4934-a004-a0f41ddd57cc\" (UID: \"649c71f4-a31f-4934-a004-a0f41ddd57cc\") "
Sep 10 19:42:08.354090 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:08.354060    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/731b5f5c-45a1-466d-b0b8-143680a1733d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "731b5f5c-45a1-466d-b0b8-143680a1733d" (UID: "731b5f5c-45a1-466d-b0b8-143680a1733d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:42:08.354183 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:08.354069    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/649c71f4-a31f-4934-a004-a0f41ddd57cc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "649c71f4-a31f-4934-a004-a0f41ddd57cc" (UID: "649c71f4-a31f-4934-a004-a0f41ddd57cc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:42:08.355732 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:08.355704    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/731b5f5c-45a1-466d-b0b8-143680a1733d-kube-api-access-zn65m" (OuterVolumeSpecName: "kube-api-access-zn65m") pod "731b5f5c-45a1-466d-b0b8-143680a1733d" (UID: "731b5f5c-45a1-466d-b0b8-143680a1733d"). InnerVolumeSpecName "kube-api-access-zn65m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:42:08.355858 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:08.355835    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/649c71f4-a31f-4934-a004-a0f41ddd57cc-kube-api-access-m4rgb" (OuterVolumeSpecName: "kube-api-access-m4rgb") pod "649c71f4-a31f-4934-a004-a0f41ddd57cc" (UID: "649c71f4-a31f-4934-a004-a0f41ddd57cc"). InnerVolumeSpecName "kube-api-access-m4rgb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:42:08.455494 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:08.455394    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m4rgb\" (UniqueName: \"kubernetes.io/projected/649c71f4-a31f-4934-a004-a0f41ddd57cc-kube-api-access-m4rgb\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:42:08.455494 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:08.455429    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/731b5f5c-45a1-466d-b0b8-143680a1733d-plugins-config-volume\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:42:08.455494 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:08.455443    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/649c71f4-a31f-4934-a004-a0f41ddd57cc-plugins-config-volume\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:42:08.455494 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:08.455454    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zn65m\" (UniqueName: \"kubernetes.io/projected/731b5f5c-45a1-466d-b0b8-143680a1733d-kube-api-access-zn65m\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:42:08.468014 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:08.467975    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:42:08.567840 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:08.567803    2564 generic.go:356] "Generic (PLEG): container finished" podID="649c71f4-a31f-4934-a004-a0f41ddd57cc" containerID="6c0a40ba9e95c80c0c433aa21ebcec5bc706e70b056d262a9ed48cd0a8ffb128" exitCode=0
Sep 10 19:42:08.568015 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:08.567870    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-32ce4f-649996998-6pzhs"
Sep 10 19:42:08.568015 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:08.567889    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-32ce4f-649996998-6pzhs" event={"ID":"649c71f4-a31f-4934-a004-a0f41ddd57cc","Type":"ContainerDied","Data":"6c0a40ba9e95c80c0c433aa21ebcec5bc706e70b056d262a9ed48cd0a8ffb128"}
Sep 10 19:42:08.568015 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:08.567985    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-32ce4f-649996998-6pzhs" event={"ID":"649c71f4-a31f-4934-a004-a0f41ddd57cc","Type":"ContainerDied","Data":"9ccff95d41a1105865c08c37b2d6771feddfe2493a4e82c73237ed909da90d65"}
Sep 10 19:42:08.568015 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:08.568013    2564 scope.go:117] "RemoveContainer" containerID="6c0a40ba9e95c80c0c433aa21ebcec5bc706e70b056d262a9ed48cd0a8ffb128"
Sep 10 19:42:08.569426 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:08.569400    2564 generic.go:356] "Generic (PLEG): container finished" podID="731b5f5c-45a1-466d-b0b8-143680a1733d" containerID="4266291e64b680947830fdbb09fe222e1f27e244918efdf108ab0c00200bcab9" exitCode=0
Sep 10 19:42:08.569546 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:08.569433    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-32ce4f-8896f4555-89kkl" event={"ID":"731b5f5c-45a1-466d-b0b8-143680a1733d","Type":"ContainerDied","Data":"4266291e64b680947830fdbb09fe222e1f27e244918efdf108ab0c00200bcab9"}
Sep 10 19:42:08.569546 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:08.569463    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-32ce4f-8896f4555-89kkl"
Sep 10 19:42:08.569546 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:08.569486    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-32ce4f-8896f4555-89kkl" event={"ID":"731b5f5c-45a1-466d-b0b8-143680a1733d","Type":"ContainerDied","Data":"f6c78e131cae99dcca607b0fdefa22aac873eeef88ad2c85d263ab117582ee1d"}
Sep 10 19:42:08.578733 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:08.578714    2564 scope.go:117] "RemoveContainer" containerID="6c0a40ba9e95c80c0c433aa21ebcec5bc706e70b056d262a9ed48cd0a8ffb128"
Sep 10 19:42:08.579064 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:42:08.579043    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c0a40ba9e95c80c0c433aa21ebcec5bc706e70b056d262a9ed48cd0a8ffb128\": container with ID starting with 6c0a40ba9e95c80c0c433aa21ebcec5bc706e70b056d262a9ed48cd0a8ffb128 not found: ID does not exist" containerID="6c0a40ba9e95c80c0c433aa21ebcec5bc706e70b056d262a9ed48cd0a8ffb128"
Sep 10 19:42:08.579158 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:08.579073    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c0a40ba9e95c80c0c433aa21ebcec5bc706e70b056d262a9ed48cd0a8ffb128"} err="failed to get container status \"6c0a40ba9e95c80c0c433aa21ebcec5bc706e70b056d262a9ed48cd0a8ffb128\": rpc error: code = NotFound desc = could not find container \"6c0a40ba9e95c80c0c433aa21ebcec5bc706e70b056d262a9ed48cd0a8ffb128\": container with ID starting with 6c0a40ba9e95c80c0c433aa21ebcec5bc706e70b056d262a9ed48cd0a8ffb128 not found: ID does not exist"
Sep 10 19:42:08.579158 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:08.579091    2564 scope.go:117] "RemoveContainer" containerID="4266291e64b680947830fdbb09fe222e1f27e244918efdf108ab0c00200bcab9"
Sep 10 19:42:08.587887 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:08.587868    2564 scope.go:117] "RemoveContainer" containerID="4266291e64b680947830fdbb09fe222e1f27e244918efdf108ab0c00200bcab9"
Sep 10 19:42:08.588189 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:42:08.588168    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4266291e64b680947830fdbb09fe222e1f27e244918efdf108ab0c00200bcab9\": container with ID starting with 4266291e64b680947830fdbb09fe222e1f27e244918efdf108ab0c00200bcab9 not found: ID does not exist" containerID="4266291e64b680947830fdbb09fe222e1f27e244918efdf108ab0c00200bcab9"
Sep 10 19:42:08.588279 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:08.588194    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4266291e64b680947830fdbb09fe222e1f27e244918efdf108ab0c00200bcab9"} err="failed to get container status \"4266291e64b680947830fdbb09fe222e1f27e244918efdf108ab0c00200bcab9\": rpc error: code = NotFound desc = could not find container \"4266291e64b680947830fdbb09fe222e1f27e244918efdf108ab0c00200bcab9\": container with ID starting with 4266291e64b680947830fdbb09fe222e1f27e244918efdf108ab0c00200bcab9 not found: ID does not exist"
Sep 10 19:42:08.592617 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:08.592593    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-32ce4f-649996998-6pzhs"]
Sep 10 19:42:08.597152 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:08.597132    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-32ce4f-649996998-6pzhs"]
Sep 10 19:42:08.608633 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:08.608597    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-32ce4f-8896f4555-89kkl"]
Sep 10 19:42:08.614514 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:08.614489    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-32ce4f-8896f4555-89kkl"]
Sep 10 19:42:08.949237 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:08.949204    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="649c71f4-a31f-4934-a004-a0f41ddd57cc" path="/var/lib/kubelet/pods/649c71f4-a31f-4934-a004-a0f41ddd57cc/volumes"
Sep 10 19:42:08.949577 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:08.949544    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7066f8b2-dc9f-4269-8d86-73d6353ad6da" path="/var/lib/kubelet/pods/7066f8b2-dc9f-4269-8d86-73d6353ad6da/volumes"
Sep 10 19:42:08.949901 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:08.949887    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="731b5f5c-45a1-466d-b0b8-143680a1733d" path="/var/lib/kubelet/pods/731b5f5c-45a1-466d-b0b8-143680a1733d/volumes"
Sep 10 19:42:15.079449 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:15.079410    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:42:15.137094 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:15.137046    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:42:15.334182 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:15.334091    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98"]
Sep 10 19:42:15.334423 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:15.334393    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98" podUID="e1301503-9300-43b1-913e-589ea3d31f71" containerName="istio-proxy" containerID="cri-o://d7e7fbdb5f09486c263183d281e571fc0dfee4002f6acb332605460a061cf76c" gracePeriod=30
Sep 10 19:42:15.350734 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:15.350688    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:42:15.707104 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:15.707020    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-7bfb5d-zbf7z"]
Sep 10 19:42:15.707458 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:15.707445    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="649c71f4-a31f-4934-a004-a0f41ddd57cc" containerName="payload-pre-processing"
Sep 10 19:42:15.707506 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:15.707459    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="649c71f4-a31f-4934-a004-a0f41ddd57cc" containerName="payload-pre-processing"
Sep 10 19:42:15.707506 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:15.707469    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="650b189f-bc2e-41a8-bc9d-d4b0cee41294" containerName="istio-proxy"
Sep 10 19:42:15.707506 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:15.707475    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="650b189f-bc2e-41a8-bc9d-d4b0cee41294" containerName="istio-proxy"
Sep 10 19:42:15.707506 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:15.707487    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7066f8b2-dc9f-4269-8d86-73d6353ad6da" containerName="revoke-keys"
Sep 10 19:42:15.707506 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:15.707494    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7066f8b2-dc9f-4269-8d86-73d6353ad6da" containerName="revoke-keys"
Sep 10 19:42:15.707506 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:15.707503    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="731b5f5c-45a1-466d-b0b8-143680a1733d" containerName="payload-processing"
Sep 10 19:42:15.707506 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:15.707507    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="731b5f5c-45a1-466d-b0b8-143680a1733d" containerName="payload-processing"
Sep 10 19:42:15.707791 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:15.707515    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="650b189f-bc2e-41a8-bc9d-d4b0cee41294" containerName="istio-proxy"
Sep 10 19:42:15.707791 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:15.707520    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="650b189f-bc2e-41a8-bc9d-d4b0cee41294" containerName="istio-proxy"
Sep 10 19:42:15.707791 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:15.707586    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="649c71f4-a31f-4934-a004-a0f41ddd57cc" containerName="payload-pre-processing"
Sep 10 19:42:15.707791 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:15.707594    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7066f8b2-dc9f-4269-8d86-73d6353ad6da" containerName="revoke-keys"
Sep 10 19:42:15.707791 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:15.707601    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="650b189f-bc2e-41a8-bc9d-d4b0cee41294" containerName="istio-proxy"
Sep 10 19:42:15.707791 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:15.707607    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="650b189f-bc2e-41a8-bc9d-d4b0cee41294" containerName="istio-proxy"
Sep 10 19:42:15.707791 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:15.707614    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="731b5f5c-45a1-466d-b0b8-143680a1733d" containerName="payload-processing"
Sep 10 19:42:15.712053 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:15.712035    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-7bfb5d-zbf7z"
Sep 10 19:42:15.714285 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:15.714265    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 19:42:15.714411 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:15.714345    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tjmqn\""
Sep 10 19:42:15.720471 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:15.720447    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-7bfb5d-zbf7z"]
Sep 10 19:42:15.827281 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:15.827243    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d216f39-a2df-4d2b-83e0-8d2bb0b34cf6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-7bfb5d-zbf7z\" (UID: \"4d216f39-a2df-4d2b-83e0-8d2bb0b34cf6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-7bfb5d-zbf7z"
Sep 10 19:42:15.864554 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:15.864519    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:42:15.928008 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:15.927975    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d216f39-a2df-4d2b-83e0-8d2bb0b34cf6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-7bfb5d-zbf7z\" (UID: \"4d216f39-a2df-4d2b-83e0-8d2bb0b34cf6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-7bfb5d-zbf7z"
Sep 10 19:42:15.928593 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:15.928575    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d216f39-a2df-4d2b-83e0-8d2bb0b34cf6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-7bfb5d-zbf7z\" (UID: \"4d216f39-a2df-4d2b-83e0-8d2bb0b34cf6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-7bfb5d-zbf7z"
Sep 10 19:42:16.024533 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:16.024495    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-7bfb5d-zbf7z"
Sep 10 19:42:16.298109 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:16.298085    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-7bfb5d-zbf7z"]
Sep 10 19:42:16.300119 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:42:16.300092    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d216f39_a2df_4d2b_83e0_8d2bb0b34cf6.slice/crio-30dc6fd08dfc58d6770b8ba5dc8e672d1cf673a4e9328c51b967958cf4c89540 WatchSource:0}: Error finding container 30dc6fd08dfc58d6770b8ba5dc8e672d1cf673a4e9328c51b967958cf4c89540: Status 404 returned error can't find the container with id 30dc6fd08dfc58d6770b8ba5dc8e672d1cf673a4e9328c51b967958cf4c89540
Sep 10 19:42:16.618848 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:16.618812    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-7bfb5d-zbf7z" event={"ID":"4d216f39-a2df-4d2b-83e0-8d2bb0b34cf6","Type":"ContainerStarted","Data":"910fcdbd1f922e9cb0b6ad83d755a51ecbcd4a6e9de78f7cadfaa8e07d536a9d"}
Sep 10 19:42:16.619034 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:16.618868    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-7bfb5d-zbf7z" event={"ID":"4d216f39-a2df-4d2b-83e0-8d2bb0b34cf6","Type":"ContainerStarted","Data":"30dc6fd08dfc58d6770b8ba5dc8e672d1cf673a4e9328c51b967958cf4c89540"}
Sep 10 19:42:16.635381 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:16.635336    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-7bfb5d-zbf7z" podStartSLOduration=1.635319994 podStartE2EDuration="1.635319994s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:42:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:42:16.634094698 +0000 UTC m=+1592.235001939" watchObservedRunningTime="2026-09-10 19:42:16.635319994 +0000 UTC m=+1592.236227256"
Sep 10 19:42:17.626391 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:17.626353    2564 generic.go:356] "Generic (PLEG): container finished" podID="4d216f39-a2df-4d2b-83e0-8d2bb0b34cf6" containerID="910fcdbd1f922e9cb0b6ad83d755a51ecbcd4a6e9de78f7cadfaa8e07d536a9d" exitCode=0
Sep 10 19:42:17.626861 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:17.626433    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-7bfb5d-zbf7z" event={"ID":"4d216f39-a2df-4d2b-83e0-8d2bb0b34cf6","Type":"ContainerDied","Data":"910fcdbd1f922e9cb0b6ad83d755a51ecbcd4a6e9de78f7cadfaa8e07d536a9d"}
Sep 10 19:42:18.780463 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:18.780431    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-7bfb5d-zbf7z"
Sep 10 19:42:18.857708 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:18.856775    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d216f39-a2df-4d2b-83e0-8d2bb0b34cf6-maas-api-service-ca\") pod \"4d216f39-a2df-4d2b-83e0-8d2bb0b34cf6\" (UID: \"4d216f39-a2df-4d2b-83e0-8d2bb0b34cf6\") "
Sep 10 19:42:18.857708 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:18.857590    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d216f39-a2df-4d2b-83e0-8d2bb0b34cf6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4d216f39-a2df-4d2b-83e0-8d2bb0b34cf6" (UID: "4d216f39-a2df-4d2b-83e0-8d2bb0b34cf6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:42:18.958046 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:18.958022    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d216f39-a2df-4d2b-83e0-8d2bb0b34cf6-maas-api-service-ca\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:42:19.639641 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:19.639593    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-7bfb5d-zbf7z"
Sep 10 19:42:19.639852 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:19.639596    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-7bfb5d-zbf7z" event={"ID":"4d216f39-a2df-4d2b-83e0-8d2bb0b34cf6","Type":"ContainerDied","Data":"30dc6fd08dfc58d6770b8ba5dc8e672d1cf673a4e9328c51b967958cf4c89540"}
Sep 10 19:42:19.639852 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:19.639752    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="30dc6fd08dfc58d6770b8ba5dc8e672d1cf673a4e9328c51b967958cf4c89540"
Sep 10 19:42:20.593799 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:20.593769    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98"
Sep 10 19:42:20.647252 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:20.647153    2564 generic.go:356] "Generic (PLEG): container finished" podID="e1301503-9300-43b1-913e-589ea3d31f71" containerID="d7e7fbdb5f09486c263183d281e571fc0dfee4002f6acb332605460a061cf76c" exitCode=0
Sep 10 19:42:20.647252 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:20.647217    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98" event={"ID":"e1301503-9300-43b1-913e-589ea3d31f71","Type":"ContainerDied","Data":"d7e7fbdb5f09486c263183d281e571fc0dfee4002f6acb332605460a061cf76c"}
Sep 10 19:42:20.647472 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:20.647266    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98"
Sep 10 19:42:20.647472 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:20.647286    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98" event={"ID":"e1301503-9300-43b1-913e-589ea3d31f71","Type":"ContainerDied","Data":"097b02b7d45ca86644ec05bf6bfad658e8a640223f9eb0b2e49ad3b846696eb2"}
Sep 10 19:42:20.647472 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:20.647315    2564 scope.go:117] "RemoveContainer" containerID="d7e7fbdb5f09486c263183d281e571fc0dfee4002f6acb332605460a061cf76c"
Sep 10 19:42:20.657512 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:20.657482    2564 scope.go:117] "RemoveContainer" containerID="62cb85ae6a53dc79bef7ef86877292824e937d96b1869593a5092bb38c578bcf"
Sep 10 19:42:20.666073 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:20.666050    2564 scope.go:117] "RemoveContainer" containerID="d7e7fbdb5f09486c263183d281e571fc0dfee4002f6acb332605460a061cf76c"
Sep 10 19:42:20.666430 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:42:20.666412    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7e7fbdb5f09486c263183d281e571fc0dfee4002f6acb332605460a061cf76c\": container with ID starting with d7e7fbdb5f09486c263183d281e571fc0dfee4002f6acb332605460a061cf76c not found: ID does not exist" containerID="d7e7fbdb5f09486c263183d281e571fc0dfee4002f6acb332605460a061cf76c"
Sep 10 19:42:20.666488 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:20.666440    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7e7fbdb5f09486c263183d281e571fc0dfee4002f6acb332605460a061cf76c"} err="failed to get container status \"d7e7fbdb5f09486c263183d281e571fc0dfee4002f6acb332605460a061cf76c\": rpc error: code = NotFound desc = could not find container \"d7e7fbdb5f09486c263183d281e571fc0dfee4002f6acb332605460a061cf76c\": container with ID starting with d7e7fbdb5f09486c263183d281e571fc0dfee4002f6acb332605460a061cf76c not found: ID does not exist"
Sep 10 19:42:20.666488 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:20.666458    2564 scope.go:117] "RemoveContainer" containerID="62cb85ae6a53dc79bef7ef86877292824e937d96b1869593a5092bb38c578bcf"
Sep 10 19:42:20.666769 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:42:20.666749    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62cb85ae6a53dc79bef7ef86877292824e937d96b1869593a5092bb38c578bcf\": container with ID starting with 62cb85ae6a53dc79bef7ef86877292824e937d96b1869593a5092bb38c578bcf not found: ID does not exist" containerID="62cb85ae6a53dc79bef7ef86877292824e937d96b1869593a5092bb38c578bcf"
Sep 10 19:42:20.666840 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:20.666782    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62cb85ae6a53dc79bef7ef86877292824e937d96b1869593a5092bb38c578bcf"} err="failed to get container status \"62cb85ae6a53dc79bef7ef86877292824e937d96b1869593a5092bb38c578bcf\": rpc error: code = NotFound desc = could not find container \"62cb85ae6a53dc79bef7ef86877292824e937d96b1869593a5092bb38c578bcf\": container with ID starting with 62cb85ae6a53dc79bef7ef86877292824e937d96b1869593a5092bb38c578bcf not found: ID does not exist"
Sep 10 19:42:20.677135 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:20.677105    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1301503-9300-43b1-913e-589ea3d31f71-istio-podinfo\") pod \"e1301503-9300-43b1-913e-589ea3d31f71\" (UID: \"e1301503-9300-43b1-913e-589ea3d31f71\") "
Sep 10 19:42:20.677226 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:20.677144    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1301503-9300-43b1-913e-589ea3d31f71-istiod-ca-cert\") pod \"e1301503-9300-43b1-913e-589ea3d31f71\" (UID: \"e1301503-9300-43b1-913e-589ea3d31f71\") "
Sep 10 19:42:20.677226 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:20.677171    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1301503-9300-43b1-913e-589ea3d31f71-workload-socket\") pod \"e1301503-9300-43b1-913e-589ea3d31f71\" (UID: \"e1301503-9300-43b1-913e-589ea3d31f71\") "
Sep 10 19:42:20.677226 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:20.677209    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sb4lw\" (UniqueName: \"kubernetes.io/projected/e1301503-9300-43b1-913e-589ea3d31f71-kube-api-access-sb4lw\") pod \"e1301503-9300-43b1-913e-589ea3d31f71\" (UID: \"e1301503-9300-43b1-913e-589ea3d31f71\") "
Sep 10 19:42:20.677374 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:20.677245    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1301503-9300-43b1-913e-589ea3d31f71-istio-data\") pod \"e1301503-9300-43b1-913e-589ea3d31f71\" (UID: \"e1301503-9300-43b1-913e-589ea3d31f71\") "
Sep 10 19:42:20.677374 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:20.677297    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1301503-9300-43b1-913e-589ea3d31f71-istio-token\") pod \"e1301503-9300-43b1-913e-589ea3d31f71\" (UID: \"e1301503-9300-43b1-913e-589ea3d31f71\") "
Sep 10 19:42:20.677374 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:20.677354    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1301503-9300-43b1-913e-589ea3d31f71-istio-envoy\") pod \"e1301503-9300-43b1-913e-589ea3d31f71\" (UID: \"e1301503-9300-43b1-913e-589ea3d31f71\") "
Sep 10 19:42:20.677533 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:20.677387    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1301503-9300-43b1-913e-589ea3d31f71-workload-certs\") pod \"e1301503-9300-43b1-913e-589ea3d31f71\" (UID: \"e1301503-9300-43b1-913e-589ea3d31f71\") "
Sep 10 19:42:20.677533 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:20.677420    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1301503-9300-43b1-913e-589ea3d31f71-credential-socket\") pod \"e1301503-9300-43b1-913e-589ea3d31f71\" (UID: \"e1301503-9300-43b1-913e-589ea3d31f71\") "
Sep 10 19:42:20.677655 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:20.677555    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1301503-9300-43b1-913e-589ea3d31f71-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e1301503-9300-43b1-913e-589ea3d31f71" (UID: "e1301503-9300-43b1-913e-589ea3d31f71"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:42:20.677655 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:20.677575    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1301503-9300-43b1-913e-589ea3d31f71-istio-data" (OuterVolumeSpecName: "istio-data") pod "e1301503-9300-43b1-913e-589ea3d31f71" (UID: "e1301503-9300-43b1-913e-589ea3d31f71"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:42:20.677655 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:20.677604    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1301503-9300-43b1-913e-589ea3d31f71-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e1301503-9300-43b1-913e-589ea3d31f71" (UID: "e1301503-9300-43b1-913e-589ea3d31f71"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:42:20.677838 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:20.677797    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1301503-9300-43b1-913e-589ea3d31f71-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e1301503-9300-43b1-913e-589ea3d31f71" (UID: "e1301503-9300-43b1-913e-589ea3d31f71"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:42:20.677919 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:20.677892    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1301503-9300-43b1-913e-589ea3d31f71-credential-socket\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:42:20.677985 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:20.677918    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1301503-9300-43b1-913e-589ea3d31f71-istiod-ca-cert\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:42:20.677985 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:20.677933    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1301503-9300-43b1-913e-589ea3d31f71-workload-socket\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:42:20.677985 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:20.677947    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1301503-9300-43b1-913e-589ea3d31f71-istio-data\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:42:20.678136 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:20.678045    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1301503-9300-43b1-913e-589ea3d31f71-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e1301503-9300-43b1-913e-589ea3d31f71" (UID: "e1301503-9300-43b1-913e-589ea3d31f71"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:42:20.679591 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:20.679568    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1301503-9300-43b1-913e-589ea3d31f71-kube-api-access-sb4lw" (OuterVolumeSpecName: "kube-api-access-sb4lw") pod "e1301503-9300-43b1-913e-589ea3d31f71" (UID: "e1301503-9300-43b1-913e-589ea3d31f71"). InnerVolumeSpecName "kube-api-access-sb4lw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:42:20.679702 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:20.679584    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1301503-9300-43b1-913e-589ea3d31f71-istio-token" (OuterVolumeSpecName: "istio-token") pod "e1301503-9300-43b1-913e-589ea3d31f71" (UID: "e1301503-9300-43b1-913e-589ea3d31f71"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:42:20.679702 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:20.679641    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e1301503-9300-43b1-913e-589ea3d31f71-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e1301503-9300-43b1-913e-589ea3d31f71" (UID: "e1301503-9300-43b1-913e-589ea3d31f71"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 19:42:20.679702 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:20.679676    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1301503-9300-43b1-913e-589ea3d31f71-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e1301503-9300-43b1-913e-589ea3d31f71" (UID: "e1301503-9300-43b1-913e-589ea3d31f71"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:42:20.779004 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:20.778966    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1301503-9300-43b1-913e-589ea3d31f71-istio-podinfo\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:42:20.779004 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:20.778997    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sb4lw\" (UniqueName: \"kubernetes.io/projected/e1301503-9300-43b1-913e-589ea3d31f71-kube-api-access-sb4lw\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:42:20.779004 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:20.779007    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1301503-9300-43b1-913e-589ea3d31f71-istio-token\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:42:20.779004 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:20.779015    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1301503-9300-43b1-913e-589ea3d31f71-istio-envoy\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:42:20.779275 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:20.779024    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1301503-9300-43b1-913e-589ea3d31f71-workload-certs\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:42:20.974489 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:20.974453    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98"]
Sep 10 19:42:20.978866 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:20.978832    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-32ce4f-openshift-default-7b7dd7f644-vtv98"]
Sep 10 19:42:22.950303 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:22.950272    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1301503-9300-43b1-913e-589ea3d31f71" path="/var/lib/kubelet/pods/e1301503-9300-43b1-913e-589ea3d31f71/volumes"
Sep 10 19:42:26.239814 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:26.239780    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:42:26.315389 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:26.315353    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:42:26.375685 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:26.375644    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:42:26.465788 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:26.465758    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-7bfb5d-zbf7z"]
Sep 10 19:42:26.469362 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:26.469327    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:42:26.469595 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:26.469577    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-7bfb5d-zbf7z"]
Sep 10 19:42:26.528423 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:26.528386    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:42:26.826540 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:26.826442    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:42:26.949138 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:26.949109    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d216f39-a2df-4d2b-83e0-8d2bb0b34cf6" path="/var/lib/kubelet/pods/4d216f39-a2df-4d2b-83e0-8d2bb0b34cf6/volumes"
Sep 10 19:42:33.247296 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:33.247265    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:42:33.612774 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:33.612681    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:42:33.670105 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:33.670059    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:42:33.868220 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:33.868127    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46"]
Sep 10 19:42:33.868423 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:33.868394    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46" podUID="b4e00663-6fca-498b-926e-716be61dff37" containerName="istio-proxy" containerID="cri-o://e984a042c1f2460152cf4e1a77f41af578f86cded44cc4f353ec087ba863aad1" gracePeriod=30
Sep 10 19:42:33.891802 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:33.891772    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:42:34.518206 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:34.518175    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:42:34.967939 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:34.967854    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:42:35.326546 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:35.326516    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg"]
Sep 10 19:42:35.327049 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:35.327022    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1301503-9300-43b1-913e-589ea3d31f71" containerName="istio-proxy"
Sep 10 19:42:35.327049 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:35.327044    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1301503-9300-43b1-913e-589ea3d31f71" containerName="istio-proxy"
Sep 10 19:42:35.327212 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:35.327055    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d216f39-a2df-4d2b-83e0-8d2bb0b34cf6" containerName="revoke-keys"
Sep 10 19:42:35.327212 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:35.327064    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d216f39-a2df-4d2b-83e0-8d2bb0b34cf6" containerName="revoke-keys"
Sep 10 19:42:35.327212 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:35.327107    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1301503-9300-43b1-913e-589ea3d31f71" containerName="istio-proxy"
Sep 10 19:42:35.327212 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:35.327118    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1301503-9300-43b1-913e-589ea3d31f71" containerName="istio-proxy"
Sep 10 19:42:35.327212 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:35.327211    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1301503-9300-43b1-913e-589ea3d31f71" containerName="istio-proxy"
Sep 10 19:42:35.327425 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:35.327222    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1301503-9300-43b1-913e-589ea3d31f71" containerName="istio-proxy"
Sep 10 19:42:35.327425 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:35.327232    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d216f39-a2df-4d2b-83e0-8d2bb0b34cf6" containerName="revoke-keys"
Sep 10 19:42:35.331916 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:35.331898    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg"
Sep 10 19:42:35.334498 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:35.334474    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-63ed30-openshift-default-dockercfg-fnbgx\""
Sep 10 19:42:35.345748 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:35.345712    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:42:35.346206 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:35.346183    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg"]
Sep 10 19:42:35.426687 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:35.426652    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d37ac8f3-e842-452e-8978-36cae1bf58e6-workload-certs\") pod \"e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg\" (UID: \"d37ac8f3-e842-452e-8978-36cae1bf58e6\") " pod="openshift-ingress/e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg"
Sep 10 19:42:35.426896 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:35.426703    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d37ac8f3-e842-452e-8978-36cae1bf58e6-istio-token\") pod \"e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg\" (UID: \"d37ac8f3-e842-452e-8978-36cae1bf58e6\") " pod="openshift-ingress/e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg"
Sep 10 19:42:35.426896 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:35.426787    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvfqk\" (UniqueName: \"kubernetes.io/projected/d37ac8f3-e842-452e-8978-36cae1bf58e6-kube-api-access-jvfqk\") pod \"e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg\" (UID: \"d37ac8f3-e842-452e-8978-36cae1bf58e6\") " pod="openshift-ingress/e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg"
Sep 10 19:42:35.426896 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:35.426850    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d37ac8f3-e842-452e-8978-36cae1bf58e6-workload-socket\") pod \"e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg\" (UID: \"d37ac8f3-e842-452e-8978-36cae1bf58e6\") " pod="openshift-ingress/e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg"
Sep 10 19:42:35.427067 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:35.426908    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d37ac8f3-e842-452e-8978-36cae1bf58e6-istio-data\") pod \"e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg\" (UID: \"d37ac8f3-e842-452e-8978-36cae1bf58e6\") " pod="openshift-ingress/e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg"
Sep 10 19:42:35.427067 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:35.426944    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d37ac8f3-e842-452e-8978-36cae1bf58e6-credential-socket\") pod \"e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg\" (UID: \"d37ac8f3-e842-452e-8978-36cae1bf58e6\") " pod="openshift-ingress/e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg"
Sep 10 19:42:35.427067 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:35.426982    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d37ac8f3-e842-452e-8978-36cae1bf58e6-istiod-ca-cert\") pod \"e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg\" (UID: \"d37ac8f3-e842-452e-8978-36cae1bf58e6\") " pod="openshift-ingress/e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg"
Sep 10 19:42:35.427067 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:35.427030    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d37ac8f3-e842-452e-8978-36cae1bf58e6-istio-envoy\") pod \"e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg\" (UID: \"d37ac8f3-e842-452e-8978-36cae1bf58e6\") " pod="openshift-ingress/e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg"
Sep 10 19:42:35.427221 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:35.427101    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d37ac8f3-e842-452e-8978-36cae1bf58e6-istio-podinfo\") pod \"e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg\" (UID: \"d37ac8f3-e842-452e-8978-36cae1bf58e6\") " pod="openshift-ingress/e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg"
Sep 10 19:42:35.458893 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:35.458856    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:42:35.528273 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:35.528240    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d37ac8f3-e842-452e-8978-36cae1bf58e6-workload-socket\") pod \"e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg\" (UID: \"d37ac8f3-e842-452e-8978-36cae1bf58e6\") " pod="openshift-ingress/e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg"
Sep 10 19:42:35.528750 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:35.528300    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d37ac8f3-e842-452e-8978-36cae1bf58e6-istio-data\") pod \"e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg\" (UID: \"d37ac8f3-e842-452e-8978-36cae1bf58e6\") " pod="openshift-ingress/e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg"
Sep 10 19:42:35.528750 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:35.528336    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d37ac8f3-e842-452e-8978-36cae1bf58e6-credential-socket\") pod \"e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg\" (UID: \"d37ac8f3-e842-452e-8978-36cae1bf58e6\") " pod="openshift-ingress/e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg"
Sep 10 19:42:35.528750 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:35.528369    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d37ac8f3-e842-452e-8978-36cae1bf58e6-istiod-ca-cert\") pod \"e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg\" (UID: \"d37ac8f3-e842-452e-8978-36cae1bf58e6\") " pod="openshift-ingress/e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg"
Sep 10 19:42:35.528750 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:35.528398    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d37ac8f3-e842-452e-8978-36cae1bf58e6-istio-envoy\") pod \"e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg\" (UID: \"d37ac8f3-e842-452e-8978-36cae1bf58e6\") " pod="openshift-ingress/e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg"
Sep 10 19:42:35.528750 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:35.528425    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d37ac8f3-e842-452e-8978-36cae1bf58e6-istio-podinfo\") pod \"e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg\" (UID: \"d37ac8f3-e842-452e-8978-36cae1bf58e6\") " pod="openshift-ingress/e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg"
Sep 10 19:42:35.528750 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:35.528472    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d37ac8f3-e842-452e-8978-36cae1bf58e6-workload-certs\") pod \"e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg\" (UID: \"d37ac8f3-e842-452e-8978-36cae1bf58e6\") " pod="openshift-ingress/e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg"
Sep 10 19:42:35.528750 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:35.528508    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d37ac8f3-e842-452e-8978-36cae1bf58e6-istio-token\") pod \"e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg\" (UID: \"d37ac8f3-e842-452e-8978-36cae1bf58e6\") " pod="openshift-ingress/e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg"
Sep 10 19:42:35.528750 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:35.528567    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvfqk\" (UniqueName: \"kubernetes.io/projected/d37ac8f3-e842-452e-8978-36cae1bf58e6-kube-api-access-jvfqk\") pod \"e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg\" (UID: \"d37ac8f3-e842-452e-8978-36cae1bf58e6\") " pod="openshift-ingress/e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg"
Sep 10 19:42:35.528750 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:35.528735    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d37ac8f3-e842-452e-8978-36cae1bf58e6-workload-socket\") pod \"e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg\" (UID: \"d37ac8f3-e842-452e-8978-36cae1bf58e6\") " pod="openshift-ingress/e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg"
Sep 10 19:42:35.529113 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:35.528778    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d37ac8f3-e842-452e-8978-36cae1bf58e6-credential-socket\") pod \"e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg\" (UID: \"d37ac8f3-e842-452e-8978-36cae1bf58e6\") " pod="openshift-ingress/e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg"
Sep 10 19:42:35.529113 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:35.528817    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d37ac8f3-e842-452e-8978-36cae1bf58e6-istio-data\") pod \"e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg\" (UID: \"d37ac8f3-e842-452e-8978-36cae1bf58e6\") " pod="openshift-ingress/e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg"
Sep 10 19:42:35.529243 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:35.529124    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d37ac8f3-e842-452e-8978-36cae1bf58e6-workload-certs\") pod \"e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg\" (UID: \"d37ac8f3-e842-452e-8978-36cae1bf58e6\") " pod="openshift-ingress/e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg"
Sep 10 19:42:35.529389 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:35.529365    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d37ac8f3-e842-452e-8978-36cae1bf58e6-istiod-ca-cert\") pod \"e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg\" (UID: \"d37ac8f3-e842-452e-8978-36cae1bf58e6\") " pod="openshift-ingress/e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg"
Sep 10 19:42:35.530740 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:35.530720    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d37ac8f3-e842-452e-8978-36cae1bf58e6-istio-envoy\") pod \"e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg\" (UID: \"d37ac8f3-e842-452e-8978-36cae1bf58e6\") " pod="openshift-ingress/e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg"
Sep 10 19:42:35.530981 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:35.530953    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d37ac8f3-e842-452e-8978-36cae1bf58e6-istio-podinfo\") pod \"e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg\" (UID: \"d37ac8f3-e842-452e-8978-36cae1bf58e6\") " pod="openshift-ingress/e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg"
Sep 10 19:42:35.538930 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:35.538908    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d37ac8f3-e842-452e-8978-36cae1bf58e6-istio-token\") pod \"e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg\" (UID: \"d37ac8f3-e842-452e-8978-36cae1bf58e6\") " pod="openshift-ingress/e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg"
Sep 10 19:42:35.539886 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:35.539867    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvfqk\" (UniqueName: \"kubernetes.io/projected/d37ac8f3-e842-452e-8978-36cae1bf58e6-kube-api-access-jvfqk\") pod \"e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg\" (UID: \"d37ac8f3-e842-452e-8978-36cae1bf58e6\") " pod="openshift-ingress/e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg"
Sep 10 19:42:35.645184 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:35.645086    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg"
Sep 10 19:42:35.926431 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:35.926404    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg"]
Sep 10 19:42:35.928495 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:42:35.928463    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd37ac8f3_e842_452e_8978_36cae1bf58e6.slice/crio-caae034f5d31b0939c0f26e41ca7a7fabab7c0a1bcf0f67c26dde220e2bdb8f3 WatchSource:0}: Error finding container caae034f5d31b0939c0f26e41ca7a7fabab7c0a1bcf0f67c26dde220e2bdb8f3: Status 404 returned error can't find the container with id caae034f5d31b0939c0f26e41ca7a7fabab7c0a1bcf0f67c26dde220e2bdb8f3
Sep 10 19:42:35.930912 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:35.930875    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:42:35.931005 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:35.930942    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:42:35.931005 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:35.930971    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:42:36.741927 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:36.741893    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg" event={"ID":"d37ac8f3-e842-452e-8978-36cae1bf58e6","Type":"ContainerStarted","Data":"23802ce943e702cf07511119b2890e683502a5e95a5d65bb9d8724018d936781"}
Sep 10 19:42:36.742427 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:36.741953    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg" event={"ID":"d37ac8f3-e842-452e-8978-36cae1bf58e6","Type":"ContainerStarted","Data":"caae034f5d31b0939c0f26e41ca7a7fabab7c0a1bcf0f67c26dde220e2bdb8f3"}
Sep 10 19:42:36.765676 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:36.765601    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg" podStartSLOduration=1.765585276 podStartE2EDuration="1.765585276s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:42:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:42:36.76379081 +0000 UTC m=+1612.364698046" watchObservedRunningTime="2026-09-10 19:42:36.765585276 +0000 UTC m=+1612.366492566"
Sep 10 19:42:37.646022 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:37.645933    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg"
Sep 10 19:42:37.651197 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:37.651174    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg"
Sep 10 19:42:37.747935 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:37.747901    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg"
Sep 10 19:42:37.749114 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:37.749092    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg"
Sep 10 19:42:37.882811 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:37.882778    2564 patch_prober.go:28] interesting pod/e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 19:42:37.882994 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:37.882839    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46" podUID="b4e00663-6fca-498b-926e-716be61dff37" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 19:42:38.680557 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:38.680526    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:42:38.776060 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:38.776021    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:42:39.120143 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:39.120118    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46"
Sep 10 19:42:39.267361 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:39.267332    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4e00663-6fca-498b-926e-716be61dff37-istio-data\") pod \"b4e00663-6fca-498b-926e-716be61dff37\" (UID: \"b4e00663-6fca-498b-926e-716be61dff37\") "
Sep 10 19:42:39.267542 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:39.267370    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4e00663-6fca-498b-926e-716be61dff37-istio-podinfo\") pod \"b4e00663-6fca-498b-926e-716be61dff37\" (UID: \"b4e00663-6fca-498b-926e-716be61dff37\") "
Sep 10 19:42:39.267542 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:39.267397    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4e00663-6fca-498b-926e-716be61dff37-istiod-ca-cert\") pod \"b4e00663-6fca-498b-926e-716be61dff37\" (UID: \"b4e00663-6fca-498b-926e-716be61dff37\") "
Sep 10 19:42:39.267542 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:39.267431    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4e00663-6fca-498b-926e-716be61dff37-workload-socket\") pod \"b4e00663-6fca-498b-926e-716be61dff37\" (UID: \"b4e00663-6fca-498b-926e-716be61dff37\") "
Sep 10 19:42:39.267542 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:39.267495    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4e00663-6fca-498b-926e-716be61dff37-istio-envoy\") pod \"b4e00663-6fca-498b-926e-716be61dff37\" (UID: \"b4e00663-6fca-498b-926e-716be61dff37\") "
Sep 10 19:42:39.267800 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:39.267552    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4e00663-6fca-498b-926e-716be61dff37-istio-token\") pod \"b4e00663-6fca-498b-926e-716be61dff37\" (UID: \"b4e00663-6fca-498b-926e-716be61dff37\") "
Sep 10 19:42:39.267800 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:39.267580    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vdwc2\" (UniqueName: \"kubernetes.io/projected/b4e00663-6fca-498b-926e-716be61dff37-kube-api-access-vdwc2\") pod \"b4e00663-6fca-498b-926e-716be61dff37\" (UID: \"b4e00663-6fca-498b-926e-716be61dff37\") "
Sep 10 19:42:39.267800 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:39.267659    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4e00663-6fca-498b-926e-716be61dff37-credential-socket\") pod \"b4e00663-6fca-498b-926e-716be61dff37\" (UID: \"b4e00663-6fca-498b-926e-716be61dff37\") "
Sep 10 19:42:39.267800 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:39.267717    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4e00663-6fca-498b-926e-716be61dff37-workload-certs\") pod \"b4e00663-6fca-498b-926e-716be61dff37\" (UID: \"b4e00663-6fca-498b-926e-716be61dff37\") "
Sep 10 19:42:39.267800 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:39.267722    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4e00663-6fca-498b-926e-716be61dff37-istio-data" (OuterVolumeSpecName: "istio-data") pod "b4e00663-6fca-498b-926e-716be61dff37" (UID: "b4e00663-6fca-498b-926e-716be61dff37"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:42:39.268072 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:39.267958    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4e00663-6fca-498b-926e-716be61dff37-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b4e00663-6fca-498b-926e-716be61dff37" (UID: "b4e00663-6fca-498b-926e-716be61dff37"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:42:39.268126 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:39.268118    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4e00663-6fca-498b-926e-716be61dff37-istio-data\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:42:39.268183 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:39.268136    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4e00663-6fca-498b-926e-716be61dff37-credential-socket\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:42:39.268265 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:39.268232    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4e00663-6fca-498b-926e-716be61dff37-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b4e00663-6fca-498b-926e-716be61dff37" (UID: "b4e00663-6fca-498b-926e-716be61dff37"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:42:39.268436 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:39.268413    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4e00663-6fca-498b-926e-716be61dff37-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b4e00663-6fca-498b-926e-716be61dff37" (UID: "b4e00663-6fca-498b-926e-716be61dff37"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:42:39.268724 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:39.268698    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4e00663-6fca-498b-926e-716be61dff37-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b4e00663-6fca-498b-926e-716be61dff37" (UID: "b4e00663-6fca-498b-926e-716be61dff37"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:42:39.269839 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:39.269819    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b4e00663-6fca-498b-926e-716be61dff37-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b4e00663-6fca-498b-926e-716be61dff37" (UID: "b4e00663-6fca-498b-926e-716be61dff37"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 19:42:39.270228 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:39.270208    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4e00663-6fca-498b-926e-716be61dff37-kube-api-access-vdwc2" (OuterVolumeSpecName: "kube-api-access-vdwc2") pod "b4e00663-6fca-498b-926e-716be61dff37" (UID: "b4e00663-6fca-498b-926e-716be61dff37"). InnerVolumeSpecName "kube-api-access-vdwc2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:42:39.270414 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:39.270396    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4e00663-6fca-498b-926e-716be61dff37-istio-token" (OuterVolumeSpecName: "istio-token") pod "b4e00663-6fca-498b-926e-716be61dff37" (UID: "b4e00663-6fca-498b-926e-716be61dff37"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:42:39.270510 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:39.270407    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4e00663-6fca-498b-926e-716be61dff37-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b4e00663-6fca-498b-926e-716be61dff37" (UID: "b4e00663-6fca-498b-926e-716be61dff37"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:42:39.368929 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:39.368894    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4e00663-6fca-498b-926e-716be61dff37-istio-envoy\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:42:39.368929 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:39.368924    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4e00663-6fca-498b-926e-716be61dff37-istio-token\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:42:39.368929 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:39.368934    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vdwc2\" (UniqueName: \"kubernetes.io/projected/b4e00663-6fca-498b-926e-716be61dff37-kube-api-access-vdwc2\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:42:39.369174 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:39.368943    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4e00663-6fca-498b-926e-716be61dff37-workload-certs\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:42:39.369174 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:39.368952    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4e00663-6fca-498b-926e-716be61dff37-istio-podinfo\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:42:39.369174 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:39.368959    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4e00663-6fca-498b-926e-716be61dff37-istiod-ca-cert\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:42:39.369174 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:39.368969    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4e00663-6fca-498b-926e-716be61dff37-workload-socket\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:42:39.772114 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:39.772071    2564 generic.go:356] "Generic (PLEG): container finished" podID="b4e00663-6fca-498b-926e-716be61dff37" containerID="e984a042c1f2460152cf4e1a77f41af578f86cded44cc4f353ec087ba863aad1" exitCode=0
Sep 10 19:42:39.772303 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:39.772128    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46"
Sep 10 19:42:39.772303 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:39.772147    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46" event={"ID":"b4e00663-6fca-498b-926e-716be61dff37","Type":"ContainerDied","Data":"e984a042c1f2460152cf4e1a77f41af578f86cded44cc4f353ec087ba863aad1"}
Sep 10 19:42:39.772303 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:39.772246    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46" event={"ID":"b4e00663-6fca-498b-926e-716be61dff37","Type":"ContainerDied","Data":"b52bb21f81d7a4353e55bfed0e62d22b20438816d6c56726a5686fc9403fcad6"}
Sep 10 19:42:39.772303 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:39.772274    2564 scope.go:117] "RemoveContainer" containerID="e984a042c1f2460152cf4e1a77f41af578f86cded44cc4f353ec087ba863aad1"
Sep 10 19:42:39.785845 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:39.785824    2564 scope.go:117] "RemoveContainer" containerID="a427ec5de8d06f85ca8c4d8bfc565986c0d431c6499242b2fde078b4232e6c4e"
Sep 10 19:42:39.796077 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:39.796060    2564 scope.go:117] "RemoveContainer" containerID="e984a042c1f2460152cf4e1a77f41af578f86cded44cc4f353ec087ba863aad1"
Sep 10 19:42:39.796369 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:42:39.796348    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e984a042c1f2460152cf4e1a77f41af578f86cded44cc4f353ec087ba863aad1\": container with ID starting with e984a042c1f2460152cf4e1a77f41af578f86cded44cc4f353ec087ba863aad1 not found: ID does not exist" containerID="e984a042c1f2460152cf4e1a77f41af578f86cded44cc4f353ec087ba863aad1"
Sep 10 19:42:39.796413 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:39.796380    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e984a042c1f2460152cf4e1a77f41af578f86cded44cc4f353ec087ba863aad1"} err="failed to get container status \"e984a042c1f2460152cf4e1a77f41af578f86cded44cc4f353ec087ba863aad1\": rpc error: code = NotFound desc = could not find container \"e984a042c1f2460152cf4e1a77f41af578f86cded44cc4f353ec087ba863aad1\": container with ID starting with e984a042c1f2460152cf4e1a77f41af578f86cded44cc4f353ec087ba863aad1 not found: ID does not exist"
Sep 10 19:42:39.796413 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:39.796400    2564 scope.go:117] "RemoveContainer" containerID="a427ec5de8d06f85ca8c4d8bfc565986c0d431c6499242b2fde078b4232e6c4e"
Sep 10 19:42:39.796719 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:42:39.796693    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a427ec5de8d06f85ca8c4d8bfc565986c0d431c6499242b2fde078b4232e6c4e\": container with ID starting with a427ec5de8d06f85ca8c4d8bfc565986c0d431c6499242b2fde078b4232e6c4e not found: ID does not exist" containerID="a427ec5de8d06f85ca8c4d8bfc565986c0d431c6499242b2fde078b4232e6c4e"
Sep 10 19:42:39.796815 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:39.796724    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a427ec5de8d06f85ca8c4d8bfc565986c0d431c6499242b2fde078b4232e6c4e"} err="failed to get container status \"a427ec5de8d06f85ca8c4d8bfc565986c0d431c6499242b2fde078b4232e6c4e\": rpc error: code = NotFound desc = could not find container \"a427ec5de8d06f85ca8c4d8bfc565986c0d431c6499242b2fde078b4232e6c4e\": container with ID starting with a427ec5de8d06f85ca8c4d8bfc565986c0d431c6499242b2fde078b4232e6c4e not found: ID does not exist"
Sep 10 19:42:39.803767 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:39.803717    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46"]
Sep 10 19:42:39.815365 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:39.815329    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-7bfb5d-openshift-default-7775577c76-chj46"]
Sep 10 19:42:39.852107 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:39.852068    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:42:39.907864 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:39.907829    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:42:40.008608 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:40.008574    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:42:40.345699 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:40.345663    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:42:40.407951 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:40.407914    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:42:40.488859 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:40.488826    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-63ed30-794f98b679-f8v4z"]
Sep 10 19:42:40.489264 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:40.489251    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4e00663-6fca-498b-926e-716be61dff37" containerName="istio-proxy"
Sep 10 19:42:40.489321 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:40.489268    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4e00663-6fca-498b-926e-716be61dff37" containerName="istio-proxy"
Sep 10 19:42:40.489321 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:40.489281    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4e00663-6fca-498b-926e-716be61dff37" containerName="istio-proxy"
Sep 10 19:42:40.489321 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:40.489287    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4e00663-6fca-498b-926e-716be61dff37" containerName="istio-proxy"
Sep 10 19:42:40.489409 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:40.489368    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4e00663-6fca-498b-926e-716be61dff37" containerName="istio-proxy"
Sep 10 19:42:40.494029 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:40.494012    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-63ed30-794f98b679-f8v4z"
Sep 10 19:42:40.496223 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:40.496204    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-63ed30-dockercfg-mthdz\""
Sep 10 19:42:40.496223 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:40.496218    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-63ed30\""
Sep 10 19:42:40.501739 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:40.501691    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-63ed30-794f98b679-f8v4z"]
Sep 10 19:42:40.536933 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:40.536899    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-63ed30-684b99ccb6-vgplv"]
Sep 10 19:42:40.537391 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:40.537379    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4e00663-6fca-498b-926e-716be61dff37" containerName="istio-proxy"
Sep 10 19:42:40.540388 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:40.540373    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-63ed30-684b99ccb6-vgplv"
Sep 10 19:42:40.555222 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:40.555196    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-63ed30-684b99ccb6-vgplv"]
Sep 10 19:42:40.684079 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:40.683992    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f85c24d6-df13-4f97-afdc-767ccf51a2a5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-63ed30-684b99ccb6-vgplv\" (UID: \"f85c24d6-df13-4f97-afdc-767ccf51a2a5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-63ed30-684b99ccb6-vgplv"
Sep 10 19:42:40.684079 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:40.684036    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-np6fk\" (UniqueName: \"kubernetes.io/projected/02704747-24a5-4961-abac-7651fadd04fb-kube-api-access-np6fk\") pod \"payload-processing-e2e-ipp-a-63ed30-794f98b679-f8v4z\" (UID: \"02704747-24a5-4961-abac-7651fadd04fb\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-63ed30-794f98b679-f8v4z"
Sep 10 19:42:40.684275 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:40.684105    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/02704747-24a5-4961-abac-7651fadd04fb-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-63ed30-794f98b679-f8v4z\" (UID: \"02704747-24a5-4961-abac-7651fadd04fb\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-63ed30-794f98b679-f8v4z"
Sep 10 19:42:40.684275 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:40.684187    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htsml\" (UniqueName: \"kubernetes.io/projected/f85c24d6-df13-4f97-afdc-767ccf51a2a5-kube-api-access-htsml\") pod \"payload-pre-processing-e2e-ipp-a-63ed30-684b99ccb6-vgplv\" (UID: \"f85c24d6-df13-4f97-afdc-767ccf51a2a5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-63ed30-684b99ccb6-vgplv"
Sep 10 19:42:40.785739 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:40.785700    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htsml\" (UniqueName: \"kubernetes.io/projected/f85c24d6-df13-4f97-afdc-767ccf51a2a5-kube-api-access-htsml\") pod \"payload-pre-processing-e2e-ipp-a-63ed30-684b99ccb6-vgplv\" (UID: \"f85c24d6-df13-4f97-afdc-767ccf51a2a5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-63ed30-684b99ccb6-vgplv"
Sep 10 19:42:40.785923 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:40.785764    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f85c24d6-df13-4f97-afdc-767ccf51a2a5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-63ed30-684b99ccb6-vgplv\" (UID: \"f85c24d6-df13-4f97-afdc-767ccf51a2a5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-63ed30-684b99ccb6-vgplv"
Sep 10 19:42:40.785923 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:40.785791    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-np6fk\" (UniqueName: \"kubernetes.io/projected/02704747-24a5-4961-abac-7651fadd04fb-kube-api-access-np6fk\") pod \"payload-processing-e2e-ipp-a-63ed30-794f98b679-f8v4z\" (UID: \"02704747-24a5-4961-abac-7651fadd04fb\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-63ed30-794f98b679-f8v4z"
Sep 10 19:42:40.785923 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:40.785827    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/02704747-24a5-4961-abac-7651fadd04fb-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-63ed30-794f98b679-f8v4z\" (UID: \"02704747-24a5-4961-abac-7651fadd04fb\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-63ed30-794f98b679-f8v4z"
Sep 10 19:42:40.786511 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:40.786489    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f85c24d6-df13-4f97-afdc-767ccf51a2a5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-63ed30-684b99ccb6-vgplv\" (UID: \"f85c24d6-df13-4f97-afdc-767ccf51a2a5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-63ed30-684b99ccb6-vgplv"
Sep 10 19:42:40.786551 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:40.786521    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/02704747-24a5-4961-abac-7651fadd04fb-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-63ed30-794f98b679-f8v4z\" (UID: \"02704747-24a5-4961-abac-7651fadd04fb\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-63ed30-794f98b679-f8v4z"
Sep 10 19:42:40.794860 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:40.794834    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-np6fk\" (UniqueName: \"kubernetes.io/projected/02704747-24a5-4961-abac-7651fadd04fb-kube-api-access-np6fk\") pod \"payload-processing-e2e-ipp-a-63ed30-794f98b679-f8v4z\" (UID: \"02704747-24a5-4961-abac-7651fadd04fb\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-63ed30-794f98b679-f8v4z"
Sep 10 19:42:40.794964 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:40.794900    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-htsml\" (UniqueName: \"kubernetes.io/projected/f85c24d6-df13-4f97-afdc-767ccf51a2a5-kube-api-access-htsml\") pod \"payload-pre-processing-e2e-ipp-a-63ed30-684b99ccb6-vgplv\" (UID: \"f85c24d6-df13-4f97-afdc-767ccf51a2a5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-63ed30-684b99ccb6-vgplv"
Sep 10 19:42:40.805837 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:40.805812    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-63ed30-794f98b679-f8v4z"
Sep 10 19:42:40.850891 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:40.850853    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-63ed30-684b99ccb6-vgplv"
Sep 10 19:42:40.949973 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:40.949892    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4e00663-6fca-498b-926e-716be61dff37" path="/var/lib/kubelet/pods/b4e00663-6fca-498b-926e-716be61dff37/volumes"
Sep 10 19:42:41.083826 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:41.083793    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-63ed30-794f98b679-f8v4z"]
Sep 10 19:42:41.085482 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:42:41.085455    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02704747_24a5_4961_abac_7651fadd04fb.slice/crio-895edac29eef4f0dd0e7b9e6f9fec7b44e4891783a9b25bee5ab03f2cf0662e7 WatchSource:0}: Error finding container 895edac29eef4f0dd0e7b9e6f9fec7b44e4891783a9b25bee5ab03f2cf0662e7: Status 404 returned error can't find the container with id 895edac29eef4f0dd0e7b9e6f9fec7b44e4891783a9b25bee5ab03f2cf0662e7
Sep 10 19:42:41.147335 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:41.147309    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-63ed30-684b99ccb6-vgplv"]
Sep 10 19:42:41.149974 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:42:41.149936    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf85c24d6_df13_4f97_afdc_767ccf51a2a5.slice/crio-31011cdd519be2363710319b54b1cb34ef35baa9874a6f318705001f8e19285d WatchSource:0}: Error finding container 31011cdd519be2363710319b54b1cb34ef35baa9874a6f318705001f8e19285d: Status 404 returned error can't find the container with id 31011cdd519be2363710319b54b1cb34ef35baa9874a6f318705001f8e19285d
Sep 10 19:42:41.467010 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:41.466916    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:42:41.788200 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:41.788166    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-63ed30-794f98b679-f8v4z" event={"ID":"02704747-24a5-4961-abac-7651fadd04fb","Type":"ContainerStarted","Data":"e79feaa0aea0e5cb31ad55d1cd653c8bf014ccdf4162afa3491f281a848c70a3"}
Sep 10 19:42:41.788717 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:41.788225    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-63ed30-794f98b679-f8v4z" event={"ID":"02704747-24a5-4961-abac-7651fadd04fb","Type":"ContainerStarted","Data":"895edac29eef4f0dd0e7b9e6f9fec7b44e4891783a9b25bee5ab03f2cf0662e7"}
Sep 10 19:42:41.788717 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:41.788307    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-63ed30-794f98b679-f8v4z"
Sep 10 19:42:41.789969 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:41.789942    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-63ed30-684b99ccb6-vgplv" event={"ID":"f85c24d6-df13-4f97-afdc-767ccf51a2a5","Type":"ContainerStarted","Data":"fa335220bda9a8133d8a068ec413ef71457d44aa955651c57df399b0e8c87c26"}
Sep 10 19:42:41.790089 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:41.789988    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-63ed30-684b99ccb6-vgplv" event={"ID":"f85c24d6-df13-4f97-afdc-767ccf51a2a5","Type":"ContainerStarted","Data":"31011cdd519be2363710319b54b1cb34ef35baa9874a6f318705001f8e19285d"}
Sep 10 19:42:41.790089 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:41.790013    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-63ed30-684b99ccb6-vgplv"
Sep 10 19:42:41.807067 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:41.807018    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-63ed30-794f98b679-f8v4z" podStartSLOduration=1.8070017539999998 podStartE2EDuration="1.807001754s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:42:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:42:41.806148772 +0000 UTC m=+1617.407056009" watchObservedRunningTime="2026-09-10 19:42:41.807001754 +0000 UTC m=+1617.407908993"
Sep 10 19:42:41.823096 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:41.823036    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-63ed30-684b99ccb6-vgplv" podStartSLOduration=1.823021853 podStartE2EDuration="1.823021853s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:42:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:42:41.822536385 +0000 UTC m=+1617.423443626" watchObservedRunningTime="2026-09-10 19:42:41.823021853 +0000 UTC m=+1617.423929094"
Sep 10 19:42:52.801555 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:52.801525    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-63ed30-684b99ccb6-vgplv"
Sep 10 19:42:52.803899 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:52.801677    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-63ed30-794f98b679-f8v4z"
Sep 10 19:42:52.900278 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:52.900242    2564 patch_prober.go:28] interesting pod/e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": read tcp 10.133.0.2:48298->10.133.0.83:15021: read: connection reset by peer" start-of-body=
Sep 10 19:42:52.900437 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:52.900334    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg" podUID="d37ac8f3-e842-452e-8978-36cae1bf58e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": read tcp 10.133.0.2:48298->10.133.0.83:15021: read: connection reset by peer"
Sep 10 19:42:53.865108 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:53.865074    2564 generic.go:356] "Generic (PLEG): container finished" podID="d37ac8f3-e842-452e-8978-36cae1bf58e6" containerID="23802ce943e702cf07511119b2890e683502a5e95a5d65bb9d8724018d936781" exitCode=0
Sep 10 19:42:53.865503 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:53.865140    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg" event={"ID":"d37ac8f3-e842-452e-8978-36cae1bf58e6","Type":"ContainerDied","Data":"23802ce943e702cf07511119b2890e683502a5e95a5d65bb9d8724018d936781"}
Sep 10 19:42:53.865558 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:53.865547    2564 scope.go:117] "RemoveContainer" containerID="23802ce943e702cf07511119b2890e683502a5e95a5d65bb9d8724018d936781"
Sep 10 19:42:53.866674 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:53.866644    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:42:53.866737 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:53.866722    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:42:53.866779 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:53.866761    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:42:54.895082 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:54.895041    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg" event={"ID":"d37ac8f3-e842-452e-8978-36cae1bf58e6","Type":"ContainerStarted","Data":"c3bf4b5c44d1f71e56c3098ff55a3bc26428089aa1a66a6bb18f654a64dd15b3"}
Sep 10 19:42:55.645956 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:55.645914    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg"
Sep 10 19:42:55.646191 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:55.646166    2564 patch_prober.go:28] interesting pod/e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body=
Sep 10 19:42:55.646245 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:55.646227    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg" podUID="d37ac8f3-e842-452e-8978-36cae1bf58e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused"
Sep 10 19:42:56.646389 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:56.646350    2564 patch_prober.go:28] interesting pod/e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body=
Sep 10 19:42:56.646805 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:56.646422    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg" podUID="d37ac8f3-e842-452e-8978-36cae1bf58e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused"
Sep 10 19:42:57.645989 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:57.645950    2564 patch_prober.go:28] interesting pod/e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body=
Sep 10 19:42:57.646180 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:57.646016    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg" podUID="d37ac8f3-e842-452e-8978-36cae1bf58e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused"
Sep 10 19:42:58.646166 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:58.646123    2564 patch_prober.go:28] interesting pod/e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body=
Sep 10 19:42:58.646680 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:58.646188    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg" podUID="d37ac8f3-e842-452e-8978-36cae1bf58e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused"
Sep 10 19:42:59.646423 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:59.646385    2564 patch_prober.go:28] interesting pod/e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body=
Sep 10 19:42:59.646844 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:42:59.646442    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg" podUID="d37ac8f3-e842-452e-8978-36cae1bf58e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused"
Sep 10 19:43:00.649751 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:00.649710    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg"
Sep 10 19:43:00.650195 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:00.649980    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg"
Sep 10 19:43:00.650708 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:00.650689    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg"
Sep 10 19:43:03.741310 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:03.741274    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:43:04.452811 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:04.452774    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:43:04.728880 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:04.728781    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-544dea-openshift-default-85985454b-qflgf"]
Sep 10 19:43:04.732968 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:04.732939    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-544dea-openshift-default-85985454b-qflgf"
Sep 10 19:43:04.735400 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:04.735377    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-544dea-openshift-default-dockercfg-7xdxt\""
Sep 10 19:43:04.746146 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:04.746118    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-544dea-openshift-default-85985454b-qflgf"]
Sep 10 19:43:04.754011 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:04.753978    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:43:04.842532 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:04.842492    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7820d541-2f9a-41a7-bf74-b0e8b4161d56-istiod-ca-cert\") pod \"e2e-ipp-b-544dea-openshift-default-85985454b-qflgf\" (UID: \"7820d541-2f9a-41a7-bf74-b0e8b4161d56\") " pod="openshift-ingress/e2e-ipp-b-544dea-openshift-default-85985454b-qflgf"
Sep 10 19:43:04.842760 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:04.842646    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7820d541-2f9a-41a7-bf74-b0e8b4161d56-workload-certs\") pod \"e2e-ipp-b-544dea-openshift-default-85985454b-qflgf\" (UID: \"7820d541-2f9a-41a7-bf74-b0e8b4161d56\") " pod="openshift-ingress/e2e-ipp-b-544dea-openshift-default-85985454b-qflgf"
Sep 10 19:43:04.842760 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:04.842708    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7820d541-2f9a-41a7-bf74-b0e8b4161d56-istio-envoy\") pod \"e2e-ipp-b-544dea-openshift-default-85985454b-qflgf\" (UID: \"7820d541-2f9a-41a7-bf74-b0e8b4161d56\") " pod="openshift-ingress/e2e-ipp-b-544dea-openshift-default-85985454b-qflgf"
Sep 10 19:43:04.842900 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:04.842768    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7820d541-2f9a-41a7-bf74-b0e8b4161d56-istio-podinfo\") pod \"e2e-ipp-b-544dea-openshift-default-85985454b-qflgf\" (UID: \"7820d541-2f9a-41a7-bf74-b0e8b4161d56\") " pod="openshift-ingress/e2e-ipp-b-544dea-openshift-default-85985454b-qflgf"
Sep 10 19:43:04.842900 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:04.842823    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7820d541-2f9a-41a7-bf74-b0e8b4161d56-workload-socket\") pod \"e2e-ipp-b-544dea-openshift-default-85985454b-qflgf\" (UID: \"7820d541-2f9a-41a7-bf74-b0e8b4161d56\") " pod="openshift-ingress/e2e-ipp-b-544dea-openshift-default-85985454b-qflgf"
Sep 10 19:43:04.842900 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:04.842851    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7820d541-2f9a-41a7-bf74-b0e8b4161d56-credential-socket\") pod \"e2e-ipp-b-544dea-openshift-default-85985454b-qflgf\" (UID: \"7820d541-2f9a-41a7-bf74-b0e8b4161d56\") " pod="openshift-ingress/e2e-ipp-b-544dea-openshift-default-85985454b-qflgf"
Sep 10 19:43:04.842900 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:04.842875    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7820d541-2f9a-41a7-bf74-b0e8b4161d56-istio-data\") pod \"e2e-ipp-b-544dea-openshift-default-85985454b-qflgf\" (UID: \"7820d541-2f9a-41a7-bf74-b0e8b4161d56\") " pod="openshift-ingress/e2e-ipp-b-544dea-openshift-default-85985454b-qflgf"
Sep 10 19:43:04.843098 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:04.842987    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7820d541-2f9a-41a7-bf74-b0e8b4161d56-istio-token\") pod \"e2e-ipp-b-544dea-openshift-default-85985454b-qflgf\" (UID: \"7820d541-2f9a-41a7-bf74-b0e8b4161d56\") " pod="openshift-ingress/e2e-ipp-b-544dea-openshift-default-85985454b-qflgf"
Sep 10 19:43:04.843098 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:04.843041    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccs96\" (UniqueName: \"kubernetes.io/projected/7820d541-2f9a-41a7-bf74-b0e8b4161d56-kube-api-access-ccs96\") pod \"e2e-ipp-b-544dea-openshift-default-85985454b-qflgf\" (UID: \"7820d541-2f9a-41a7-bf74-b0e8b4161d56\") " pod="openshift-ingress/e2e-ipp-b-544dea-openshift-default-85985454b-qflgf"
Sep 10 19:43:04.944473 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:04.944435    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7820d541-2f9a-41a7-bf74-b0e8b4161d56-workload-certs\") pod \"e2e-ipp-b-544dea-openshift-default-85985454b-qflgf\" (UID: \"7820d541-2f9a-41a7-bf74-b0e8b4161d56\") " pod="openshift-ingress/e2e-ipp-b-544dea-openshift-default-85985454b-qflgf"
Sep 10 19:43:04.944702 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:04.944494    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7820d541-2f9a-41a7-bf74-b0e8b4161d56-istio-envoy\") pod \"e2e-ipp-b-544dea-openshift-default-85985454b-qflgf\" (UID: \"7820d541-2f9a-41a7-bf74-b0e8b4161d56\") " pod="openshift-ingress/e2e-ipp-b-544dea-openshift-default-85985454b-qflgf"
Sep 10 19:43:04.944702 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:04.944540    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7820d541-2f9a-41a7-bf74-b0e8b4161d56-istio-podinfo\") pod \"e2e-ipp-b-544dea-openshift-default-85985454b-qflgf\" (UID: \"7820d541-2f9a-41a7-bf74-b0e8b4161d56\") " pod="openshift-ingress/e2e-ipp-b-544dea-openshift-default-85985454b-qflgf"
Sep 10 19:43:04.944702 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:04.944585    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7820d541-2f9a-41a7-bf74-b0e8b4161d56-workload-socket\") pod \"e2e-ipp-b-544dea-openshift-default-85985454b-qflgf\" (UID: \"7820d541-2f9a-41a7-bf74-b0e8b4161d56\") " pod="openshift-ingress/e2e-ipp-b-544dea-openshift-default-85985454b-qflgf"
Sep 10 19:43:04.944702 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:04.944610    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7820d541-2f9a-41a7-bf74-b0e8b4161d56-credential-socket\") pod \"e2e-ipp-b-544dea-openshift-default-85985454b-qflgf\" (UID: \"7820d541-2f9a-41a7-bf74-b0e8b4161d56\") " pod="openshift-ingress/e2e-ipp-b-544dea-openshift-default-85985454b-qflgf"
Sep 10 19:43:04.944702 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:04.944664    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7820d541-2f9a-41a7-bf74-b0e8b4161d56-istio-data\") pod \"e2e-ipp-b-544dea-openshift-default-85985454b-qflgf\" (UID: \"7820d541-2f9a-41a7-bf74-b0e8b4161d56\") " pod="openshift-ingress/e2e-ipp-b-544dea-openshift-default-85985454b-qflgf"
Sep 10 19:43:04.945003 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:04.944716    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7820d541-2f9a-41a7-bf74-b0e8b4161d56-istio-token\") pod \"e2e-ipp-b-544dea-openshift-default-85985454b-qflgf\" (UID: \"7820d541-2f9a-41a7-bf74-b0e8b4161d56\") " pod="openshift-ingress/e2e-ipp-b-544dea-openshift-default-85985454b-qflgf"
Sep 10 19:43:04.945003 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:04.944764    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccs96\" (UniqueName: \"kubernetes.io/projected/7820d541-2f9a-41a7-bf74-b0e8b4161d56-kube-api-access-ccs96\") pod \"e2e-ipp-b-544dea-openshift-default-85985454b-qflgf\" (UID: \"7820d541-2f9a-41a7-bf74-b0e8b4161d56\") " pod="openshift-ingress/e2e-ipp-b-544dea-openshift-default-85985454b-qflgf"
Sep 10 19:43:04.945003 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:04.944829    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7820d541-2f9a-41a7-bf74-b0e8b4161d56-istiod-ca-cert\") pod \"e2e-ipp-b-544dea-openshift-default-85985454b-qflgf\" (UID: \"7820d541-2f9a-41a7-bf74-b0e8b4161d56\") " pod="openshift-ingress/e2e-ipp-b-544dea-openshift-default-85985454b-qflgf"
Sep 10 19:43:04.945003 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:04.944947    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7820d541-2f9a-41a7-bf74-b0e8b4161d56-workload-certs\") pod \"e2e-ipp-b-544dea-openshift-default-85985454b-qflgf\" (UID: \"7820d541-2f9a-41a7-bf74-b0e8b4161d56\") " pod="openshift-ingress/e2e-ipp-b-544dea-openshift-default-85985454b-qflgf"
Sep 10 19:43:04.945228 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:04.945201    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7820d541-2f9a-41a7-bf74-b0e8b4161d56-istio-data\") pod \"e2e-ipp-b-544dea-openshift-default-85985454b-qflgf\" (UID: \"7820d541-2f9a-41a7-bf74-b0e8b4161d56\") " pod="openshift-ingress/e2e-ipp-b-544dea-openshift-default-85985454b-qflgf"
Sep 10 19:43:04.945450 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:04.945427    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7820d541-2f9a-41a7-bf74-b0e8b4161d56-workload-socket\") pod \"e2e-ipp-b-544dea-openshift-default-85985454b-qflgf\" (UID: \"7820d541-2f9a-41a7-bf74-b0e8b4161d56\") " pod="openshift-ingress/e2e-ipp-b-544dea-openshift-default-85985454b-qflgf"
Sep 10 19:43:04.945662 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:04.945635    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7820d541-2f9a-41a7-bf74-b0e8b4161d56-istiod-ca-cert\") pod \"e2e-ipp-b-544dea-openshift-default-85985454b-qflgf\" (UID: \"7820d541-2f9a-41a7-bf74-b0e8b4161d56\") " pod="openshift-ingress/e2e-ipp-b-544dea-openshift-default-85985454b-qflgf"
Sep 10 19:43:04.945978 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:04.945952    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7820d541-2f9a-41a7-bf74-b0e8b4161d56-credential-socket\") pod \"e2e-ipp-b-544dea-openshift-default-85985454b-qflgf\" (UID: \"7820d541-2f9a-41a7-bf74-b0e8b4161d56\") " pod="openshift-ingress/e2e-ipp-b-544dea-openshift-default-85985454b-qflgf"
Sep 10 19:43:04.947509 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:04.947487    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7820d541-2f9a-41a7-bf74-b0e8b4161d56-istio-envoy\") pod \"e2e-ipp-b-544dea-openshift-default-85985454b-qflgf\" (UID: \"7820d541-2f9a-41a7-bf74-b0e8b4161d56\") " pod="openshift-ingress/e2e-ipp-b-544dea-openshift-default-85985454b-qflgf"
Sep 10 19:43:04.947680 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:04.947657    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7820d541-2f9a-41a7-bf74-b0e8b4161d56-istio-podinfo\") pod \"e2e-ipp-b-544dea-openshift-default-85985454b-qflgf\" (UID: \"7820d541-2f9a-41a7-bf74-b0e8b4161d56\") " pod="openshift-ingress/e2e-ipp-b-544dea-openshift-default-85985454b-qflgf"
Sep 10 19:43:04.954918 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:04.954891    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7820d541-2f9a-41a7-bf74-b0e8b4161d56-istio-token\") pod \"e2e-ipp-b-544dea-openshift-default-85985454b-qflgf\" (UID: \"7820d541-2f9a-41a7-bf74-b0e8b4161d56\") " pod="openshift-ingress/e2e-ipp-b-544dea-openshift-default-85985454b-qflgf"
Sep 10 19:43:04.955046 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:04.955028    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccs96\" (UniqueName: \"kubernetes.io/projected/7820d541-2f9a-41a7-bf74-b0e8b4161d56-kube-api-access-ccs96\") pod \"e2e-ipp-b-544dea-openshift-default-85985454b-qflgf\" (UID: \"7820d541-2f9a-41a7-bf74-b0e8b4161d56\") " pod="openshift-ingress/e2e-ipp-b-544dea-openshift-default-85985454b-qflgf"
Sep 10 19:43:05.043863 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:05.043824    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-544dea-openshift-default-85985454b-qflgf"
Sep 10 19:43:05.157074 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:05.157015    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:43:05.324394 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:05.324366    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-544dea-openshift-default-85985454b-qflgf"]
Sep 10 19:43:05.327201 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:43:05.327164    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7820d541_2f9a_41a7_bf74_b0e8b4161d56.slice/crio-6f72410c78079536f45bc33f7e04784917fdff34f4318409727102bffcf122c0 WatchSource:0}: Error finding container 6f72410c78079536f45bc33f7e04784917fdff34f4318409727102bffcf122c0: Status 404 returned error can't find the container with id 6f72410c78079536f45bc33f7e04784917fdff34f4318409727102bffcf122c0
Sep 10 19:43:05.329456 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:05.329421    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:43:05.329557 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:05.329498    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:43:05.329557 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:05.329531    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:43:05.959345 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:05.959309    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-544dea-openshift-default-85985454b-qflgf" event={"ID":"7820d541-2f9a-41a7-bf74-b0e8b4161d56","Type":"ContainerStarted","Data":"2f976196109681dfe049e4ddb3dc83783db69106d7b959f5116ed6579e328824"}
Sep 10 19:43:05.959765 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:05.959390    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-544dea-openshift-default-85985454b-qflgf" event={"ID":"7820d541-2f9a-41a7-bf74-b0e8b4161d56","Type":"ContainerStarted","Data":"6f72410c78079536f45bc33f7e04784917fdff34f4318409727102bffcf122c0"}
Sep 10 19:43:05.992463 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:05.992406    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-544dea-openshift-default-85985454b-qflgf" podStartSLOduration=1.992390936 podStartE2EDuration="1.992390936s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:43:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:43:05.991393363 +0000 UTC m=+1641.592300618" watchObservedRunningTime="2026-09-10 19:43:05.992390936 +0000 UTC m=+1641.593298176"
Sep 10 19:43:06.044975 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:06.044943    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-544dea-openshift-default-85985454b-qflgf"
Sep 10 19:43:06.050065 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:06.050036    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-544dea-openshift-default-85985454b-qflgf"
Sep 10 19:43:06.964704 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:06.964671    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-544dea-openshift-default-85985454b-qflgf"
Sep 10 19:43:06.965994 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:06.965968    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-544dea-openshift-default-85985454b-qflgf"
Sep 10 19:43:07.932132 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:07.932084    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:43:08.026697 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:08.026656    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:43:09.090460 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:09.090425    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:43:09.190466 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:09.190425    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:43:09.290656 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:09.290590    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:43:09.596877 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:09.596841    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:43:09.653484 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:09.653451    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:43:27.084772 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:27.084737    2564 generic.go:356] "Generic (PLEG): container finished" podID="7820d541-2f9a-41a7-bf74-b0e8b4161d56" containerID="2f976196109681dfe049e4ddb3dc83783db69106d7b959f5116ed6579e328824" exitCode=0
Sep 10 19:43:27.085184 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:27.084805    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-544dea-openshift-default-85985454b-qflgf" event={"ID":"7820d541-2f9a-41a7-bf74-b0e8b4161d56","Type":"ContainerDied","Data":"2f976196109681dfe049e4ddb3dc83783db69106d7b959f5116ed6579e328824"}
Sep 10 19:43:27.085184 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:27.085179    2564 scope.go:117] "RemoveContainer" containerID="2f976196109681dfe049e4ddb3dc83783db69106d7b959f5116ed6579e328824"
Sep 10 19:43:27.086280 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:27.086250    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:43:27.086395 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:27.086331    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:43:27.086395 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:27.086379    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:43:28.092931 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:28.092897    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-544dea-openshift-default-85985454b-qflgf" event={"ID":"7820d541-2f9a-41a7-bf74-b0e8b4161d56","Type":"ContainerStarted","Data":"ed5ab0d5bb026b670029c5924f013d29775e467cac0cf217413137ca556dddb8"}
Sep 10 19:43:29.044644 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:29.044595    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-544dea-openshift-default-85985454b-qflgf"
Sep 10 19:43:29.044920 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:29.044899    2564 patch_prober.go:28] interesting pod/e2e-ipp-b-544dea-openshift-default-85985454b-qflgf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Sep 10 19:43:29.044968 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:29.044948    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-544dea-openshift-default-85985454b-qflgf" podUID="7820d541-2f9a-41a7-bf74-b0e8b4161d56" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Sep 10 19:43:30.044513 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:30.044476    2564 patch_prober.go:28] interesting pod/e2e-ipp-b-544dea-openshift-default-85985454b-qflgf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Sep 10 19:43:30.044931 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:30.044532    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-544dea-openshift-default-85985454b-qflgf" podUID="7820d541-2f9a-41a7-bf74-b0e8b4161d56" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Sep 10 19:43:31.045160 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:31.045124    2564 patch_prober.go:28] interesting pod/e2e-ipp-b-544dea-openshift-default-85985454b-qflgf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Sep 10 19:43:31.045518 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:31.045183    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-544dea-openshift-default-85985454b-qflgf" podUID="7820d541-2f9a-41a7-bf74-b0e8b4161d56" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Sep 10 19:43:32.044877 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:32.044839    2564 patch_prober.go:28] interesting pod/e2e-ipp-b-544dea-openshift-default-85985454b-qflgf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Sep 10 19:43:32.045057 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:32.044903    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-544dea-openshift-default-85985454b-qflgf" podUID="7820d541-2f9a-41a7-bf74-b0e8b4161d56" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Sep 10 19:43:32.934478 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:32.934436    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:43:33.044667 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:33.044598    2564 patch_prober.go:28] interesting pod/e2e-ipp-b-544dea-openshift-default-85985454b-qflgf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Sep 10 19:43:33.044868 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:33.044697    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-544dea-openshift-default-85985454b-qflgf" podUID="7820d541-2f9a-41a7-bf74-b0e8b4161d56" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Sep 10 19:43:34.048117 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:34.048088    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-544dea-openshift-default-85985454b-qflgf"
Sep 10 19:43:34.048502 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:34.048353    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-544dea-openshift-default-85985454b-qflgf"
Sep 10 19:43:34.049214 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:34.049197    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-544dea-openshift-default-85985454b-qflgf"
Sep 10 19:43:38.134905 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:38.134874    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-ll4jr_f80fdb52-770e-4a98-904f-556d9a3d9b28/payload-processing/0.log"
Sep 10 19:43:48.546234 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:43:48.546197    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:19.089825 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:19.089740    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-63ed30-794f98b679-f8v4z_02704747-24a5-4961-abac-7651fadd04fb/payload-processing/0.log"
Sep 10 19:44:19.219276 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:19.219241    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-ll4jr_f80fdb52-770e-4a98-904f-556d9a3d9b28/payload-processing/0.log"
Sep 10 19:44:19.635228 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:19.635194    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:19.929698 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:19.929589    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6"]
Sep 10 19:44:19.933839 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:19.933811    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6"
Sep 10 19:44:19.936160 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:19.936138    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-07ffe4-openshift-default-dockercfg-ggxkq\""
Sep 10 19:44:19.944712 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:19.944686    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6"]
Sep 10 19:44:19.953472 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:19.953443    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:20.112025 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:20.111980    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/646f01f5-e49a-43fd-8bc8-244e65ac9f90-istio-token\") pod \"e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6\" (UID: \"646f01f5-e49a-43fd-8bc8-244e65ac9f90\") " pod="openshift-ingress/e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6"
Sep 10 19:44:20.112525 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:20.112075    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/646f01f5-e49a-43fd-8bc8-244e65ac9f90-istiod-ca-cert\") pod \"e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6\" (UID: \"646f01f5-e49a-43fd-8bc8-244e65ac9f90\") " pod="openshift-ingress/e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6"
Sep 10 19:44:20.112525 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:20.112251    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/646f01f5-e49a-43fd-8bc8-244e65ac9f90-workload-certs\") pod \"e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6\" (UID: \"646f01f5-e49a-43fd-8bc8-244e65ac9f90\") " pod="openshift-ingress/e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6"
Sep 10 19:44:20.112525 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:20.112290    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8276f\" (UniqueName: \"kubernetes.io/projected/646f01f5-e49a-43fd-8bc8-244e65ac9f90-kube-api-access-8276f\") pod \"e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6\" (UID: \"646f01f5-e49a-43fd-8bc8-244e65ac9f90\") " pod="openshift-ingress/e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6"
Sep 10 19:44:20.112525 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:20.112403    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/646f01f5-e49a-43fd-8bc8-244e65ac9f90-istio-podinfo\") pod \"e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6\" (UID: \"646f01f5-e49a-43fd-8bc8-244e65ac9f90\") " pod="openshift-ingress/e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6"
Sep 10 19:44:20.112525 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:20.112476    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/646f01f5-e49a-43fd-8bc8-244e65ac9f90-credential-socket\") pod \"e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6\" (UID: \"646f01f5-e49a-43fd-8bc8-244e65ac9f90\") " pod="openshift-ingress/e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6"
Sep 10 19:44:20.112789 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:20.112555    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/646f01f5-e49a-43fd-8bc8-244e65ac9f90-workload-socket\") pod \"e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6\" (UID: \"646f01f5-e49a-43fd-8bc8-244e65ac9f90\") " pod="openshift-ingress/e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6"
Sep 10 19:44:20.112789 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:20.112739    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/646f01f5-e49a-43fd-8bc8-244e65ac9f90-istio-data\") pod \"e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6\" (UID: \"646f01f5-e49a-43fd-8bc8-244e65ac9f90\") " pod="openshift-ingress/e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6"
Sep 10 19:44:20.112789 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:20.112765    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/646f01f5-e49a-43fd-8bc8-244e65ac9f90-istio-envoy\") pod \"e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6\" (UID: \"646f01f5-e49a-43fd-8bc8-244e65ac9f90\") " pod="openshift-ingress/e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6"
Sep 10 19:44:20.214026 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:20.213915    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/646f01f5-e49a-43fd-8bc8-244e65ac9f90-workload-socket\") pod \"e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6\" (UID: \"646f01f5-e49a-43fd-8bc8-244e65ac9f90\") " pod="openshift-ingress/e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6"
Sep 10 19:44:20.214026 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:20.213975    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/646f01f5-e49a-43fd-8bc8-244e65ac9f90-istio-data\") pod \"e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6\" (UID: \"646f01f5-e49a-43fd-8bc8-244e65ac9f90\") " pod="openshift-ingress/e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6"
Sep 10 19:44:20.214026 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:20.214002    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/646f01f5-e49a-43fd-8bc8-244e65ac9f90-istio-envoy\") pod \"e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6\" (UID: \"646f01f5-e49a-43fd-8bc8-244e65ac9f90\") " pod="openshift-ingress/e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6"
Sep 10 19:44:20.214310 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:20.214093    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/646f01f5-e49a-43fd-8bc8-244e65ac9f90-istio-token\") pod \"e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6\" (UID: \"646f01f5-e49a-43fd-8bc8-244e65ac9f90\") " pod="openshift-ingress/e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6"
Sep 10 19:44:20.214310 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:20.214120    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/646f01f5-e49a-43fd-8bc8-244e65ac9f90-istiod-ca-cert\") pod \"e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6\" (UID: \"646f01f5-e49a-43fd-8bc8-244e65ac9f90\") " pod="openshift-ingress/e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6"
Sep 10 19:44:20.214310 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:20.214154    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/646f01f5-e49a-43fd-8bc8-244e65ac9f90-workload-certs\") pod \"e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6\" (UID: \"646f01f5-e49a-43fd-8bc8-244e65ac9f90\") " pod="openshift-ingress/e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6"
Sep 10 19:44:20.214310 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:20.214194    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8276f\" (UniqueName: \"kubernetes.io/projected/646f01f5-e49a-43fd-8bc8-244e65ac9f90-kube-api-access-8276f\") pod \"e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6\" (UID: \"646f01f5-e49a-43fd-8bc8-244e65ac9f90\") " pod="openshift-ingress/e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6"
Sep 10 19:44:20.214310 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:20.214254    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/646f01f5-e49a-43fd-8bc8-244e65ac9f90-istio-podinfo\") pod \"e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6\" (UID: \"646f01f5-e49a-43fd-8bc8-244e65ac9f90\") " pod="openshift-ingress/e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6"
Sep 10 19:44:20.214310 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:20.214298    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/646f01f5-e49a-43fd-8bc8-244e65ac9f90-credential-socket\") pod \"e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6\" (UID: \"646f01f5-e49a-43fd-8bc8-244e65ac9f90\") " pod="openshift-ingress/e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6"
Sep 10 19:44:20.214683 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:20.214389    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/646f01f5-e49a-43fd-8bc8-244e65ac9f90-workload-socket\") pod \"e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6\" (UID: \"646f01f5-e49a-43fd-8bc8-244e65ac9f90\") " pod="openshift-ingress/e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6"
Sep 10 19:44:20.214683 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:20.214443    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/646f01f5-e49a-43fd-8bc8-244e65ac9f90-istio-data\") pod \"e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6\" (UID: \"646f01f5-e49a-43fd-8bc8-244e65ac9f90\") " pod="openshift-ingress/e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6"
Sep 10 19:44:20.214683 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:20.214639    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/646f01f5-e49a-43fd-8bc8-244e65ac9f90-credential-socket\") pod \"e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6\" (UID: \"646f01f5-e49a-43fd-8bc8-244e65ac9f90\") " pod="openshift-ingress/e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6"
Sep 10 19:44:20.214828 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:20.214720    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/646f01f5-e49a-43fd-8bc8-244e65ac9f90-workload-certs\") pod \"e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6\" (UID: \"646f01f5-e49a-43fd-8bc8-244e65ac9f90\") " pod="openshift-ingress/e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6"
Sep 10 19:44:20.215010 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:20.214984    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/646f01f5-e49a-43fd-8bc8-244e65ac9f90-istiod-ca-cert\") pod \"e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6\" (UID: \"646f01f5-e49a-43fd-8bc8-244e65ac9f90\") " pod="openshift-ingress/e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6"
Sep 10 19:44:20.216389 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:20.216362    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/646f01f5-e49a-43fd-8bc8-244e65ac9f90-istio-envoy\") pod \"e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6\" (UID: \"646f01f5-e49a-43fd-8bc8-244e65ac9f90\") " pod="openshift-ingress/e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6"
Sep 10 19:44:20.216802 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:20.216781    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/646f01f5-e49a-43fd-8bc8-244e65ac9f90-istio-podinfo\") pod \"e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6\" (UID: \"646f01f5-e49a-43fd-8bc8-244e65ac9f90\") " pod="openshift-ingress/e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6"
Sep 10 19:44:20.232289 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:20.232263    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/646f01f5-e49a-43fd-8bc8-244e65ac9f90-istio-token\") pod \"e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6\" (UID: \"646f01f5-e49a-43fd-8bc8-244e65ac9f90\") " pod="openshift-ingress/e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6"
Sep 10 19:44:20.232701 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:20.232680    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8276f\" (UniqueName: \"kubernetes.io/projected/646f01f5-e49a-43fd-8bc8-244e65ac9f90-kube-api-access-8276f\") pod \"e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6\" (UID: \"646f01f5-e49a-43fd-8bc8-244e65ac9f90\") " pod="openshift-ingress/e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6"
Sep 10 19:44:20.246769 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:20.246745    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6"
Sep 10 19:44:20.333251 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:20.333203    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:20.524920 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:20.524892    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6"]
Sep 10 19:44:20.526533 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:44:20.526506    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod646f01f5_e49a_43fd_8bc8_244e65ac9f90.slice/crio-3ebebbf808f3c6a538fb46225ed467190e800d7e73ff69bfadf54b420685e88d WatchSource:0}: Error finding container 3ebebbf808f3c6a538fb46225ed467190e800d7e73ff69bfadf54b420685e88d: Status 404 returned error can't find the container with id 3ebebbf808f3c6a538fb46225ed467190e800d7e73ff69bfadf54b420685e88d
Sep 10 19:44:20.528692 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:20.528655    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:44:20.528798 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:20.528740    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:44:20.528798 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:20.528787    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:44:21.407515 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:21.407478    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6" event={"ID":"646f01f5-e49a-43fd-8bc8-244e65ac9f90","Type":"ContainerStarted","Data":"360fa2eba0b92e9c92c73801c4a3b8d39206857e526f9151c8c77b58f6d52c59"}
Sep 10 19:44:21.407932 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:21.407534    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6" event={"ID":"646f01f5-e49a-43fd-8bc8-244e65ac9f90","Type":"ContainerStarted","Data":"3ebebbf808f3c6a538fb46225ed467190e800d7e73ff69bfadf54b420685e88d"}
Sep 10 19:44:21.430809 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:21.430753    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6" podStartSLOduration=2.430735967 podStartE2EDuration="2.430735967s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:44:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:44:21.428393126 +0000 UTC m=+1717.029300366" watchObservedRunningTime="2026-09-10 19:44:21.430735967 +0000 UTC m=+1717.031643206"
Sep 10 19:44:22.247632 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:22.247590    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6"
Sep 10 19:44:22.253468 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:22.253440    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6"
Sep 10 19:44:22.413264 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:22.413235    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6"
Sep 10 19:44:22.414458 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:22.414433    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6"
Sep 10 19:44:23.498322 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:23.498288    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:23.590754 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:23.590717    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:24.705016 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:24.704979    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:24.795669 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:24.795637    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:24.988166 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:24.988077    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:25.226040 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:25.226011    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:25.328453 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:25.328413    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:25.410082 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:25.410042    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-07ffe4-76dd4b678-qjsqg"]
Sep 10 19:44:25.413897 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:25.413873    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-07ffe4-76dd4b678-qjsqg"
Sep 10 19:44:25.416095 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:25.416069    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-07ffe4-dockercfg-2526q\""
Sep 10 19:44:25.416244 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:25.416076    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-07ffe4\""
Sep 10 19:44:25.423831 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:25.423808    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-07ffe4-76dd4b678-qjsqg"]
Sep 10 19:44:25.572941 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:25.572909    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4d9mx\" (UniqueName: \"kubernetes.io/projected/76b20f89-da47-425e-a7ca-69c41609dd9a-kube-api-access-4d9mx\") pod \"payload-pre-processing-e2e-ipp-cleanup-07ffe4-76dd4b678-qjsqg\" (UID: \"76b20f89-da47-425e-a7ca-69c41609dd9a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-07ffe4-76dd4b678-qjsqg"
Sep 10 19:44:25.572941 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:25.572946    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76b20f89-da47-425e-a7ca-69c41609dd9a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-07ffe4-76dd4b678-qjsqg\" (UID: \"76b20f89-da47-425e-a7ca-69c41609dd9a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-07ffe4-76dd4b678-qjsqg"
Sep 10 19:44:25.673825 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:25.673711    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4d9mx\" (UniqueName: \"kubernetes.io/projected/76b20f89-da47-425e-a7ca-69c41609dd9a-kube-api-access-4d9mx\") pod \"payload-pre-processing-e2e-ipp-cleanup-07ffe4-76dd4b678-qjsqg\" (UID: \"76b20f89-da47-425e-a7ca-69c41609dd9a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-07ffe4-76dd4b678-qjsqg"
Sep 10 19:44:25.674000 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:25.673859    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76b20f89-da47-425e-a7ca-69c41609dd9a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-07ffe4-76dd4b678-qjsqg\" (UID: \"76b20f89-da47-425e-a7ca-69c41609dd9a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-07ffe4-76dd4b678-qjsqg"
Sep 10 19:44:25.674580 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:25.674557    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76b20f89-da47-425e-a7ca-69c41609dd9a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-07ffe4-76dd4b678-qjsqg\" (UID: \"76b20f89-da47-425e-a7ca-69c41609dd9a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-07ffe4-76dd4b678-qjsqg"
Sep 10 19:44:25.684424 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:25.684399    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4d9mx\" (UniqueName: \"kubernetes.io/projected/76b20f89-da47-425e-a7ca-69c41609dd9a-kube-api-access-4d9mx\") pod \"payload-pre-processing-e2e-ipp-cleanup-07ffe4-76dd4b678-qjsqg\" (UID: \"76b20f89-da47-425e-a7ca-69c41609dd9a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-07ffe4-76dd4b678-qjsqg"
Sep 10 19:44:25.727033 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:25.726993    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-07ffe4-76dd4b678-qjsqg"
Sep 10 19:44:26.000477 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:26.000448    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-07ffe4-76dd4b678-qjsqg"]
Sep 10 19:44:26.002177 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:44:26.002153    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76b20f89_da47_425e_a7ca_69c41609dd9a.slice/crio-e0f9a713f9babe1452b394c27edb9856927135a21d257a59d7ef839d0b89acc6 WatchSource:0}: Error finding container e0f9a713f9babe1452b394c27edb9856927135a21d257a59d7ef839d0b89acc6: Status 404 returned error can't find the container with id e0f9a713f9babe1452b394c27edb9856927135a21d257a59d7ef839d0b89acc6
Sep 10 19:44:26.438987 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:26.438891    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-07ffe4-76dd4b678-qjsqg" event={"ID":"76b20f89-da47-425e-a7ca-69c41609dd9a","Type":"ContainerStarted","Data":"210b02a628e76cc1fa9de27d33eaa554de3d94136118716a6bdeb801cef1cb86"}
Sep 10 19:44:26.438987 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:26.438943    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-07ffe4-76dd4b678-qjsqg" event={"ID":"76b20f89-da47-425e-a7ca-69c41609dd9a","Type":"ContainerStarted","Data":"e0f9a713f9babe1452b394c27edb9856927135a21d257a59d7ef839d0b89acc6"}
Sep 10 19:44:26.439220 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:26.439031    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-07ffe4-76dd4b678-qjsqg"
Sep 10 19:44:26.456339 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:26.456285    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-07ffe4-76dd4b678-qjsqg" podStartSLOduration=1.456264735 podStartE2EDuration="1.456264735s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:44:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:44:26.455307344 +0000 UTC m=+1722.056214584" watchObservedRunningTime="2026-09-10 19:44:26.456264735 +0000 UTC m=+1722.057171980"
Sep 10 19:44:37.447851 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:37.447816    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-07ffe4-76dd4b678-qjsqg"
Sep 10 19:44:43.541430 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:43.541353    2564 generic.go:356] "Generic (PLEG): container finished" podID="646f01f5-e49a-43fd-8bc8-244e65ac9f90" containerID="360fa2eba0b92e9c92c73801c4a3b8d39206857e526f9151c8c77b58f6d52c59" exitCode=0
Sep 10 19:44:43.541837 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:43.541427    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6" event={"ID":"646f01f5-e49a-43fd-8bc8-244e65ac9f90","Type":"ContainerDied","Data":"360fa2eba0b92e9c92c73801c4a3b8d39206857e526f9151c8c77b58f6d52c59"}
Sep 10 19:44:43.541928 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:43.541912    2564 scope.go:117] "RemoveContainer" containerID="360fa2eba0b92e9c92c73801c4a3b8d39206857e526f9151c8c77b58f6d52c59"
Sep 10 19:44:43.543094 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:43.543063    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:44:43.543152 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:43.543136    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:44:43.543192 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:43.543168    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 19:44:44.549909 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:44.549873    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6" event={"ID":"646f01f5-e49a-43fd-8bc8-244e65ac9f90","Type":"ContainerStarted","Data":"7b85c8c3b871a6ae5040db8bd825661e6cc3a2bcbc6316276fb4aea975aa462e"}
Sep 10 19:44:45.247781 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:45.247752    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6"
Sep 10 19:44:45.247955 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:45.247935    2564 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Sep 10 19:44:45.248005 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:45.247986    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6" podUID="646f01f5-e49a-43fd-8bc8-244e65ac9f90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Sep 10 19:44:45.291806 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:45.291777    2564 scope.go:117] "RemoveContainer" containerID="34f727a62c4874b5ff5b7f84c97bec6f8b713673f07705d4df685405611b5add"
Sep 10 19:44:46.247644 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:46.247579    2564 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Sep 10 19:44:46.248082 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:46.247668    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6" podUID="646f01f5-e49a-43fd-8bc8-244e65ac9f90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Sep 10 19:44:47.248116 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:47.248078    2564 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Sep 10 19:44:47.248495 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:47.248141    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6" podUID="646f01f5-e49a-43fd-8bc8-244e65ac9f90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Sep 10 19:44:48.248122 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:48.248086    2564 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Sep 10 19:44:48.248516 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:48.248153    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6" podUID="646f01f5-e49a-43fd-8bc8-244e65ac9f90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Sep 10 19:44:48.466847 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:48.466810    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:49.178238 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:49.178191    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-07ffe4-pmgwc"]
Sep 10 19:44:49.183145 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:49.183109    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-07ffe4-pmgwc"
Sep 10 19:44:49.192664 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:49.188470    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 19:44:49.192664 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:49.192388    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tjmqn\""
Sep 10 19:44:49.192887 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:49.192667    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-07ffe4-pmgwc"]
Sep 10 19:44:49.247258 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:49.247222    2564 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Sep 10 19:44:49.247457 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:49.247298    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6" podUID="646f01f5-e49a-43fd-8bc8-244e65ac9f90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Sep 10 19:44:49.324203 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:49.324165    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7db127c0-f77a-46c8-81ab-fa4ddce6ecff-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-07ffe4-pmgwc\" (UID: \"7db127c0-f77a-46c8-81ab-fa4ddce6ecff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-07ffe4-pmgwc"
Sep 10 19:44:49.424975 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:49.424931    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7db127c0-f77a-46c8-81ab-fa4ddce6ecff-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-07ffe4-pmgwc\" (UID: \"7db127c0-f77a-46c8-81ab-fa4ddce6ecff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-07ffe4-pmgwc"
Sep 10 19:44:49.425564 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:49.425543    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7db127c0-f77a-46c8-81ab-fa4ddce6ecff-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-07ffe4-pmgwc\" (UID: \"7db127c0-f77a-46c8-81ab-fa4ddce6ecff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-07ffe4-pmgwc"
Sep 10 19:44:49.504311 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:49.504281    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-07ffe4-pmgwc"
Sep 10 19:44:49.776924 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:49.776884    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-07ffe4-pmgwc"]
Sep 10 19:44:49.778850 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:44:49.778821    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7db127c0_f77a_46c8_81ab_fa4ddce6ecff.slice/crio-7e32e861871c3e024e72085a74c140655a99415663bedd84a8115c1174a13521 WatchSource:0}: Error finding container 7e32e861871c3e024e72085a74c140655a99415663bedd84a8115c1174a13521: Status 404 returned error can't find the container with id 7e32e861871c3e024e72085a74c140655a99415663bedd84a8115c1174a13521
Sep 10 19:44:50.251567 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:50.251537    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6"
Sep 10 19:44:50.251849 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:50.251831    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6"
Sep 10 19:44:50.252600 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:50.252577    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6"
Sep 10 19:44:50.600447 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:50.600407    2564 generic.go:356] "Generic (PLEG): container finished" podID="7db127c0-f77a-46c8-81ab-fa4ddce6ecff" containerID="6025d64d91debe66bdb2f9ff9afb7668f8177d77b012722d6514af2fdcab0143" exitCode=0
Sep 10 19:44:50.600845 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:50.600493    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-07ffe4-pmgwc" event={"ID":"7db127c0-f77a-46c8-81ab-fa4ddce6ecff","Type":"ContainerDied","Data":"6025d64d91debe66bdb2f9ff9afb7668f8177d77b012722d6514af2fdcab0143"}
Sep 10 19:44:50.600845 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:50.600576    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-07ffe4-pmgwc" event={"ID":"7db127c0-f77a-46c8-81ab-fa4ddce6ecff","Type":"ContainerStarted","Data":"7e32e861871c3e024e72085a74c140655a99415663bedd84a8115c1174a13521"}
Sep 10 19:44:51.755916 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:51.755891    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-07ffe4-pmgwc"
Sep 10 19:44:51.849236 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:51.849200    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7db127c0-f77a-46c8-81ab-fa4ddce6ecff-maas-api-service-ca\") pod \"7db127c0-f77a-46c8-81ab-fa4ddce6ecff\" (UID: \"7db127c0-f77a-46c8-81ab-fa4ddce6ecff\") "
Sep 10 19:44:51.849591 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:51.849567    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7db127c0-f77a-46c8-81ab-fa4ddce6ecff-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7db127c0-f77a-46c8-81ab-fa4ddce6ecff" (UID: "7db127c0-f77a-46c8-81ab-fa4ddce6ecff"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:44:51.950498 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:51.950421    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7db127c0-f77a-46c8-81ab-fa4ddce6ecff-maas-api-service-ca\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:44:52.613426 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:52.613389    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-07ffe4-pmgwc"
Sep 10 19:44:52.613426 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:52.613417    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-07ffe4-pmgwc" event={"ID":"7db127c0-f77a-46c8-81ab-fa4ddce6ecff","Type":"ContainerDied","Data":"7e32e861871c3e024e72085a74c140655a99415663bedd84a8115c1174a13521"}
Sep 10 19:44:52.613660 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:52.613464    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7e32e861871c3e024e72085a74c140655a99415663bedd84a8115c1174a13521"
Sep 10 19:44:55.300367 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:55.300331    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:55.347905 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:55.347870    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-07ffe4-76dd4b678-qjsqg"]
Sep 10 19:44:55.348151 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:55.348122    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-07ffe4-76dd4b678-qjsqg" podUID="76b20f89-da47-425e-a7ca-69c41609dd9a" containerName="payload-pre-processing" containerID="cri-o://210b02a628e76cc1fa9de27d33eaa554de3d94136118716a6bdeb801cef1cb86" gracePeriod=30
Sep 10 19:44:55.399707 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:55.399670    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:55.499004 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:55.498963    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:55.534353 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:55.534321    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-07ffe4-pmgwc"]
Sep 10 19:44:55.538005 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:55.537975    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-07ffe4-pmgwc"]
Sep 10 19:44:55.558521 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:55.558451    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:55.615369 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:55.615343    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-07ffe4-76dd4b678-qjsqg"
Sep 10 19:44:55.635153 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:55.634405    2564 generic.go:356] "Generic (PLEG): container finished" podID="76b20f89-da47-425e-a7ca-69c41609dd9a" containerID="210b02a628e76cc1fa9de27d33eaa554de3d94136118716a6bdeb801cef1cb86" exitCode=0
Sep 10 19:44:55.635153 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:55.634443    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-07ffe4-76dd4b678-qjsqg" event={"ID":"76b20f89-da47-425e-a7ca-69c41609dd9a","Type":"ContainerDied","Data":"210b02a628e76cc1fa9de27d33eaa554de3d94136118716a6bdeb801cef1cb86"}
Sep 10 19:44:55.635153 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:55.634490    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-07ffe4-76dd4b678-qjsqg"
Sep 10 19:44:55.635153 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:55.634519    2564 scope.go:117] "RemoveContainer" containerID="210b02a628e76cc1fa9de27d33eaa554de3d94136118716a6bdeb801cef1cb86"
Sep 10 19:44:55.635153 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:55.634503    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-07ffe4-76dd4b678-qjsqg" event={"ID":"76b20f89-da47-425e-a7ca-69c41609dd9a","Type":"ContainerDied","Data":"e0f9a713f9babe1452b394c27edb9856927135a21d257a59d7ef839d0b89acc6"}
Sep 10 19:44:55.650882 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:55.650859    2564 scope.go:117] "RemoveContainer" containerID="210b02a628e76cc1fa9de27d33eaa554de3d94136118716a6bdeb801cef1cb86"
Sep 10 19:44:55.651230 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:44:55.651207    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"210b02a628e76cc1fa9de27d33eaa554de3d94136118716a6bdeb801cef1cb86\": container with ID starting with 210b02a628e76cc1fa9de27d33eaa554de3d94136118716a6bdeb801cef1cb86 not found: ID does not exist" containerID="210b02a628e76cc1fa9de27d33eaa554de3d94136118716a6bdeb801cef1cb86"
Sep 10 19:44:55.651326 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:55.651236    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"210b02a628e76cc1fa9de27d33eaa554de3d94136118716a6bdeb801cef1cb86"} err="failed to get container status \"210b02a628e76cc1fa9de27d33eaa554de3d94136118716a6bdeb801cef1cb86\": rpc error: code = NotFound desc = could not find container \"210b02a628e76cc1fa9de27d33eaa554de3d94136118716a6bdeb801cef1cb86\": container with ID starting with 210b02a628e76cc1fa9de27d33eaa554de3d94136118716a6bdeb801cef1cb86 not found: ID does not exist"
Sep 10 19:44:55.789541 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:55.789508    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4d9mx\" (UniqueName: \"kubernetes.io/projected/76b20f89-da47-425e-a7ca-69c41609dd9a-kube-api-access-4d9mx\") pod \"76b20f89-da47-425e-a7ca-69c41609dd9a\" (UID: \"76b20f89-da47-425e-a7ca-69c41609dd9a\") "
Sep 10 19:44:55.789873 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:55.789570    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76b20f89-da47-425e-a7ca-69c41609dd9a-plugins-config-volume\") pod \"76b20f89-da47-425e-a7ca-69c41609dd9a\" (UID: \"76b20f89-da47-425e-a7ca-69c41609dd9a\") "
Sep 10 19:44:55.789994 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:55.789966    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76b20f89-da47-425e-a7ca-69c41609dd9a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "76b20f89-da47-425e-a7ca-69c41609dd9a" (UID: "76b20f89-da47-425e-a7ca-69c41609dd9a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:44:55.791694 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:55.791670    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76b20f89-da47-425e-a7ca-69c41609dd9a-kube-api-access-4d9mx" (OuterVolumeSpecName: "kube-api-access-4d9mx") pod "76b20f89-da47-425e-a7ca-69c41609dd9a" (UID: "76b20f89-da47-425e-a7ca-69c41609dd9a"). InnerVolumeSpecName "kube-api-access-4d9mx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:44:55.891107 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:55.891024    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4d9mx\" (UniqueName: \"kubernetes.io/projected/76b20f89-da47-425e-a7ca-69c41609dd9a-kube-api-access-4d9mx\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:44:55.891107 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:55.891056    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76b20f89-da47-425e-a7ca-69c41609dd9a-plugins-config-volume\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:44:55.907945 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:55.907915    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:55.968018 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:55.967956    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-07ffe4-76dd4b678-qjsqg"]
Sep 10 19:44:55.969673 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:55.969647    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-07ffe4-76dd4b678-qjsqg"]
Sep 10 19:44:56.950103 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:56.950056    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76b20f89-da47-425e-a7ca-69c41609dd9a" path="/var/lib/kubelet/pods/76b20f89-da47-425e-a7ca-69c41609dd9a/volumes"
Sep 10 19:44:56.950598 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:44:56.950575    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7db127c0-f77a-46c8-81ab-fa4ddce6ecff" path="/var/lib/kubelet/pods/7db127c0-f77a-46c8-81ab-fa4ddce6ecff/volumes"
Sep 10 19:45:00.134561 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:00.134526    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-63ed30-29817825-rx9ch"]
Sep 10 19:45:00.135042 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:00.135029    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76b20f89-da47-425e-a7ca-69c41609dd9a" containerName="payload-pre-processing"
Sep 10 19:45:00.135042 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:00.135042    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="76b20f89-da47-425e-a7ca-69c41609dd9a" containerName="payload-pre-processing"
Sep 10 19:45:00.135136 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:00.135063    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7db127c0-f77a-46c8-81ab-fa4ddce6ecff" containerName="revoke-keys"
Sep 10 19:45:00.135136 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:00.135072    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7db127c0-f77a-46c8-81ab-fa4ddce6ecff" containerName="revoke-keys"
Sep 10 19:45:00.135214 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:00.135152    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="76b20f89-da47-425e-a7ca-69c41609dd9a" containerName="payload-pre-processing"
Sep 10 19:45:00.135214 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:00.135165    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7db127c0-f77a-46c8-81ab-fa4ddce6ecff" containerName="revoke-keys"
Sep 10 19:45:00.139779 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:00.139759    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-63ed30-29817825-rx9ch"
Sep 10 19:45:00.140675 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:00.140614    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29817825-sb2x8"]
Sep 10 19:45:00.142010 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:00.141993    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tjmqn\""
Sep 10 19:45:00.144708 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:00.144688    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-544dea-29817825-5z6l5"]
Sep 10 19:45:00.144859 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:00.144840    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817825-sb2x8"
Sep 10 19:45:00.148502 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:00.148473    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-63ed30-29817825-rx9ch"]
Sep 10 19:45:00.148591 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:00.148569    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-544dea-29817825-5z6l5"
Sep 10 19:45:00.155725 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:00.155539    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-63ed30-29817825-rx9ch"
Sep 10 19:45:00.159292 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:00.159271    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29817825-sb2x8"]
Sep 10 19:45:00.167397 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:00.167357    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-544dea-29817825-5z6l5"]
Sep 10 19:45:00.169460 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:00.169431    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817825-sb2x8"
Sep 10 19:45:00.209988 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:00.209963    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-544dea-29817825-5z6l5"
Sep 10 19:45:00.327410 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:00.327378    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:45:00.444769 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:00.444745    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-63ed30-29817825-rx9ch"]
Sep 10 19:45:00.446561 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:45:00.446535    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode425a113_9d02_4daa_ae4d_79d64eb193a1.slice/crio-43f66c290890e87c57518eb87304b8ba5a1797b903d89090284085f089752e5e WatchSource:0}: Error finding container 43f66c290890e87c57518eb87304b8ba5a1797b903d89090284085f089752e5e: Status 404 returned error can't find the container with id 43f66c290890e87c57518eb87304b8ba5a1797b903d89090284085f089752e5e
Sep 10 19:45:00.461486 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:45:00.461461    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04e90788_4814_4c5e_9d46_8df1cb15563e.slice/crio-9bd67a5139912cd944fce14f2176ef7c6d29def47130519f83edec59252798a7 WatchSource:0}: Error finding container 9bd67a5139912cd944fce14f2176ef7c6d29def47130519f83edec59252798a7: Status 404 returned error can't find the container with id 9bd67a5139912cd944fce14f2176ef7c6d29def47130519f83edec59252798a7
Sep 10 19:45:00.461586 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:00.461565    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29817825-sb2x8"]
Sep 10 19:45:00.511056 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:00.511032    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-544dea-29817825-5z6l5"]
Sep 10 19:45:00.513647 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:45:00.513596    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc07ad6e5_ee85_4b1f_abfd_2511027b2383.slice/crio-7276311b7004df399484061de70a1e49b1af47c6206aa62613ac69df5d7f1be1 WatchSource:0}: Error finding container 7276311b7004df399484061de70a1e49b1af47c6206aa62613ac69df5d7f1be1: Status 404 returned error can't find the container with id 7276311b7004df399484061de70a1e49b1af47c6206aa62613ac69df5d7f1be1
Sep 10 19:45:00.666850 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:00.666816    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-63ed30-29817825-rx9ch" event={"ID":"e425a113-9d02-4daa-ae4d-79d64eb193a1","Type":"ContainerStarted","Data":"43f66c290890e87c57518eb87304b8ba5a1797b903d89090284085f089752e5e"}
Sep 10 19:45:00.668268 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:00.668238    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-544dea-29817825-5z6l5" event={"ID":"c07ad6e5-ee85-4b1f-abfd-2511027b2383","Type":"ContainerStarted","Data":"7276311b7004df399484061de70a1e49b1af47c6206aa62613ac69df5d7f1be1"}
Sep 10 19:45:00.669543 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:00.669522    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817825-sb2x8" event={"ID":"04e90788-4814-4c5e-9d46-8df1cb15563e","Type":"ContainerStarted","Data":"9bd67a5139912cd944fce14f2176ef7c6d29def47130519f83edec59252798a7"}
Sep 10 19:45:01.678280 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:01.678243    2564 generic.go:356] "Generic (PLEG): container finished" podID="c07ad6e5-ee85-4b1f-abfd-2511027b2383" containerID="587a8de717e7d2ed4d0db03ef4efed0d76e4c8e5e3bbbacf8172ca7bbd5a15a9" exitCode=0
Sep 10 19:45:01.678724 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:01.678321    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-544dea-29817825-5z6l5" event={"ID":"c07ad6e5-ee85-4b1f-abfd-2511027b2383","Type":"ContainerDied","Data":"587a8de717e7d2ed4d0db03ef4efed0d76e4c8e5e3bbbacf8172ca7bbd5a15a9"}
Sep 10 19:45:01.680073 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:01.680044    2564 generic.go:356] "Generic (PLEG): container finished" podID="04e90788-4814-4c5e-9d46-8df1cb15563e" containerID="30c0707ac564d275742094baff4e3ed59da03829b389cf4448cee2f856e64f01" exitCode=0
Sep 10 19:45:01.680192 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:01.680122    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817825-sb2x8" event={"ID":"04e90788-4814-4c5e-9d46-8df1cb15563e","Type":"ContainerDied","Data":"30c0707ac564d275742094baff4e3ed59da03829b389cf4448cee2f856e64f01"}
Sep 10 19:45:01.681809 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:01.681783    2564 generic.go:356] "Generic (PLEG): container finished" podID="e425a113-9d02-4daa-ae4d-79d64eb193a1" containerID="e29736aedf250506b47a808b1509f0ae1a71c60894ad569e230e093edaf8f2fc" exitCode=0
Sep 10 19:45:01.681927 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:01.681826    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-63ed30-29817825-rx9ch" event={"ID":"e425a113-9d02-4daa-ae4d-79d64eb193a1","Type":"ContainerDied","Data":"e29736aedf250506b47a808b1509f0ae1a71c60894ad569e230e093edaf8f2fc"}
Sep 10 19:45:02.267688 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:02.267645    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:45:02.367437 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:02.367401    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:45:02.538005 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:02.537916    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6"]
Sep 10 19:45:02.538179 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:02.538149    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6" podUID="646f01f5-e49a-43fd-8bc8-244e65ac9f90" containerName="istio-proxy" containerID="cri-o://7b85c8c3b871a6ae5040db8bd825661e6cc3a2bcbc6316276fb4aea975aa462e" gracePeriod=30
Sep 10 19:45:02.604422 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:02.604384    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:45:02.835013 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:02.834981    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-544dea-29817825-5z6l5"
Sep 10 19:45:02.893825 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:02.893802    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-63ed30-29817825-rx9ch"
Sep 10 19:45:02.896903 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:02.896883    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817825-sb2x8"
Sep 10 19:45:03.695016 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:03.694983    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-63ed30-29817825-rx9ch"
Sep 10 19:45:03.695016 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:03.695012    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-63ed30-29817825-rx9ch" event={"ID":"e425a113-9d02-4daa-ae4d-79d64eb193a1","Type":"ContainerDied","Data":"43f66c290890e87c57518eb87304b8ba5a1797b903d89090284085f089752e5e"}
Sep 10 19:45:03.695234 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:03.695063    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="43f66c290890e87c57518eb87304b8ba5a1797b903d89090284085f089752e5e"
Sep 10 19:45:03.696434 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:03.696417    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-544dea-29817825-5z6l5"
Sep 10 19:45:03.696577 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:03.696437    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-544dea-29817825-5z6l5" event={"ID":"c07ad6e5-ee85-4b1f-abfd-2511027b2383","Type":"ContainerDied","Data":"7276311b7004df399484061de70a1e49b1af47c6206aa62613ac69df5d7f1be1"}
Sep 10 19:45:03.696577 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:03.696482    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7276311b7004df399484061de70a1e49b1af47c6206aa62613ac69df5d7f1be1"
Sep 10 19:45:03.698036 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:03.698019    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817825-sb2x8"
Sep 10 19:45:03.698036 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:03.698025    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817825-sb2x8" event={"ID":"04e90788-4814-4c5e-9d46-8df1cb15563e","Type":"ContainerDied","Data":"9bd67a5139912cd944fce14f2176ef7c6d29def47130519f83edec59252798a7"}
Sep 10 19:45:03.698168 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:03.698057    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9bd67a5139912cd944fce14f2176ef7c6d29def47130519f83edec59252798a7"
Sep 10 19:45:05.255071 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:05.255035    2564 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 19:45:05.255457 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:05.255102    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6" podUID="646f01f5-e49a-43fd-8bc8-244e65ac9f90" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 19:45:06.277569 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:06.277532    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-63ed30-rfc8j"]
Sep 10 19:45:06.278089 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:06.278073    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04e90788-4814-4c5e-9d46-8df1cb15563e" containerName="cleanup"
Sep 10 19:45:06.278135 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:06.278090    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="04e90788-4814-4c5e-9d46-8df1cb15563e" containerName="cleanup"
Sep 10 19:45:06.278135 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:06.278100    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e425a113-9d02-4daa-ae4d-79d64eb193a1" containerName="cleanup"
Sep 10 19:45:06.278135 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:06.278106    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e425a113-9d02-4daa-ae4d-79d64eb193a1" containerName="cleanup"
Sep 10 19:45:06.278135 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:06.278113    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c07ad6e5-ee85-4b1f-abfd-2511027b2383" containerName="cleanup"
Sep 10 19:45:06.278135 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:06.278119    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="c07ad6e5-ee85-4b1f-abfd-2511027b2383" containerName="cleanup"
Sep 10 19:45:06.278290 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:06.278242    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="04e90788-4814-4c5e-9d46-8df1cb15563e" containerName="cleanup"
Sep 10 19:45:06.278290 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:06.278252    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e425a113-9d02-4daa-ae4d-79d64eb193a1" containerName="cleanup"
Sep 10 19:45:06.278290 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:06.278260    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="c07ad6e5-ee85-4b1f-abfd-2511027b2383" containerName="cleanup"
Sep 10 19:45:06.281519 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:06.281501    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-63ed30-rfc8j"
Sep 10 19:45:06.284117 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:06.284092    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tjmqn\""
Sep 10 19:45:06.284549 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:06.284530    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 19:45:06.289202 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:06.289174    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-63ed30-rfc8j"]
Sep 10 19:45:06.296825 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:06.296723    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66eb2786-c1ff-4023-8575-b8fc2a5cb48d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-63ed30-rfc8j\" (UID: \"66eb2786-c1ff-4023-8575-b8fc2a5cb48d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-63ed30-rfc8j"
Sep 10 19:45:06.393468 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:06.393434    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:45:06.397605 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:06.397579    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66eb2786-c1ff-4023-8575-b8fc2a5cb48d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-63ed30-rfc8j\" (UID: \"66eb2786-c1ff-4023-8575-b8fc2a5cb48d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-63ed30-rfc8j"
Sep 10 19:45:06.398203 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:06.398185    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66eb2786-c1ff-4023-8575-b8fc2a5cb48d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-63ed30-rfc8j\" (UID: \"66eb2786-c1ff-4023-8575-b8fc2a5cb48d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-63ed30-rfc8j"
Sep 10 19:45:06.593729 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:06.593642    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-63ed30-rfc8j"
Sep 10 19:45:06.868813 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:06.868783    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-63ed30-rfc8j"]
Sep 10 19:45:06.870767 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:45:06.870729    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66eb2786_c1ff_4023_8575_b8fc2a5cb48d.slice/crio-db785eb3a827a402be710e7d69c1004e27700358023689a96fab0767e21e8733 WatchSource:0}: Error finding container db785eb3a827a402be710e7d69c1004e27700358023689a96fab0767e21e8733: Status 404 returned error can't find the container with id db785eb3a827a402be710e7d69c1004e27700358023689a96fab0767e21e8733
Sep 10 19:45:07.724938 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:07.724903    2564 generic.go:356] "Generic (PLEG): container finished" podID="66eb2786-c1ff-4023-8575-b8fc2a5cb48d" containerID="6af467b62eec005fbe371eaf37ca4afe9f3df9473dbb340a7a538257b2240cea" exitCode=0
Sep 10 19:45:07.725434 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:07.725005    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-63ed30-rfc8j" event={"ID":"66eb2786-c1ff-4023-8575-b8fc2a5cb48d","Type":"ContainerDied","Data":"6af467b62eec005fbe371eaf37ca4afe9f3df9473dbb340a7a538257b2240cea"}
Sep 10 19:45:07.725434 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:07.725087    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-63ed30-rfc8j" event={"ID":"66eb2786-c1ff-4023-8575-b8fc2a5cb48d","Type":"ContainerStarted","Data":"db785eb3a827a402be710e7d69c1004e27700358023689a96fab0767e21e8733"}
Sep 10 19:45:07.728212 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:07.728079    2564 generic.go:356] "Generic (PLEG): container finished" podID="646f01f5-e49a-43fd-8bc8-244e65ac9f90" containerID="7b85c8c3b871a6ae5040db8bd825661e6cc3a2bcbc6316276fb4aea975aa462e" exitCode=0
Sep 10 19:45:07.728212 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:07.728142    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6" event={"ID":"646f01f5-e49a-43fd-8bc8-244e65ac9f90","Type":"ContainerDied","Data":"7b85c8c3b871a6ae5040db8bd825661e6cc3a2bcbc6316276fb4aea975aa462e"}
Sep 10 19:45:07.728212 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:07.728195    2564 scope.go:117] "RemoveContainer" containerID="360fa2eba0b92e9c92c73801c4a3b8d39206857e526f9151c8c77b58f6d52c59"
Sep 10 19:45:07.804339 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:07.804278    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6"
Sep 10 19:45:07.912248 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:07.912203    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/646f01f5-e49a-43fd-8bc8-244e65ac9f90-istio-data\") pod \"646f01f5-e49a-43fd-8bc8-244e65ac9f90\" (UID: \"646f01f5-e49a-43fd-8bc8-244e65ac9f90\") "
Sep 10 19:45:07.912401 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:07.912268    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/646f01f5-e49a-43fd-8bc8-244e65ac9f90-istio-podinfo\") pod \"646f01f5-e49a-43fd-8bc8-244e65ac9f90\" (UID: \"646f01f5-e49a-43fd-8bc8-244e65ac9f90\") "
Sep 10 19:45:07.912401 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:07.912305    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/646f01f5-e49a-43fd-8bc8-244e65ac9f90-credential-socket\") pod \"646f01f5-e49a-43fd-8bc8-244e65ac9f90\" (UID: \"646f01f5-e49a-43fd-8bc8-244e65ac9f90\") "
Sep 10 19:45:07.912401 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:07.912323    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/646f01f5-e49a-43fd-8bc8-244e65ac9f90-istio-envoy\") pod \"646f01f5-e49a-43fd-8bc8-244e65ac9f90\" (UID: \"646f01f5-e49a-43fd-8bc8-244e65ac9f90\") "
Sep 10 19:45:07.912401 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:07.912352    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/646f01f5-e49a-43fd-8bc8-244e65ac9f90-workload-socket\") pod \"646f01f5-e49a-43fd-8bc8-244e65ac9f90\" (UID: \"646f01f5-e49a-43fd-8bc8-244e65ac9f90\") "
Sep 10 19:45:07.912401 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:07.912376    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/646f01f5-e49a-43fd-8bc8-244e65ac9f90-workload-certs\") pod \"646f01f5-e49a-43fd-8bc8-244e65ac9f90\" (UID: \"646f01f5-e49a-43fd-8bc8-244e65ac9f90\") "
Sep 10 19:45:07.912663 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:07.912427    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/646f01f5-e49a-43fd-8bc8-244e65ac9f90-istio-token\") pod \"646f01f5-e49a-43fd-8bc8-244e65ac9f90\" (UID: \"646f01f5-e49a-43fd-8bc8-244e65ac9f90\") "
Sep 10 19:45:07.912663 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:07.912452    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8276f\" (UniqueName: \"kubernetes.io/projected/646f01f5-e49a-43fd-8bc8-244e65ac9f90-kube-api-access-8276f\") pod \"646f01f5-e49a-43fd-8bc8-244e65ac9f90\" (UID: \"646f01f5-e49a-43fd-8bc8-244e65ac9f90\") "
Sep 10 19:45:07.912663 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:07.912513    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/646f01f5-e49a-43fd-8bc8-244e65ac9f90-istiod-ca-cert\") pod \"646f01f5-e49a-43fd-8bc8-244e65ac9f90\" (UID: \"646f01f5-e49a-43fd-8bc8-244e65ac9f90\") "
Sep 10 19:45:07.912663 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:07.912546    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/646f01f5-e49a-43fd-8bc8-244e65ac9f90-istio-data" (OuterVolumeSpecName: "istio-data") pod "646f01f5-e49a-43fd-8bc8-244e65ac9f90" (UID: "646f01f5-e49a-43fd-8bc8-244e65ac9f90"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:45:07.912958 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:07.912698    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/646f01f5-e49a-43fd-8bc8-244e65ac9f90-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "646f01f5-e49a-43fd-8bc8-244e65ac9f90" (UID: "646f01f5-e49a-43fd-8bc8-244e65ac9f90"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:45:07.912958 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:07.912891    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/646f01f5-e49a-43fd-8bc8-244e65ac9f90-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "646f01f5-e49a-43fd-8bc8-244e65ac9f90" (UID: "646f01f5-e49a-43fd-8bc8-244e65ac9f90"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:45:07.913046 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:07.912957    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/646f01f5-e49a-43fd-8bc8-244e65ac9f90-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "646f01f5-e49a-43fd-8bc8-244e65ac9f90" (UID: "646f01f5-e49a-43fd-8bc8-244e65ac9f90"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:45:07.913046 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:07.912982    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/646f01f5-e49a-43fd-8bc8-244e65ac9f90-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "646f01f5-e49a-43fd-8bc8-244e65ac9f90" (UID: "646f01f5-e49a-43fd-8bc8-244e65ac9f90"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:45:07.913046 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:07.912996    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/646f01f5-e49a-43fd-8bc8-244e65ac9f90-istio-data\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:45:07.913046 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:07.913019    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/646f01f5-e49a-43fd-8bc8-244e65ac9f90-credential-socket\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:45:07.913046 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:07.913035    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/646f01f5-e49a-43fd-8bc8-244e65ac9f90-workload-certs\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:45:07.914917 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:07.914643    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/646f01f5-e49a-43fd-8bc8-244e65ac9f90-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "646f01f5-e49a-43fd-8bc8-244e65ac9f90" (UID: "646f01f5-e49a-43fd-8bc8-244e65ac9f90"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 19:45:07.914917 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:07.914699    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/646f01f5-e49a-43fd-8bc8-244e65ac9f90-istio-token" (OuterVolumeSpecName: "istio-token") pod "646f01f5-e49a-43fd-8bc8-244e65ac9f90" (UID: "646f01f5-e49a-43fd-8bc8-244e65ac9f90"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:45:07.915062 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:07.914964    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/646f01f5-e49a-43fd-8bc8-244e65ac9f90-kube-api-access-8276f" (OuterVolumeSpecName: "kube-api-access-8276f") pod "646f01f5-e49a-43fd-8bc8-244e65ac9f90" (UID: "646f01f5-e49a-43fd-8bc8-244e65ac9f90"). InnerVolumeSpecName "kube-api-access-8276f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:45:07.915062 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:07.914964    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/646f01f5-e49a-43fd-8bc8-244e65ac9f90-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "646f01f5-e49a-43fd-8bc8-244e65ac9f90" (UID: "646f01f5-e49a-43fd-8bc8-244e65ac9f90"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:45:08.013736 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:08.013691    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/646f01f5-e49a-43fd-8bc8-244e65ac9f90-istio-envoy\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:45:08.013736 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:08.013726    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/646f01f5-e49a-43fd-8bc8-244e65ac9f90-workload-socket\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:45:08.013736 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:08.013745    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/646f01f5-e49a-43fd-8bc8-244e65ac9f90-istio-token\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:45:08.013990 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:08.013757    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8276f\" (UniqueName: \"kubernetes.io/projected/646f01f5-e49a-43fd-8bc8-244e65ac9f90-kube-api-access-8276f\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:45:08.013990 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:08.013767    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/646f01f5-e49a-43fd-8bc8-244e65ac9f90-istiod-ca-cert\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:45:08.013990 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:08.013776    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/646f01f5-e49a-43fd-8bc8-244e65ac9f90-istio-podinfo\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:45:08.735429 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:08.735398    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6"
Sep 10 19:45:08.735429 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:08.735419    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6" event={"ID":"646f01f5-e49a-43fd-8bc8-244e65ac9f90","Type":"ContainerDied","Data":"3ebebbf808f3c6a538fb46225ed467190e800d7e73ff69bfadf54b420685e88d"}
Sep 10 19:45:08.735987 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:08.735488    2564 scope.go:117] "RemoveContainer" containerID="7b85c8c3b871a6ae5040db8bd825661e6cc3a2bcbc6316276fb4aea975aa462e"
Sep 10 19:45:08.774049 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:08.774011    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6"]
Sep 10 19:45:08.778899 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:08.778871    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-07ffe4-openshift-default-85c6f64bf6-gm8k6"]
Sep 10 19:45:08.880458 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:08.880432    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-63ed30-rfc8j"
Sep 10 19:45:08.921559 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:08.921514    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66eb2786-c1ff-4023-8575-b8fc2a5cb48d-maas-api-service-ca\") pod \"66eb2786-c1ff-4023-8575-b8fc2a5cb48d\" (UID: \"66eb2786-c1ff-4023-8575-b8fc2a5cb48d\") "
Sep 10 19:45:08.921979 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:08.921951    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66eb2786-c1ff-4023-8575-b8fc2a5cb48d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "66eb2786-c1ff-4023-8575-b8fc2a5cb48d" (UID: "66eb2786-c1ff-4023-8575-b8fc2a5cb48d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:45:08.949565 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:08.949529    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="646f01f5-e49a-43fd-8bc8-244e65ac9f90" path="/var/lib/kubelet/pods/646f01f5-e49a-43fd-8bc8-244e65ac9f90/volumes"
Sep 10 19:45:09.022504 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:09.022468    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66eb2786-c1ff-4023-8575-b8fc2a5cb48d-maas-api-service-ca\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:45:09.742306 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:09.742276    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-63ed30-rfc8j"
Sep 10 19:45:09.742306 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:09.742302    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-63ed30-rfc8j" event={"ID":"66eb2786-c1ff-4023-8575-b8fc2a5cb48d","Type":"ContainerDied","Data":"db785eb3a827a402be710e7d69c1004e27700358023689a96fab0767e21e8733"}
Sep 10 19:45:09.742809 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:09.742343    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="db785eb3a827a402be710e7d69c1004e27700358023689a96fab0767e21e8733"
Sep 10 19:45:21.879266 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:21.879224    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:45:21.880664 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:21.880639    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-63ed30-29817825-rx9ch"]
Sep 10 19:45:21.886445 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:21.886422    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-63ed30-29817825-rx9ch"]
Sep 10 19:45:21.908993 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:21.908947    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-63ed30-794f98b679-f8v4z"]
Sep 10 19:45:21.909209 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:21.909182    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-63ed30-794f98b679-f8v4z" podUID="02704747-24a5-4961-abac-7651fadd04fb" containerName="payload-processing" containerID="cri-o://e79feaa0aea0e5cb31ad55d1cd653c8bf014ccdf4162afa3491f281a848c70a3" gracePeriod=30
Sep 10 19:45:21.914448 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:21.914421    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-63ed30-684b99ccb6-vgplv"]
Sep 10 19:45:21.914836 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:21.914803    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-63ed30-684b99ccb6-vgplv" podUID="f85c24d6-df13-4f97-afdc-767ccf51a2a5" containerName="payload-pre-processing" containerID="cri-o://fa335220bda9a8133d8a068ec413ef71457d44aa955651c57df399b0e8c87c26" gracePeriod=30
Sep 10 19:45:21.952325 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:21.952282    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:45:22.009112 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:22.009081    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:45:22.114095 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:22.114058    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:45:22.124504 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:22.124473    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-63ed30-rfc8j"]
Sep 10 19:45:22.135610 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:22.135546    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-63ed30-rfc8j"]
Sep 10 19:45:22.175949 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:22.175925    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-63ed30-794f98b679-f8v4z"
Sep 10 19:45:22.200475 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:22.200450    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:45:22.209441 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:22.209418    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-63ed30-684b99ccb6-vgplv"
Sep 10 19:45:22.267612 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:22.263967    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-np6fk\" (UniqueName: \"kubernetes.io/projected/02704747-24a5-4961-abac-7651fadd04fb-kube-api-access-np6fk\") pod \"02704747-24a5-4961-abac-7651fadd04fb\" (UID: \"02704747-24a5-4961-abac-7651fadd04fb\") "
Sep 10 19:45:22.267612 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:22.264138    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/02704747-24a5-4961-abac-7651fadd04fb-plugins-config-volume\") pod \"02704747-24a5-4961-abac-7651fadd04fb\" (UID: \"02704747-24a5-4961-abac-7651fadd04fb\") "
Sep 10 19:45:22.267612 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:22.267397    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/02704747-24a5-4961-abac-7651fadd04fb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "02704747-24a5-4961-abac-7651fadd04fb" (UID: "02704747-24a5-4961-abac-7651fadd04fb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:45:22.270542 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:22.269128    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02704747-24a5-4961-abac-7651fadd04fb-kube-api-access-np6fk" (OuterVolumeSpecName: "kube-api-access-np6fk") pod "02704747-24a5-4961-abac-7651fadd04fb" (UID: "02704747-24a5-4961-abac-7651fadd04fb"). InnerVolumeSpecName "kube-api-access-np6fk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:45:22.365038 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:22.365006    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f85c24d6-df13-4f97-afdc-767ccf51a2a5-plugins-config-volume\") pod \"f85c24d6-df13-4f97-afdc-767ccf51a2a5\" (UID: \"f85c24d6-df13-4f97-afdc-767ccf51a2a5\") "
Sep 10 19:45:22.365233 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:22.365141    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-htsml\" (UniqueName: \"kubernetes.io/projected/f85c24d6-df13-4f97-afdc-767ccf51a2a5-kube-api-access-htsml\") pod \"f85c24d6-df13-4f97-afdc-767ccf51a2a5\" (UID: \"f85c24d6-df13-4f97-afdc-767ccf51a2a5\") "
Sep 10 19:45:22.365432 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:22.365404    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f85c24d6-df13-4f97-afdc-767ccf51a2a5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f85c24d6-df13-4f97-afdc-767ccf51a2a5" (UID: "f85c24d6-df13-4f97-afdc-767ccf51a2a5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:45:22.365508 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:22.365412    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-np6fk\" (UniqueName: \"kubernetes.io/projected/02704747-24a5-4961-abac-7651fadd04fb-kube-api-access-np6fk\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:45:22.365508 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:22.365460    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/02704747-24a5-4961-abac-7651fadd04fb-plugins-config-volume\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:45:22.367366 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:22.367338    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f85c24d6-df13-4f97-afdc-767ccf51a2a5-kube-api-access-htsml" (OuterVolumeSpecName: "kube-api-access-htsml") pod "f85c24d6-df13-4f97-afdc-767ccf51a2a5" (UID: "f85c24d6-df13-4f97-afdc-767ccf51a2a5"). InnerVolumeSpecName "kube-api-access-htsml". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:45:22.466695 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:22.466604    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f85c24d6-df13-4f97-afdc-767ccf51a2a5-plugins-config-volume\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:45:22.466695 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:22.466646    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-htsml\" (UniqueName: \"kubernetes.io/projected/f85c24d6-df13-4f97-afdc-767ccf51a2a5-kube-api-access-htsml\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:45:22.485594 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:22.485560    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:45:22.822331 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:22.822296    2564 generic.go:356] "Generic (PLEG): container finished" podID="02704747-24a5-4961-abac-7651fadd04fb" containerID="e79feaa0aea0e5cb31ad55d1cd653c8bf014ccdf4162afa3491f281a848c70a3" exitCode=0
Sep 10 19:45:22.822533 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:22.822338    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-63ed30-794f98b679-f8v4z"
Sep 10 19:45:22.822533 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:22.822388    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-63ed30-794f98b679-f8v4z" event={"ID":"02704747-24a5-4961-abac-7651fadd04fb","Type":"ContainerDied","Data":"e79feaa0aea0e5cb31ad55d1cd653c8bf014ccdf4162afa3491f281a848c70a3"}
Sep 10 19:45:22.822533 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:22.822444    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-63ed30-794f98b679-f8v4z" event={"ID":"02704747-24a5-4961-abac-7651fadd04fb","Type":"ContainerDied","Data":"895edac29eef4f0dd0e7b9e6f9fec7b44e4891783a9b25bee5ab03f2cf0662e7"}
Sep 10 19:45:22.822533 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:22.822476    2564 scope.go:117] "RemoveContainer" containerID="e79feaa0aea0e5cb31ad55d1cd653c8bf014ccdf4162afa3491f281a848c70a3"
Sep 10 19:45:22.823859 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:22.823832    2564 generic.go:356] "Generic (PLEG): container finished" podID="f85c24d6-df13-4f97-afdc-767ccf51a2a5" containerID="fa335220bda9a8133d8a068ec413ef71457d44aa955651c57df399b0e8c87c26" exitCode=0
Sep 10 19:45:22.823966 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:22.823871    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-63ed30-684b99ccb6-vgplv" event={"ID":"f85c24d6-df13-4f97-afdc-767ccf51a2a5","Type":"ContainerDied","Data":"fa335220bda9a8133d8a068ec413ef71457d44aa955651c57df399b0e8c87c26"}
Sep 10 19:45:22.823966 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:22.823889    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-63ed30-684b99ccb6-vgplv"
Sep 10 19:45:22.823966 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:22.823913    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-63ed30-684b99ccb6-vgplv" event={"ID":"f85c24d6-df13-4f97-afdc-767ccf51a2a5","Type":"ContainerDied","Data":"31011cdd519be2363710319b54b1cb34ef35baa9874a6f318705001f8e19285d"}
Sep 10 19:45:22.833557 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:22.833533    2564 scope.go:117] "RemoveContainer" containerID="e79feaa0aea0e5cb31ad55d1cd653c8bf014ccdf4162afa3491f281a848c70a3"
Sep 10 19:45:22.833841 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:45:22.833823    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e79feaa0aea0e5cb31ad55d1cd653c8bf014ccdf4162afa3491f281a848c70a3\": container with ID starting with e79feaa0aea0e5cb31ad55d1cd653c8bf014ccdf4162afa3491f281a848c70a3 not found: ID does not exist" containerID="e79feaa0aea0e5cb31ad55d1cd653c8bf014ccdf4162afa3491f281a848c70a3"
Sep 10 19:45:22.833917 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:22.833849    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e79feaa0aea0e5cb31ad55d1cd653c8bf014ccdf4162afa3491f281a848c70a3"} err="failed to get container status \"e79feaa0aea0e5cb31ad55d1cd653c8bf014ccdf4162afa3491f281a848c70a3\": rpc error: code = NotFound desc = could not find container \"e79feaa0aea0e5cb31ad55d1cd653c8bf014ccdf4162afa3491f281a848c70a3\": container with ID starting with e79feaa0aea0e5cb31ad55d1cd653c8bf014ccdf4162afa3491f281a848c70a3 not found: ID does not exist"
Sep 10 19:45:22.833917 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:22.833869    2564 scope.go:117] "RemoveContainer" containerID="fa335220bda9a8133d8a068ec413ef71457d44aa955651c57df399b0e8c87c26"
Sep 10 19:45:22.841645 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:22.841612    2564 scope.go:117] "RemoveContainer" containerID="fa335220bda9a8133d8a068ec413ef71457d44aa955651c57df399b0e8c87c26"
Sep 10 19:45:22.841911 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:45:22.841895    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa335220bda9a8133d8a068ec413ef71457d44aa955651c57df399b0e8c87c26\": container with ID starting with fa335220bda9a8133d8a068ec413ef71457d44aa955651c57df399b0e8c87c26 not found: ID does not exist" containerID="fa335220bda9a8133d8a068ec413ef71457d44aa955651c57df399b0e8c87c26"
Sep 10 19:45:22.841957 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:22.841918    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa335220bda9a8133d8a068ec413ef71457d44aa955651c57df399b0e8c87c26"} err="failed to get container status \"fa335220bda9a8133d8a068ec413ef71457d44aa955651c57df399b0e8c87c26\": rpc error: code = NotFound desc = could not find container \"fa335220bda9a8133d8a068ec413ef71457d44aa955651c57df399b0e8c87c26\": container with ID starting with fa335220bda9a8133d8a068ec413ef71457d44aa955651c57df399b0e8c87c26 not found: ID does not exist"
Sep 10 19:45:22.849474 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:22.849451    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-63ed30-794f98b679-f8v4z"]
Sep 10 19:45:22.853749 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:22.853728    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-63ed30-794f98b679-f8v4z"]
Sep 10 19:45:22.865383 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:22.865361    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-63ed30-684b99ccb6-vgplv"]
Sep 10 19:45:22.873211 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:22.873189    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-63ed30-684b99ccb6-vgplv"]
Sep 10 19:45:22.949674 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:22.949643    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="02704747-24a5-4961-abac-7651fadd04fb" path="/var/lib/kubelet/pods/02704747-24a5-4961-abac-7651fadd04fb/volumes"
Sep 10 19:45:22.950029 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:22.950015    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66eb2786-c1ff-4023-8575-b8fc2a5cb48d" path="/var/lib/kubelet/pods/66eb2786-c1ff-4023-8575-b8fc2a5cb48d/volumes"
Sep 10 19:45:22.950260 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:22.950249    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e425a113-9d02-4daa-ae4d-79d64eb193a1" path="/var/lib/kubelet/pods/e425a113-9d02-4daa-ae4d-79d64eb193a1/volumes"
Sep 10 19:45:22.950523 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:22.950512    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f85c24d6-df13-4f97-afdc-767ccf51a2a5" path="/var/lib/kubelet/pods/f85c24d6-df13-4f97-afdc-767ccf51a2a5/volumes"
Sep 10 19:45:27.514017 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:27.513981    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:45:28.849670 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:45:28.849610    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:04.244747 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:04.242175    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:05.014357 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:05.014300    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:11.432335 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:11.432300    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:11.487174 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:11.487135    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:11.715951 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:11.715846    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg"]
Sep 10 19:46:11.716132 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:11.716076    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg" podUID="d37ac8f3-e842-452e-8978-36cae1bf58e6" containerName="istio-proxy" containerID="cri-o://c3bf4b5c44d1f71e56c3098ff55a3bc26428089aa1a66a6bb18f654a64dd15b3" gracePeriod=30
Sep 10 19:46:11.728009 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:11.727981    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:12.115490 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:12.115456    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-544dea-qz2ph"]
Sep 10 19:46:12.115934 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:12.115919    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="646f01f5-e49a-43fd-8bc8-244e65ac9f90" containerName="istio-proxy"
Sep 10 19:46:12.115934 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:12.115935    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="646f01f5-e49a-43fd-8bc8-244e65ac9f90" containerName="istio-proxy"
Sep 10 19:46:12.116029 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:12.115943    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66eb2786-c1ff-4023-8575-b8fc2a5cb48d" containerName="revoke-keys"
Sep 10 19:46:12.116029 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:12.115949    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="66eb2786-c1ff-4023-8575-b8fc2a5cb48d" containerName="revoke-keys"
Sep 10 19:46:12.116029 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:12.115962    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02704747-24a5-4961-abac-7651fadd04fb" containerName="payload-processing"
Sep 10 19:46:12.116029 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:12.115968    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="02704747-24a5-4961-abac-7651fadd04fb" containerName="payload-processing"
Sep 10 19:46:12.116029 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:12.115976    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f85c24d6-df13-4f97-afdc-767ccf51a2a5" containerName="payload-pre-processing"
Sep 10 19:46:12.116029 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:12.115982    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f85c24d6-df13-4f97-afdc-767ccf51a2a5" containerName="payload-pre-processing"
Sep 10 19:46:12.116239 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:12.116038    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="02704747-24a5-4961-abac-7651fadd04fb" containerName="payload-processing"
Sep 10 19:46:12.116239 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:12.116044    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f85c24d6-df13-4f97-afdc-767ccf51a2a5" containerName="payload-pre-processing"
Sep 10 19:46:12.116239 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:12.116053    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="646f01f5-e49a-43fd-8bc8-244e65ac9f90" containerName="istio-proxy"
Sep 10 19:46:12.116239 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:12.116062    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="66eb2786-c1ff-4023-8575-b8fc2a5cb48d" containerName="revoke-keys"
Sep 10 19:46:12.116239 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:12.116075    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="646f01f5-e49a-43fd-8bc8-244e65ac9f90" containerName="istio-proxy"
Sep 10 19:46:12.119666 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:12.119642    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-544dea-qz2ph"
Sep 10 19:46:12.122223 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:12.122198    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tjmqn\""
Sep 10 19:46:12.122361 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:12.122197    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 19:46:12.127347 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:12.127316    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-544dea-qz2ph"]
Sep 10 19:46:12.225921 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:12.225875    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:12.260740 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:12.260698    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2cb86ad3-b88d-4fda-a827-64e129a887c3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-544dea-qz2ph\" (UID: \"2cb86ad3-b88d-4fda-a827-64e129a887c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-544dea-qz2ph"
Sep 10 19:46:12.362037 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:12.361977    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2cb86ad3-b88d-4fda-a827-64e129a887c3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-544dea-qz2ph\" (UID: \"2cb86ad3-b88d-4fda-a827-64e129a887c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-544dea-qz2ph"
Sep 10 19:46:12.362965 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:12.362933    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2cb86ad3-b88d-4fda-a827-64e129a887c3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-544dea-qz2ph\" (UID: \"2cb86ad3-b88d-4fda-a827-64e129a887c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-544dea-qz2ph"
Sep 10 19:46:12.434831 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:12.434740    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-544dea-qz2ph"
Sep 10 19:46:12.717560 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:12.712689    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-544dea-qz2ph"]
Sep 10 19:46:13.135810 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:13.135769    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-544dea-qz2ph" event={"ID":"2cb86ad3-b88d-4fda-a827-64e129a887c3","Type":"ContainerStarted","Data":"f101030fb3affd3faae4bc9daced684d89b049a405f7d31d33ef3ae54b6d6891"}
Sep 10 19:46:13.136027 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:13.135865    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-544dea-qz2ph" event={"ID":"2cb86ad3-b88d-4fda-a827-64e129a887c3","Type":"ContainerStarted","Data":"6c7cdb9993a5170c09100caa95734f80987250990c77dd69717f18e85adaa582"}
Sep 10 19:46:13.153204 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:13.153148    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-544dea-qz2ph" podStartSLOduration=1.153129797 podStartE2EDuration="1.153129797s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:46:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:46:13.152133787 +0000 UTC m=+1828.753041029" watchObservedRunningTime="2026-09-10 19:46:13.153129797 +0000 UTC m=+1828.754037036"
Sep 10 19:46:14.143550 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:14.143514    2564 generic.go:356] "Generic (PLEG): container finished" podID="2cb86ad3-b88d-4fda-a827-64e129a887c3" containerID="f101030fb3affd3faae4bc9daced684d89b049a405f7d31d33ef3ae54b6d6891" exitCode=0
Sep 10 19:46:14.143975 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:14.143588    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-544dea-qz2ph" event={"ID":"2cb86ad3-b88d-4fda-a827-64e129a887c3","Type":"ContainerDied","Data":"f101030fb3affd3faae4bc9daced684d89b049a405f7d31d33ef3ae54b6d6891"}
Sep 10 19:46:15.281216 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:15.281188    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-544dea-qz2ph"
Sep 10 19:46:15.294794 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:15.294767    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2cb86ad3-b88d-4fda-a827-64e129a887c3-maas-api-service-ca\") pod \"2cb86ad3-b88d-4fda-a827-64e129a887c3\" (UID: \"2cb86ad3-b88d-4fda-a827-64e129a887c3\") "
Sep 10 19:46:15.295097 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:15.295075    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2cb86ad3-b88d-4fda-a827-64e129a887c3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2cb86ad3-b88d-4fda-a827-64e129a887c3" (UID: "2cb86ad3-b88d-4fda-a827-64e129a887c3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:46:15.395799 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:15.395768    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2cb86ad3-b88d-4fda-a827-64e129a887c3-maas-api-service-ca\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:46:15.652828 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:15.652788    2564 patch_prober.go:28] interesting pod/e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 19:46:15.653013 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:15.652850    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg" podUID="d37ac8f3-e842-452e-8978-36cae1bf58e6" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 19:46:16.157561 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:16.157530    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-544dea-qz2ph"
Sep 10 19:46:16.157756 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:16.157560    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-544dea-qz2ph" event={"ID":"2cb86ad3-b88d-4fda-a827-64e129a887c3","Type":"ContainerDied","Data":"6c7cdb9993a5170c09100caa95734f80987250990c77dd69717f18e85adaa582"}
Sep 10 19:46:16.157756 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:16.157609    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6c7cdb9993a5170c09100caa95734f80987250990c77dd69717f18e85adaa582"
Sep 10 19:46:16.973500 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:16.973471    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg"
Sep 10 19:46:17.013923 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:17.013895    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d37ac8f3-e842-452e-8978-36cae1bf58e6-workload-socket\") pod \"d37ac8f3-e842-452e-8978-36cae1bf58e6\" (UID: \"d37ac8f3-e842-452e-8978-36cae1bf58e6\") "
Sep 10 19:46:17.014095 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:17.013940    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d37ac8f3-e842-452e-8978-36cae1bf58e6-istio-data\") pod \"d37ac8f3-e842-452e-8978-36cae1bf58e6\" (UID: \"d37ac8f3-e842-452e-8978-36cae1bf58e6\") "
Sep 10 19:46:17.014095 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:17.013963    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d37ac8f3-e842-452e-8978-36cae1bf58e6-istio-token\") pod \"d37ac8f3-e842-452e-8978-36cae1bf58e6\" (UID: \"d37ac8f3-e842-452e-8978-36cae1bf58e6\") "
Sep 10 19:46:17.014095 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:17.013994    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d37ac8f3-e842-452e-8978-36cae1bf58e6-istio-envoy\") pod \"d37ac8f3-e842-452e-8978-36cae1bf58e6\" (UID: \"d37ac8f3-e842-452e-8978-36cae1bf58e6\") "
Sep 10 19:46:17.014095 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:17.014023    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d37ac8f3-e842-452e-8978-36cae1bf58e6-istiod-ca-cert\") pod \"d37ac8f3-e842-452e-8978-36cae1bf58e6\" (UID: \"d37ac8f3-e842-452e-8978-36cae1bf58e6\") "
Sep 10 19:46:17.014319 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:17.014095    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d37ac8f3-e842-452e-8978-36cae1bf58e6-istio-podinfo\") pod \"d37ac8f3-e842-452e-8978-36cae1bf58e6\" (UID: \"d37ac8f3-e842-452e-8978-36cae1bf58e6\") "
Sep 10 19:46:17.014319 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:17.014134    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d37ac8f3-e842-452e-8978-36cae1bf58e6-workload-certs\") pod \"d37ac8f3-e842-452e-8978-36cae1bf58e6\" (UID: \"d37ac8f3-e842-452e-8978-36cae1bf58e6\") "
Sep 10 19:46:17.014319 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:17.014174    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d37ac8f3-e842-452e-8978-36cae1bf58e6-credential-socket\") pod \"d37ac8f3-e842-452e-8978-36cae1bf58e6\" (UID: \"d37ac8f3-e842-452e-8978-36cae1bf58e6\") "
Sep 10 19:46:17.014319 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:17.014199    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jvfqk\" (UniqueName: \"kubernetes.io/projected/d37ac8f3-e842-452e-8978-36cae1bf58e6-kube-api-access-jvfqk\") pod \"d37ac8f3-e842-452e-8978-36cae1bf58e6\" (UID: \"d37ac8f3-e842-452e-8978-36cae1bf58e6\") "
Sep 10 19:46:17.014319 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:17.014218    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d37ac8f3-e842-452e-8978-36cae1bf58e6-istio-data" (OuterVolumeSpecName: "istio-data") pod "d37ac8f3-e842-452e-8978-36cae1bf58e6" (UID: "d37ac8f3-e842-452e-8978-36cae1bf58e6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:46:17.014319 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:17.014240    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d37ac8f3-e842-452e-8978-36cae1bf58e6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d37ac8f3-e842-452e-8978-36cae1bf58e6" (UID: "d37ac8f3-e842-452e-8978-36cae1bf58e6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:46:17.014669 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:17.014473    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d37ac8f3-e842-452e-8978-36cae1bf58e6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d37ac8f3-e842-452e-8978-36cae1bf58e6" (UID: "d37ac8f3-e842-452e-8978-36cae1bf58e6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:46:17.014669 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:17.014493    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d37ac8f3-e842-452e-8978-36cae1bf58e6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d37ac8f3-e842-452e-8978-36cae1bf58e6" (UID: "d37ac8f3-e842-452e-8978-36cae1bf58e6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:46:17.014669 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:17.014514    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d37ac8f3-e842-452e-8978-36cae1bf58e6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d37ac8f3-e842-452e-8978-36cae1bf58e6" (UID: "d37ac8f3-e842-452e-8978-36cae1bf58e6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:46:17.014832 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:17.014686    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d37ac8f3-e842-452e-8978-36cae1bf58e6-istiod-ca-cert\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:46:17.014832 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:17.014703    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d37ac8f3-e842-452e-8978-36cae1bf58e6-workload-certs\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:46:17.014832 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:17.014716    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d37ac8f3-e842-452e-8978-36cae1bf58e6-credential-socket\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:46:17.014832 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:17.014729    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d37ac8f3-e842-452e-8978-36cae1bf58e6-workload-socket\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:46:17.014832 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:17.014742    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d37ac8f3-e842-452e-8978-36cae1bf58e6-istio-data\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:46:17.016465 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:17.016430    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d37ac8f3-e842-452e-8978-36cae1bf58e6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d37ac8f3-e842-452e-8978-36cae1bf58e6" (UID: "d37ac8f3-e842-452e-8978-36cae1bf58e6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 19:46:17.016582 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:17.016464    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d37ac8f3-e842-452e-8978-36cae1bf58e6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d37ac8f3-e842-452e-8978-36cae1bf58e6" (UID: "d37ac8f3-e842-452e-8978-36cae1bf58e6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:46:17.016582 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:17.016500    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d37ac8f3-e842-452e-8978-36cae1bf58e6-kube-api-access-jvfqk" (OuterVolumeSpecName: "kube-api-access-jvfqk") pod "d37ac8f3-e842-452e-8978-36cae1bf58e6" (UID: "d37ac8f3-e842-452e-8978-36cae1bf58e6"). InnerVolumeSpecName "kube-api-access-jvfqk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:46:17.016790 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:17.016775    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d37ac8f3-e842-452e-8978-36cae1bf58e6-istio-token" (OuterVolumeSpecName: "istio-token") pod "d37ac8f3-e842-452e-8978-36cae1bf58e6" (UID: "d37ac8f3-e842-452e-8978-36cae1bf58e6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:46:17.116331 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:17.116292    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d37ac8f3-e842-452e-8978-36cae1bf58e6-istio-token\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:46:17.116331 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:17.116322    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d37ac8f3-e842-452e-8978-36cae1bf58e6-istio-envoy\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:46:17.116331 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:17.116335    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d37ac8f3-e842-452e-8978-36cae1bf58e6-istio-podinfo\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:46:17.116575 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:17.116349    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jvfqk\" (UniqueName: \"kubernetes.io/projected/d37ac8f3-e842-452e-8978-36cae1bf58e6-kube-api-access-jvfqk\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:46:17.165178 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:17.165142    2564 generic.go:356] "Generic (PLEG): container finished" podID="d37ac8f3-e842-452e-8978-36cae1bf58e6" containerID="c3bf4b5c44d1f71e56c3098ff55a3bc26428089aa1a66a6bb18f654a64dd15b3" exitCode=0
Sep 10 19:46:17.165363 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:17.165193    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg"
Sep 10 19:46:17.165363 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:17.165229    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg" event={"ID":"d37ac8f3-e842-452e-8978-36cae1bf58e6","Type":"ContainerDied","Data":"c3bf4b5c44d1f71e56c3098ff55a3bc26428089aa1a66a6bb18f654a64dd15b3"}
Sep 10 19:46:17.165363 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:17.165279    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg" event={"ID":"d37ac8f3-e842-452e-8978-36cae1bf58e6","Type":"ContainerDied","Data":"caae034f5d31b0939c0f26e41ca7a7fabab7c0a1bcf0f67c26dde220e2bdb8f3"}
Sep 10 19:46:17.165363 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:17.165301    2564 scope.go:117] "RemoveContainer" containerID="c3bf4b5c44d1f71e56c3098ff55a3bc26428089aa1a66a6bb18f654a64dd15b3"
Sep 10 19:46:17.174920 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:17.174903    2564 scope.go:117] "RemoveContainer" containerID="23802ce943e702cf07511119b2890e683502a5e95a5d65bb9d8724018d936781"
Sep 10 19:46:17.183354 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:17.183336    2564 scope.go:117] "RemoveContainer" containerID="c3bf4b5c44d1f71e56c3098ff55a3bc26428089aa1a66a6bb18f654a64dd15b3"
Sep 10 19:46:17.183647 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:46:17.183608    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3bf4b5c44d1f71e56c3098ff55a3bc26428089aa1a66a6bb18f654a64dd15b3\": container with ID starting with c3bf4b5c44d1f71e56c3098ff55a3bc26428089aa1a66a6bb18f654a64dd15b3 not found: ID does not exist" containerID="c3bf4b5c44d1f71e56c3098ff55a3bc26428089aa1a66a6bb18f654a64dd15b3"
Sep 10 19:46:17.183763 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:17.183649    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3bf4b5c44d1f71e56c3098ff55a3bc26428089aa1a66a6bb18f654a64dd15b3"} err="failed to get container status \"c3bf4b5c44d1f71e56c3098ff55a3bc26428089aa1a66a6bb18f654a64dd15b3\": rpc error: code = NotFound desc = could not find container \"c3bf4b5c44d1f71e56c3098ff55a3bc26428089aa1a66a6bb18f654a64dd15b3\": container with ID starting with c3bf4b5c44d1f71e56c3098ff55a3bc26428089aa1a66a6bb18f654a64dd15b3 not found: ID does not exist"
Sep 10 19:46:17.183763 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:17.183665    2564 scope.go:117] "RemoveContainer" containerID="23802ce943e702cf07511119b2890e683502a5e95a5d65bb9d8724018d936781"
Sep 10 19:46:17.183928 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:46:17.183912    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"23802ce943e702cf07511119b2890e683502a5e95a5d65bb9d8724018d936781\": container with ID starting with 23802ce943e702cf07511119b2890e683502a5e95a5d65bb9d8724018d936781 not found: ID does not exist" containerID="23802ce943e702cf07511119b2890e683502a5e95a5d65bb9d8724018d936781"
Sep 10 19:46:17.183972 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:17.183933    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23802ce943e702cf07511119b2890e683502a5e95a5d65bb9d8724018d936781"} err="failed to get container status \"23802ce943e702cf07511119b2890e683502a5e95a5d65bb9d8724018d936781\": rpc error: code = NotFound desc = could not find container \"23802ce943e702cf07511119b2890e683502a5e95a5d65bb9d8724018d936781\": container with ID starting with 23802ce943e702cf07511119b2890e683502a5e95a5d65bb9d8724018d936781 not found: ID does not exist"
Sep 10 19:46:17.193139 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:17.193108    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg"]
Sep 10 19:46:17.195876 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:17.195853    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-63ed30-openshift-default-bcbf79db4-cn5tg"]
Sep 10 19:46:18.948793 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:18.948758    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d37ac8f3-e842-452e-8978-36cae1bf58e6" path="/var/lib/kubelet/pods/d37ac8f3-e842-452e-8978-36cae1bf58e6/volumes"
Sep 10 19:46:22.621432 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:22.621388    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:22.629238 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:22.629210    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-544dea-29817825-5z6l5"]
Sep 10 19:46:22.631609 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:22.631584    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-544dea-29817825-5z6l5"]
Sep 10 19:46:22.708112 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:22.708073    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:22.768430 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:22.768393    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:22.852932 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:22.852885    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-544dea-qz2ph"]
Sep 10 19:46:22.854991 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:22.854965    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-544dea-qz2ph"]
Sep 10 19:46:22.855972 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:22.855946    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:22.916133 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:22.916042    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:22.949588 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:22.949560    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2cb86ad3-b88d-4fda-a827-64e129a887c3" path="/var/lib/kubelet/pods/2cb86ad3-b88d-4fda-a827-64e129a887c3/volumes"
Sep 10 19:46:22.949911 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:22.949897    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c07ad6e5-ee85-4b1f-abfd-2511027b2383" path="/var/lib/kubelet/pods/c07ad6e5-ee85-4b1f-abfd-2511027b2383/volumes"
Sep 10 19:46:23.243047 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:23.242955    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:29.405887 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:29.405851    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:29.472002 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:29.471960    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:29.652862 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:29.652825    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-544dea-openshift-default-85985454b-qflgf"]
Sep 10 19:46:29.653141 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:29.653110    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-544dea-openshift-default-85985454b-qflgf" podUID="7820d541-2f9a-41a7-bf74-b0e8b4161d56" containerName="istio-proxy" containerID="cri-o://ed5ab0d5bb026b670029c5924f013d29775e467cac0cf217413137ca556dddb8" gracePeriod=30
Sep 10 19:46:29.675020 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:29.674929    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:30.005902 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:30.001175    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-28d250-4ddmg"]
Sep 10 19:46:30.005902 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:30.002196    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d37ac8f3-e842-452e-8978-36cae1bf58e6" containerName="istio-proxy"
Sep 10 19:46:30.005902 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:30.002218    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d37ac8f3-e842-452e-8978-36cae1bf58e6" containerName="istio-proxy"
Sep 10 19:46:30.005902 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:30.002238    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2cb86ad3-b88d-4fda-a827-64e129a887c3" containerName="revoke-keys"
Sep 10 19:46:30.005902 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:30.002249    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2cb86ad3-b88d-4fda-a827-64e129a887c3" containerName="revoke-keys"
Sep 10 19:46:30.005902 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:30.002278    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d37ac8f3-e842-452e-8978-36cae1bf58e6" containerName="istio-proxy"
Sep 10 19:46:30.005902 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:30.002289    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d37ac8f3-e842-452e-8978-36cae1bf58e6" containerName="istio-proxy"
Sep 10 19:46:30.005902 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:30.002301    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="646f01f5-e49a-43fd-8bc8-244e65ac9f90" containerName="istio-proxy"
Sep 10 19:46:30.005902 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:30.002310    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="646f01f5-e49a-43fd-8bc8-244e65ac9f90" containerName="istio-proxy"
Sep 10 19:46:30.005902 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:30.002522    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2cb86ad3-b88d-4fda-a827-64e129a887c3" containerName="revoke-keys"
Sep 10 19:46:30.005902 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:30.002545    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d37ac8f3-e842-452e-8978-36cae1bf58e6" containerName="istio-proxy"
Sep 10 19:46:30.005902 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:30.002563    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d37ac8f3-e842-452e-8978-36cae1bf58e6" containerName="istio-proxy"
Sep 10 19:46:30.007727 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:30.007700    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-28d250-4ddmg"
Sep 10 19:46:30.010138 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:30.010118    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 19:46:30.010519 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:30.010197    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tjmqn\""
Sep 10 19:46:30.010519 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:30.010399    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-28d250-4ddmg"]
Sep 10 19:46:30.044945 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:30.044911    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc7a4026-0eca-46de-a1bf-7e3ac3742eb1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-28d250-4ddmg\" (UID: \"fc7a4026-0eca-46de-a1bf-7e3ac3742eb1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-28d250-4ddmg"
Sep 10 19:46:30.146003 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:30.145968    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc7a4026-0eca-46de-a1bf-7e3ac3742eb1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-28d250-4ddmg\" (UID: \"fc7a4026-0eca-46de-a1bf-7e3ac3742eb1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-28d250-4ddmg"
Sep 10 19:46:30.146660 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:30.146611    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc7a4026-0eca-46de-a1bf-7e3ac3742eb1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-28d250-4ddmg\" (UID: \"fc7a4026-0eca-46de-a1bf-7e3ac3742eb1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-28d250-4ddmg"
Sep 10 19:46:30.245412 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:30.245373    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:30.319936 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:30.319848    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-28d250-4ddmg"
Sep 10 19:46:30.593690 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:30.593660    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-28d250-4ddmg"]
Sep 10 19:46:30.595879 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:46:30.595852    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc7a4026_0eca_46de_a1bf_7e3ac3742eb1.slice/crio-d2357fe1a8b8b003ae2b582b705cd46e96d2d25193068b39e1e5157d6280f3bc WatchSource:0}: Error finding container d2357fe1a8b8b003ae2b582b705cd46e96d2d25193068b39e1e5157d6280f3bc: Status 404 returned error can't find the container with id d2357fe1a8b8b003ae2b582b705cd46e96d2d25193068b39e1e5157d6280f3bc
Sep 10 19:46:30.981359 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:30.981323    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:31.248733 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:31.248696    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-28d250-4ddmg" event={"ID":"fc7a4026-0eca-46de-a1bf-7e3ac3742eb1","Type":"ContainerStarted","Data":"4ae38d4fdddb58bcfa5ca609b5ddc8f1445024f689ce6c33cf4464b05191003c"}
Sep 10 19:46:31.248881 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:31.248762    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-28d250-4ddmg" event={"ID":"fc7a4026-0eca-46de-a1bf-7e3ac3742eb1","Type":"ContainerStarted","Data":"d2357fe1a8b8b003ae2b582b705cd46e96d2d25193068b39e1e5157d6280f3bc"}
Sep 10 19:46:31.264374 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:31.264326    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-28d250-4ddmg" podStartSLOduration=2.26430818 podStartE2EDuration="2.26430818s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:46:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:46:31.263459754 +0000 UTC m=+1846.864366995" watchObservedRunningTime="2026-09-10 19:46:31.26430818 +0000 UTC m=+1846.865215420"
Sep 10 19:46:31.706352 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:31.706243    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:32.256393 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:32.256354    2564 generic.go:356] "Generic (PLEG): container finished" podID="fc7a4026-0eca-46de-a1bf-7e3ac3742eb1" containerID="4ae38d4fdddb58bcfa5ca609b5ddc8f1445024f689ce6c33cf4464b05191003c" exitCode=0
Sep 10 19:46:32.256565 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:32.256437    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-28d250-4ddmg" event={"ID":"fc7a4026-0eca-46de-a1bf-7e3ac3742eb1","Type":"ContainerDied","Data":"4ae38d4fdddb58bcfa5ca609b5ddc8f1445024f689ce6c33cf4464b05191003c"}
Sep 10 19:46:33.402157 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:33.402131    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-28d250-4ddmg"
Sep 10 19:46:33.478246 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:33.478214    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc7a4026-0eca-46de-a1bf-7e3ac3742eb1-maas-api-service-ca\") pod \"fc7a4026-0eca-46de-a1bf-7e3ac3742eb1\" (UID: \"fc7a4026-0eca-46de-a1bf-7e3ac3742eb1\") "
Sep 10 19:46:33.478567 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:33.478537    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc7a4026-0eca-46de-a1bf-7e3ac3742eb1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fc7a4026-0eca-46de-a1bf-7e3ac3742eb1" (UID: "fc7a4026-0eca-46de-a1bf-7e3ac3742eb1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:46:33.579727 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:33.579650    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc7a4026-0eca-46de-a1bf-7e3ac3742eb1-maas-api-service-ca\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:46:34.051396 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:34.051357    2564 patch_prober.go:28] interesting pod/e2e-ipp-b-544dea-openshift-default-85985454b-qflgf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 19:46:34.051573 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:34.051414    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-544dea-openshift-default-85985454b-qflgf" podUID="7820d541-2f9a-41a7-bf74-b0e8b4161d56" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 19:46:34.272544 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:34.272514    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-28d250-4ddmg"
Sep 10 19:46:34.272755 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:34.272512    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-28d250-4ddmg" event={"ID":"fc7a4026-0eca-46de-a1bf-7e3ac3742eb1","Type":"ContainerDied","Data":"d2357fe1a8b8b003ae2b582b705cd46e96d2d25193068b39e1e5157d6280f3bc"}
Sep 10 19:46:34.272755 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:34.272661    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d2357fe1a8b8b003ae2b582b705cd46e96d2d25193068b39e1e5157d6280f3bc"
Sep 10 19:46:34.907321 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:34.907299    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-544dea-openshift-default-85985454b-qflgf"
Sep 10 19:46:34.996203 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:34.996126    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7820d541-2f9a-41a7-bf74-b0e8b4161d56-workload-socket\") pod \"7820d541-2f9a-41a7-bf74-b0e8b4161d56\" (UID: \"7820d541-2f9a-41a7-bf74-b0e8b4161d56\") "
Sep 10 19:46:34.996203 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:34.996160    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7820d541-2f9a-41a7-bf74-b0e8b4161d56-istio-token\") pod \"7820d541-2f9a-41a7-bf74-b0e8b4161d56\" (UID: \"7820d541-2f9a-41a7-bf74-b0e8b4161d56\") "
Sep 10 19:46:34.996441 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:34.996214    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7820d541-2f9a-41a7-bf74-b0e8b4161d56-workload-certs\") pod \"7820d541-2f9a-41a7-bf74-b0e8b4161d56\" (UID: \"7820d541-2f9a-41a7-bf74-b0e8b4161d56\") "
Sep 10 19:46:34.996441 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:34.996237    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7820d541-2f9a-41a7-bf74-b0e8b4161d56-istio-data\") pod \"7820d541-2f9a-41a7-bf74-b0e8b4161d56\" (UID: \"7820d541-2f9a-41a7-bf74-b0e8b4161d56\") "
Sep 10 19:46:34.996441 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:34.996272    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7820d541-2f9a-41a7-bf74-b0e8b4161d56-istiod-ca-cert\") pod \"7820d541-2f9a-41a7-bf74-b0e8b4161d56\" (UID: \"7820d541-2f9a-41a7-bf74-b0e8b4161d56\") "
Sep 10 19:46:34.996441 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:34.996321    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7820d541-2f9a-41a7-bf74-b0e8b4161d56-istio-envoy\") pod \"7820d541-2f9a-41a7-bf74-b0e8b4161d56\" (UID: \"7820d541-2f9a-41a7-bf74-b0e8b4161d56\") "
Sep 10 19:46:34.996441 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:34.996362    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7820d541-2f9a-41a7-bf74-b0e8b4161d56-credential-socket\") pod \"7820d541-2f9a-41a7-bf74-b0e8b4161d56\" (UID: \"7820d541-2f9a-41a7-bf74-b0e8b4161d56\") "
Sep 10 19:46:34.996441 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:34.996392    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ccs96\" (UniqueName: \"kubernetes.io/projected/7820d541-2f9a-41a7-bf74-b0e8b4161d56-kube-api-access-ccs96\") pod \"7820d541-2f9a-41a7-bf74-b0e8b4161d56\" (UID: \"7820d541-2f9a-41a7-bf74-b0e8b4161d56\") "
Sep 10 19:46:34.996441 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:34.996428    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7820d541-2f9a-41a7-bf74-b0e8b4161d56-istio-podinfo\") pod \"7820d541-2f9a-41a7-bf74-b0e8b4161d56\" (UID: \"7820d541-2f9a-41a7-bf74-b0e8b4161d56\") "
Sep 10 19:46:34.996441 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:34.996426    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7820d541-2f9a-41a7-bf74-b0e8b4161d56-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7820d541-2f9a-41a7-bf74-b0e8b4161d56" (UID: "7820d541-2f9a-41a7-bf74-b0e8b4161d56"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:46:34.996864 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:34.996551    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7820d541-2f9a-41a7-bf74-b0e8b4161d56-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7820d541-2f9a-41a7-bf74-b0e8b4161d56" (UID: "7820d541-2f9a-41a7-bf74-b0e8b4161d56"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:46:34.996864 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:34.996569    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7820d541-2f9a-41a7-bf74-b0e8b4161d56-istio-data" (OuterVolumeSpecName: "istio-data") pod "7820d541-2f9a-41a7-bf74-b0e8b4161d56" (UID: "7820d541-2f9a-41a7-bf74-b0e8b4161d56"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:46:34.996864 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:34.996688    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7820d541-2f9a-41a7-bf74-b0e8b4161d56-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7820d541-2f9a-41a7-bf74-b0e8b4161d56" (UID: "7820d541-2f9a-41a7-bf74-b0e8b4161d56"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:46:34.996864 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:34.996789    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7820d541-2f9a-41a7-bf74-b0e8b4161d56-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7820d541-2f9a-41a7-bf74-b0e8b4161d56" (UID: "7820d541-2f9a-41a7-bf74-b0e8b4161d56"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:46:34.997135 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:34.996885    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7820d541-2f9a-41a7-bf74-b0e8b4161d56-credential-socket\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:46:34.997135 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:34.996903    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7820d541-2f9a-41a7-bf74-b0e8b4161d56-workload-socket\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:46:34.997135 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:34.996920    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7820d541-2f9a-41a7-bf74-b0e8b4161d56-workload-certs\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:46:34.997135 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:34.996933    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7820d541-2f9a-41a7-bf74-b0e8b4161d56-istio-data\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:46:34.997135 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:34.996945    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7820d541-2f9a-41a7-bf74-b0e8b4161d56-istiod-ca-cert\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:46:34.998675 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:34.998598    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7820d541-2f9a-41a7-bf74-b0e8b4161d56-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7820d541-2f9a-41a7-bf74-b0e8b4161d56" (UID: "7820d541-2f9a-41a7-bf74-b0e8b4161d56"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 19:46:34.998675 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:34.998656    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7820d541-2f9a-41a7-bf74-b0e8b4161d56-kube-api-access-ccs96" (OuterVolumeSpecName: "kube-api-access-ccs96") pod "7820d541-2f9a-41a7-bf74-b0e8b4161d56" (UID: "7820d541-2f9a-41a7-bf74-b0e8b4161d56"). InnerVolumeSpecName "kube-api-access-ccs96". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:46:34.998884 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:34.998860    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7820d541-2f9a-41a7-bf74-b0e8b4161d56-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7820d541-2f9a-41a7-bf74-b0e8b4161d56" (UID: "7820d541-2f9a-41a7-bf74-b0e8b4161d56"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:46:34.998984 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:34.998922    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7820d541-2f9a-41a7-bf74-b0e8b4161d56-istio-token" (OuterVolumeSpecName: "istio-token") pod "7820d541-2f9a-41a7-bf74-b0e8b4161d56" (UID: "7820d541-2f9a-41a7-bf74-b0e8b4161d56"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:46:35.097752 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:35.097715    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7820d541-2f9a-41a7-bf74-b0e8b4161d56-istio-envoy\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:46:35.097752 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:35.097750    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ccs96\" (UniqueName: \"kubernetes.io/projected/7820d541-2f9a-41a7-bf74-b0e8b4161d56-kube-api-access-ccs96\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:46:35.097953 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:35.097766    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7820d541-2f9a-41a7-bf74-b0e8b4161d56-istio-podinfo\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:46:35.097953 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:35.097780    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7820d541-2f9a-41a7-bf74-b0e8b4161d56-istio-token\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:46:35.280358 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:35.280322    2564 generic.go:356] "Generic (PLEG): container finished" podID="7820d541-2f9a-41a7-bf74-b0e8b4161d56" containerID="ed5ab0d5bb026b670029c5924f013d29775e467cac0cf217413137ca556dddb8" exitCode=0
Sep 10 19:46:35.280533 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:35.280375    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-544dea-openshift-default-85985454b-qflgf"
Sep 10 19:46:35.280533 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:35.280403    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-544dea-openshift-default-85985454b-qflgf" event={"ID":"7820d541-2f9a-41a7-bf74-b0e8b4161d56","Type":"ContainerDied","Data":"ed5ab0d5bb026b670029c5924f013d29775e467cac0cf217413137ca556dddb8"}
Sep 10 19:46:35.280533 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:35.280453    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-544dea-openshift-default-85985454b-qflgf" event={"ID":"7820d541-2f9a-41a7-bf74-b0e8b4161d56","Type":"ContainerDied","Data":"6f72410c78079536f45bc33f7e04784917fdff34f4318409727102bffcf122c0"}
Sep 10 19:46:35.280533 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:35.280476    2564 scope.go:117] "RemoveContainer" containerID="ed5ab0d5bb026b670029c5924f013d29775e467cac0cf217413137ca556dddb8"
Sep 10 19:46:35.290436 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:35.290417    2564 scope.go:117] "RemoveContainer" containerID="2f976196109681dfe049e4ddb3dc83783db69106d7b959f5116ed6579e328824"
Sep 10 19:46:35.298814 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:35.298797    2564 scope.go:117] "RemoveContainer" containerID="ed5ab0d5bb026b670029c5924f013d29775e467cac0cf217413137ca556dddb8"
Sep 10 19:46:35.299103 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:46:35.299084    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed5ab0d5bb026b670029c5924f013d29775e467cac0cf217413137ca556dddb8\": container with ID starting with ed5ab0d5bb026b670029c5924f013d29775e467cac0cf217413137ca556dddb8 not found: ID does not exist" containerID="ed5ab0d5bb026b670029c5924f013d29775e467cac0cf217413137ca556dddb8"
Sep 10 19:46:35.299172 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:35.299115    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed5ab0d5bb026b670029c5924f013d29775e467cac0cf217413137ca556dddb8"} err="failed to get container status \"ed5ab0d5bb026b670029c5924f013d29775e467cac0cf217413137ca556dddb8\": rpc error: code = NotFound desc = could not find container \"ed5ab0d5bb026b670029c5924f013d29775e467cac0cf217413137ca556dddb8\": container with ID starting with ed5ab0d5bb026b670029c5924f013d29775e467cac0cf217413137ca556dddb8 not found: ID does not exist"
Sep 10 19:46:35.299172 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:35.299139    2564 scope.go:117] "RemoveContainer" containerID="2f976196109681dfe049e4ddb3dc83783db69106d7b959f5116ed6579e328824"
Sep 10 19:46:35.299427 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:46:35.299412    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f976196109681dfe049e4ddb3dc83783db69106d7b959f5116ed6579e328824\": container with ID starting with 2f976196109681dfe049e4ddb3dc83783db69106d7b959f5116ed6579e328824 not found: ID does not exist" containerID="2f976196109681dfe049e4ddb3dc83783db69106d7b959f5116ed6579e328824"
Sep 10 19:46:35.299470 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:35.299431    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f976196109681dfe049e4ddb3dc83783db69106d7b959f5116ed6579e328824"} err="failed to get container status \"2f976196109681dfe049e4ddb3dc83783db69106d7b959f5116ed6579e328824\": rpc error: code = NotFound desc = could not find container \"2f976196109681dfe049e4ddb3dc83783db69106d7b959f5116ed6579e328824\": container with ID starting with 2f976196109681dfe049e4ddb3dc83783db69106d7b959f5116ed6579e328824 not found: ID does not exist"
Sep 10 19:46:35.307371 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:35.307346    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-544dea-openshift-default-85985454b-qflgf"]
Sep 10 19:46:35.313495 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:35.313471    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-544dea-openshift-default-85985454b-qflgf"]
Sep 10 19:46:36.949170 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:36.949136    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7820d541-2f9a-41a7-bf74-b0e8b4161d56" path="/var/lib/kubelet/pods/7820d541-2f9a-41a7-bf74-b0e8b4161d56/volumes"
Sep 10 19:46:40.529116 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:40.529072    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:40.574910 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:40.574876    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-28d250-5d86c7b9b6-v766w"]
Sep 10 19:46:40.575426 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:40.575367    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-28d250-5d86c7b9b6-v766w" podUID="12bab5be-61db-4403-b133-064439213d04" containerName="payload-pre-processing" containerID="cri-o://4c3c97efcbdb091aa9fa2ff142d47f3a21254eccb249a218e19e874fad595f61" gracePeriod=30
Sep 10 19:46:40.808300 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:40.808218    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-28d250-4ddmg"]
Sep 10 19:46:40.812695 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:40.812666    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-28d250-4ddmg"]
Sep 10 19:46:40.848067 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:40.848043    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-28d250-5d86c7b9b6-v766w"
Sep 10 19:46:40.949576 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:40.949542    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc7a4026-0eca-46de-a1bf-7e3ac3742eb1" path="/var/lib/kubelet/pods/fc7a4026-0eca-46de-a1bf-7e3ac3742eb1/volumes"
Sep 10 19:46:40.956729 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:40.956707    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hl5ml\" (UniqueName: \"kubernetes.io/projected/12bab5be-61db-4403-b133-064439213d04-kube-api-access-hl5ml\") pod \"12bab5be-61db-4403-b133-064439213d04\" (UID: \"12bab5be-61db-4403-b133-064439213d04\") "
Sep 10 19:46:40.956848 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:40.956835    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12bab5be-61db-4403-b133-064439213d04-plugins-config-volume\") pod \"12bab5be-61db-4403-b133-064439213d04\" (UID: \"12bab5be-61db-4403-b133-064439213d04\") "
Sep 10 19:46:40.957183 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:40.957162    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12bab5be-61db-4403-b133-064439213d04-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "12bab5be-61db-4403-b133-064439213d04" (UID: "12bab5be-61db-4403-b133-064439213d04"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:46:40.958825 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:40.958805    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12bab5be-61db-4403-b133-064439213d04-kube-api-access-hl5ml" (OuterVolumeSpecName: "kube-api-access-hl5ml") pod "12bab5be-61db-4403-b133-064439213d04" (UID: "12bab5be-61db-4403-b133-064439213d04"). InnerVolumeSpecName "kube-api-access-hl5ml". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:46:41.057864 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:41.057817    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12bab5be-61db-4403-b133-064439213d04-plugins-config-volume\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:46:41.057864 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:41.057860    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hl5ml\" (UniqueName: \"kubernetes.io/projected/12bab5be-61db-4403-b133-064439213d04-kube-api-access-hl5ml\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:46:41.318705 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:41.318665    2564 generic.go:356] "Generic (PLEG): container finished" podID="12bab5be-61db-4403-b133-064439213d04" containerID="4c3c97efcbdb091aa9fa2ff142d47f3a21254eccb249a218e19e874fad595f61" exitCode=0
Sep 10 19:46:41.318705 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:41.318707    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-28d250-5d86c7b9b6-v766w"
Sep 10 19:46:41.318938 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:41.318750    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-28d250-5d86c7b9b6-v766w" event={"ID":"12bab5be-61db-4403-b133-064439213d04","Type":"ContainerDied","Data":"4c3c97efcbdb091aa9fa2ff142d47f3a21254eccb249a218e19e874fad595f61"}
Sep 10 19:46:41.318938 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:41.318804    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-28d250-5d86c7b9b6-v766w" event={"ID":"12bab5be-61db-4403-b133-064439213d04","Type":"ContainerDied","Data":"2808db91542f736c5991b8792fe20eb0a469055865580bbfe512a7955a327d52"}
Sep 10 19:46:41.318938 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:41.318827    2564 scope.go:117] "RemoveContainer" containerID="4c3c97efcbdb091aa9fa2ff142d47f3a21254eccb249a218e19e874fad595f61"
Sep 10 19:46:41.328487 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:41.328470    2564 scope.go:117] "RemoveContainer" containerID="4c3c97efcbdb091aa9fa2ff142d47f3a21254eccb249a218e19e874fad595f61"
Sep 10 19:46:41.328809 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:46:41.328790    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c3c97efcbdb091aa9fa2ff142d47f3a21254eccb249a218e19e874fad595f61\": container with ID starting with 4c3c97efcbdb091aa9fa2ff142d47f3a21254eccb249a218e19e874fad595f61 not found: ID does not exist" containerID="4c3c97efcbdb091aa9fa2ff142d47f3a21254eccb249a218e19e874fad595f61"
Sep 10 19:46:41.328864 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:41.328820    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c3c97efcbdb091aa9fa2ff142d47f3a21254eccb249a218e19e874fad595f61"} err="failed to get container status \"4c3c97efcbdb091aa9fa2ff142d47f3a21254eccb249a218e19e874fad595f61\": rpc error: code = NotFound desc = could not find container \"4c3c97efcbdb091aa9fa2ff142d47f3a21254eccb249a218e19e874fad595f61\": container with ID starting with 4c3c97efcbdb091aa9fa2ff142d47f3a21254eccb249a218e19e874fad595f61 not found: ID does not exist"
Sep 10 19:46:41.342160 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:41.342135    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-28d250-5d86c7b9b6-v766w"]
Sep 10 19:46:41.347845 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:41.347822    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-28d250-5d86c7b9b6-v766w"]
Sep 10 19:46:41.778885 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:41.778837    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:41.792806 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:41.792759    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-28d250-5d86c7b9b6-v766w" podUID="12bab5be-61db-4403-b133-064439213d04" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.133.0.66:9004: i/o timeout"
Sep 10 19:46:42.949747 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:42.949715    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12bab5be-61db-4403-b133-064439213d04" path="/var/lib/kubelet/pods/12bab5be-61db-4403-b133-064439213d04/volumes"
Sep 10 19:46:47.802138 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:47.802104    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:47.865885 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:47.865849    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:48.063323 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:48.063232    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx"]
Sep 10 19:46:48.063568 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:48.063467    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx" podUID="837f44f6-1b71-41a2-8bcf-364fcfcb8a85" containerName="istio-proxy" containerID="cri-o://52caaa63fb8a34548e59f54fced77f89c08838e0085488253ee9946103ccd8b7" gracePeriod=30
Sep 10 19:46:48.083994 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:48.083957    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:48.416198 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:48.416117    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-b83810-d2rg9"]
Sep 10 19:46:48.416521 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:48.416509    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12bab5be-61db-4403-b133-064439213d04" containerName="payload-pre-processing"
Sep 10 19:46:48.416565 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:48.416523    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="12bab5be-61db-4403-b133-064439213d04" containerName="payload-pre-processing"
Sep 10 19:46:48.416565 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:48.416532    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc7a4026-0eca-46de-a1bf-7e3ac3742eb1" containerName="revoke-keys"
Sep 10 19:46:48.416565 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:48.416537    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc7a4026-0eca-46de-a1bf-7e3ac3742eb1" containerName="revoke-keys"
Sep 10 19:46:48.416565 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:48.416545    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7820d541-2f9a-41a7-bf74-b0e8b4161d56" containerName="istio-proxy"
Sep 10 19:46:48.416565 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:48.416550    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7820d541-2f9a-41a7-bf74-b0e8b4161d56" containerName="istio-proxy"
Sep 10 19:46:48.416565 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:48.416557    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7820d541-2f9a-41a7-bf74-b0e8b4161d56" containerName="istio-proxy"
Sep 10 19:46:48.416565 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:48.416563    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7820d541-2f9a-41a7-bf74-b0e8b4161d56" containerName="istio-proxy"
Sep 10 19:46:48.416804 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:48.416651    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc7a4026-0eca-46de-a1bf-7e3ac3742eb1" containerName="revoke-keys"
Sep 10 19:46:48.416804 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:48.416659    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="12bab5be-61db-4403-b133-064439213d04" containerName="payload-pre-processing"
Sep 10 19:46:48.416804 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:48.416667    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7820d541-2f9a-41a7-bf74-b0e8b4161d56" containerName="istio-proxy"
Sep 10 19:46:48.416804 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:48.416673    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7820d541-2f9a-41a7-bf74-b0e8b4161d56" containerName="istio-proxy"
Sep 10 19:46:48.419822 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:48.419806    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-b83810-d2rg9"
Sep 10 19:46:48.422041 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:48.422019    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 19:46:48.422166 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:48.422023    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tjmqn\""
Sep 10 19:46:48.427986 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:48.427963    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-b83810-d2rg9"]
Sep 10 19:46:48.528590 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:48.528554    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d5554e7-bed1-459c-bc8d-fb1d68a1ee93-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-b83810-d2rg9\" (UID: \"4d5554e7-bed1-459c-bc8d-fb1d68a1ee93\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-b83810-d2rg9"
Sep 10 19:46:48.629458 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:48.629415    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d5554e7-bed1-459c-bc8d-fb1d68a1ee93-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-b83810-d2rg9\" (UID: \"4d5554e7-bed1-459c-bc8d-fb1d68a1ee93\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-b83810-d2rg9"
Sep 10 19:46:48.630077 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:48.630056    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d5554e7-bed1-459c-bc8d-fb1d68a1ee93-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-b83810-d2rg9\" (UID: \"4d5554e7-bed1-459c-bc8d-fb1d68a1ee93\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-b83810-d2rg9"
Sep 10 19:46:48.647025 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:48.646993    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:48.730415 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:48.730330    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-b83810-d2rg9"
Sep 10 19:46:49.006875 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:49.006845    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-b83810-d2rg9"]
Sep 10 19:46:49.008958 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:46:49.008929    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d5554e7_bed1_459c_bc8d_fb1d68a1ee93.slice/crio-2ab72a49fa0236a40d162d214ae1033f3c66e942a133c27a29679cc76554732b WatchSource:0}: Error finding container 2ab72a49fa0236a40d162d214ae1033f3c66e942a133c27a29679cc76554732b: Status 404 returned error can't find the container with id 2ab72a49fa0236a40d162d214ae1033f3c66e942a133c27a29679cc76554732b
Sep 10 19:46:49.367648 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:49.367537    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-b83810-d2rg9" event={"ID":"4d5554e7-bed1-459c-bc8d-fb1d68a1ee93","Type":"ContainerStarted","Data":"d4b10eb3d9bde5a96f5947f05636069aab77f16cce0b5bd1e59cd6ffdabb256f"}
Sep 10 19:46:49.367648 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:49.367646    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-b83810-d2rg9" event={"ID":"4d5554e7-bed1-459c-bc8d-fb1d68a1ee93","Type":"ContainerStarted","Data":"2ab72a49fa0236a40d162d214ae1033f3c66e942a133c27a29679cc76554732b"}
Sep 10 19:46:49.385552 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:49.385500    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-b83810-d2rg9" podStartSLOduration=1.385484902 podStartE2EDuration="1.385484902s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:46:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:46:49.383784496 +0000 UTC m=+1864.984691735" watchObservedRunningTime="2026-09-10 19:46:49.385484902 +0000 UTC m=+1864.986392168"
Sep 10 19:46:50.378226 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:50.378189    2564 generic.go:356] "Generic (PLEG): container finished" podID="4d5554e7-bed1-459c-bc8d-fb1d68a1ee93" containerID="d4b10eb3d9bde5a96f5947f05636069aab77f16cce0b5bd1e59cd6ffdabb256f" exitCode=0
Sep 10 19:46:50.378604 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:50.378282    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-b83810-d2rg9" event={"ID":"4d5554e7-bed1-459c-bc8d-fb1d68a1ee93","Type":"ContainerDied","Data":"d4b10eb3d9bde5a96f5947f05636069aab77f16cce0b5bd1e59cd6ffdabb256f"}
Sep 10 19:46:51.514393 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:51.514368    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-b83810-d2rg9"
Sep 10 19:46:51.661040 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:51.660956    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d5554e7-bed1-459c-bc8d-fb1d68a1ee93-maas-api-service-ca\") pod \"4d5554e7-bed1-459c-bc8d-fb1d68a1ee93\" (UID: \"4d5554e7-bed1-459c-bc8d-fb1d68a1ee93\") "
Sep 10 19:46:51.661289 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:51.661266    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d5554e7-bed1-459c-bc8d-fb1d68a1ee93-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4d5554e7-bed1-459c-bc8d-fb1d68a1ee93" (UID: "4d5554e7-bed1-459c-bc8d-fb1d68a1ee93"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:46:51.762116 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:51.762080    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d5554e7-bed1-459c-bc8d-fb1d68a1ee93-maas-api-service-ca\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:46:52.390172 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:52.390137    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-b83810-d2rg9"
Sep 10 19:46:52.390172 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:52.390155    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-b83810-d2rg9" event={"ID":"4d5554e7-bed1-459c-bc8d-fb1d68a1ee93","Type":"ContainerDied","Data":"2ab72a49fa0236a40d162d214ae1033f3c66e942a133c27a29679cc76554732b"}
Sep 10 19:46:52.390427 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:52.390201    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2ab72a49fa0236a40d162d214ae1033f3c66e942a133c27a29679cc76554732b"
Sep 10 19:46:53.321301 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:53.321276    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx"
Sep 10 19:46:53.372902 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:53.372814    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vlkbg\" (UniqueName: \"kubernetes.io/projected/837f44f6-1b71-41a2-8bcf-364fcfcb8a85-kube-api-access-vlkbg\") pod \"837f44f6-1b71-41a2-8bcf-364fcfcb8a85\" (UID: \"837f44f6-1b71-41a2-8bcf-364fcfcb8a85\") "
Sep 10 19:46:53.372902 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:53.372855    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/837f44f6-1b71-41a2-8bcf-364fcfcb8a85-istio-envoy\") pod \"837f44f6-1b71-41a2-8bcf-364fcfcb8a85\" (UID: \"837f44f6-1b71-41a2-8bcf-364fcfcb8a85\") "
Sep 10 19:46:53.372902 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:53.372886    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/837f44f6-1b71-41a2-8bcf-364fcfcb8a85-workload-certs\") pod \"837f44f6-1b71-41a2-8bcf-364fcfcb8a85\" (UID: \"837f44f6-1b71-41a2-8bcf-364fcfcb8a85\") "
Sep 10 19:46:53.373197 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:53.372914    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/837f44f6-1b71-41a2-8bcf-364fcfcb8a85-istio-token\") pod \"837f44f6-1b71-41a2-8bcf-364fcfcb8a85\" (UID: \"837f44f6-1b71-41a2-8bcf-364fcfcb8a85\") "
Sep 10 19:46:53.373197 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:53.372931    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/837f44f6-1b71-41a2-8bcf-364fcfcb8a85-istiod-ca-cert\") pod \"837f44f6-1b71-41a2-8bcf-364fcfcb8a85\" (UID: \"837f44f6-1b71-41a2-8bcf-364fcfcb8a85\") "
Sep 10 19:46:53.373197 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:53.372953    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/837f44f6-1b71-41a2-8bcf-364fcfcb8a85-workload-socket\") pod \"837f44f6-1b71-41a2-8bcf-364fcfcb8a85\" (UID: \"837f44f6-1b71-41a2-8bcf-364fcfcb8a85\") "
Sep 10 19:46:53.373197 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:53.372993    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/837f44f6-1b71-41a2-8bcf-364fcfcb8a85-istio-podinfo\") pod \"837f44f6-1b71-41a2-8bcf-364fcfcb8a85\" (UID: \"837f44f6-1b71-41a2-8bcf-364fcfcb8a85\") "
Sep 10 19:46:53.373197 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:53.373090    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/837f44f6-1b71-41a2-8bcf-364fcfcb8a85-istio-data\") pod \"837f44f6-1b71-41a2-8bcf-364fcfcb8a85\" (UID: \"837f44f6-1b71-41a2-8bcf-364fcfcb8a85\") "
Sep 10 19:46:53.373197 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:53.373141    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/837f44f6-1b71-41a2-8bcf-364fcfcb8a85-credential-socket\") pod \"837f44f6-1b71-41a2-8bcf-364fcfcb8a85\" (UID: \"837f44f6-1b71-41a2-8bcf-364fcfcb8a85\") "
Sep 10 19:46:53.373501 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:53.373263    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/837f44f6-1b71-41a2-8bcf-364fcfcb8a85-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "837f44f6-1b71-41a2-8bcf-364fcfcb8a85" (UID: "837f44f6-1b71-41a2-8bcf-364fcfcb8a85"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:46:53.373501 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:53.373450    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/837f44f6-1b71-41a2-8bcf-364fcfcb8a85-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "837f44f6-1b71-41a2-8bcf-364fcfcb8a85" (UID: "837f44f6-1b71-41a2-8bcf-364fcfcb8a85"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:46:53.374340 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:53.374300    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/837f44f6-1b71-41a2-8bcf-364fcfcb8a85-workload-certs\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:46:53.383947 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:53.374473    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/837f44f6-1b71-41a2-8bcf-364fcfcb8a85-istio-data" (OuterVolumeSpecName: "istio-data") pod "837f44f6-1b71-41a2-8bcf-364fcfcb8a85" (UID: "837f44f6-1b71-41a2-8bcf-364fcfcb8a85"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:46:53.383947 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:53.374658    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/837f44f6-1b71-41a2-8bcf-364fcfcb8a85-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "837f44f6-1b71-41a2-8bcf-364fcfcb8a85" (UID: "837f44f6-1b71-41a2-8bcf-364fcfcb8a85"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:46:53.383947 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:53.374685    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/837f44f6-1b71-41a2-8bcf-364fcfcb8a85-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "837f44f6-1b71-41a2-8bcf-364fcfcb8a85" (UID: "837f44f6-1b71-41a2-8bcf-364fcfcb8a85"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:46:53.383947 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:53.376361    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/837f44f6-1b71-41a2-8bcf-364fcfcb8a85-kube-api-access-vlkbg" (OuterVolumeSpecName: "kube-api-access-vlkbg") pod "837f44f6-1b71-41a2-8bcf-364fcfcb8a85" (UID: "837f44f6-1b71-41a2-8bcf-364fcfcb8a85"). InnerVolumeSpecName "kube-api-access-vlkbg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:46:53.383947 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:53.377836    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/837f44f6-1b71-41a2-8bcf-364fcfcb8a85-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "837f44f6-1b71-41a2-8bcf-364fcfcb8a85" (UID: "837f44f6-1b71-41a2-8bcf-364fcfcb8a85"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:46:53.384424 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:53.384180    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/837f44f6-1b71-41a2-8bcf-364fcfcb8a85-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "837f44f6-1b71-41a2-8bcf-364fcfcb8a85" (UID: "837f44f6-1b71-41a2-8bcf-364fcfcb8a85"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 19:46:53.384424 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:53.384244    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/837f44f6-1b71-41a2-8bcf-364fcfcb8a85-istio-token" (OuterVolumeSpecName: "istio-token") pod "837f44f6-1b71-41a2-8bcf-364fcfcb8a85" (UID: "837f44f6-1b71-41a2-8bcf-364fcfcb8a85"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:46:53.396602 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:53.396572    2564 generic.go:356] "Generic (PLEG): container finished" podID="837f44f6-1b71-41a2-8bcf-364fcfcb8a85" containerID="52caaa63fb8a34548e59f54fced77f89c08838e0085488253ee9946103ccd8b7" exitCode=0
Sep 10 19:46:53.396746 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:53.396645    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx"
Sep 10 19:46:53.396746 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:53.396660    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx" event={"ID":"837f44f6-1b71-41a2-8bcf-364fcfcb8a85","Type":"ContainerDied","Data":"52caaa63fb8a34548e59f54fced77f89c08838e0085488253ee9946103ccd8b7"}
Sep 10 19:46:53.396868 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:53.396751    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx" event={"ID":"837f44f6-1b71-41a2-8bcf-364fcfcb8a85","Type":"ContainerDied","Data":"a16156b755c2985ad0de7d19c2d933468868dbcb2f1a6b9e7257f29946211c17"}
Sep 10 19:46:53.396868 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:53.396785    2564 scope.go:117] "RemoveContainer" containerID="52caaa63fb8a34548e59f54fced77f89c08838e0085488253ee9946103ccd8b7"
Sep 10 19:46:53.405985 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:53.405966    2564 scope.go:117] "RemoveContainer" containerID="270126e4f4a48f525d1e9df11844143adc92e03d9f53ef018e497928c8ffee6d"
Sep 10 19:46:53.414682 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:53.414664    2564 scope.go:117] "RemoveContainer" containerID="52caaa63fb8a34548e59f54fced77f89c08838e0085488253ee9946103ccd8b7"
Sep 10 19:46:53.414955 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:46:53.414935    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52caaa63fb8a34548e59f54fced77f89c08838e0085488253ee9946103ccd8b7\": container with ID starting with 52caaa63fb8a34548e59f54fced77f89c08838e0085488253ee9946103ccd8b7 not found: ID does not exist" containerID="52caaa63fb8a34548e59f54fced77f89c08838e0085488253ee9946103ccd8b7"
Sep 10 19:46:53.415029 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:53.414967    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52caaa63fb8a34548e59f54fced77f89c08838e0085488253ee9946103ccd8b7"} err="failed to get container status \"52caaa63fb8a34548e59f54fced77f89c08838e0085488253ee9946103ccd8b7\": rpc error: code = NotFound desc = could not find container \"52caaa63fb8a34548e59f54fced77f89c08838e0085488253ee9946103ccd8b7\": container with ID starting with 52caaa63fb8a34548e59f54fced77f89c08838e0085488253ee9946103ccd8b7 not found: ID does not exist"
Sep 10 19:46:53.415029 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:53.414992    2564 scope.go:117] "RemoveContainer" containerID="270126e4f4a48f525d1e9df11844143adc92e03d9f53ef018e497928c8ffee6d"
Sep 10 19:46:53.415254 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:46:53.415237    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"270126e4f4a48f525d1e9df11844143adc92e03d9f53ef018e497928c8ffee6d\": container with ID starting with 270126e4f4a48f525d1e9df11844143adc92e03d9f53ef018e497928c8ffee6d not found: ID does not exist" containerID="270126e4f4a48f525d1e9df11844143adc92e03d9f53ef018e497928c8ffee6d"
Sep 10 19:46:53.415291 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:53.415261    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"270126e4f4a48f525d1e9df11844143adc92e03d9f53ef018e497928c8ffee6d"} err="failed to get container status \"270126e4f4a48f525d1e9df11844143adc92e03d9f53ef018e497928c8ffee6d\": rpc error: code = NotFound desc = could not find container \"270126e4f4a48f525d1e9df11844143adc92e03d9f53ef018e497928c8ffee6d\": container with ID starting with 270126e4f4a48f525d1e9df11844143adc92e03d9f53ef018e497928c8ffee6d not found: ID does not exist"
Sep 10 19:46:53.422204 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:53.422182    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx"]
Sep 10 19:46:53.429410 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:53.429388    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-28d250-openshift-default-756c495486-vgllx"]
Sep 10 19:46:53.475641 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:53.475598    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/837f44f6-1b71-41a2-8bcf-364fcfcb8a85-istio-podinfo\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:46:53.475641 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:53.475631    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/837f44f6-1b71-41a2-8bcf-364fcfcb8a85-istio-data\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:46:53.475641 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:53.475642    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/837f44f6-1b71-41a2-8bcf-364fcfcb8a85-credential-socket\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:46:53.475844 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:53.475653    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vlkbg\" (UniqueName: \"kubernetes.io/projected/837f44f6-1b71-41a2-8bcf-364fcfcb8a85-kube-api-access-vlkbg\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:46:53.475844 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:53.475663    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/837f44f6-1b71-41a2-8bcf-364fcfcb8a85-istio-envoy\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:46:53.475844 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:53.475671    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/837f44f6-1b71-41a2-8bcf-364fcfcb8a85-istio-token\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:46:53.475844 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:53.475680    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/837f44f6-1b71-41a2-8bcf-364fcfcb8a85-istiod-ca-cert\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:46:53.475844 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:53.475689    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/837f44f6-1b71-41a2-8bcf-364fcfcb8a85-workload-socket\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:46:54.948641 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:54.948593    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="837f44f6-1b71-41a2-8bcf-364fcfcb8a85" path="/var/lib/kubelet/pods/837f44f6-1b71-41a2-8bcf-364fcfcb8a85/volumes"
Sep 10 19:46:58.910543 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:58.910513    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:58.957803 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:58.957753    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b83810-96d75b4fd-6rmvx"]
Sep 10 19:46:58.958510 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:58.958477    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b83810-96d75b4fd-6rmvx" podUID="d78eafd1-ceaa-4e14-9d59-e71b95f08ae7" containerName="payload-pre-processing" containerID="cri-o://f5e28604804ffbe7e03edac3813eaff2e902e5f5b273ea709b08dc24b88afacb" gracePeriod=30
Sep 10 19:46:58.986376 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:58.986341    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:59.046085 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:59.046050    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:59.122070 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:59.122037    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-b83810-d2rg9"]
Sep 10 19:46:59.125918 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:59.125883    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:59.126225 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:59.126206    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-b83810-d2rg9"]
Sep 10 19:46:59.186802 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:59.186682    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:59.222238 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:59.222212    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b83810-96d75b4fd-6rmvx"
Sep 10 19:46:59.328472 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:59.328436    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vdqbm\" (UniqueName: \"kubernetes.io/projected/d78eafd1-ceaa-4e14-9d59-e71b95f08ae7-kube-api-access-vdqbm\") pod \"d78eafd1-ceaa-4e14-9d59-e71b95f08ae7\" (UID: \"d78eafd1-ceaa-4e14-9d59-e71b95f08ae7\") "
Sep 10 19:46:59.328472 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:59.328475    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d78eafd1-ceaa-4e14-9d59-e71b95f08ae7-plugins-config-volume\") pod \"d78eafd1-ceaa-4e14-9d59-e71b95f08ae7\" (UID: \"d78eafd1-ceaa-4e14-9d59-e71b95f08ae7\") "
Sep 10 19:46:59.328923 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:59.328897    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d78eafd1-ceaa-4e14-9d59-e71b95f08ae7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d78eafd1-ceaa-4e14-9d59-e71b95f08ae7" (UID: "d78eafd1-ceaa-4e14-9d59-e71b95f08ae7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:46:59.330638 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:59.330598    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d78eafd1-ceaa-4e14-9d59-e71b95f08ae7-kube-api-access-vdqbm" (OuterVolumeSpecName: "kube-api-access-vdqbm") pod "d78eafd1-ceaa-4e14-9d59-e71b95f08ae7" (UID: "d78eafd1-ceaa-4e14-9d59-e71b95f08ae7"). InnerVolumeSpecName "kube-api-access-vdqbm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:46:59.429540 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:59.429509    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vdqbm\" (UniqueName: \"kubernetes.io/projected/d78eafd1-ceaa-4e14-9d59-e71b95f08ae7-kube-api-access-vdqbm\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:46:59.429540 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:59.429536    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d78eafd1-ceaa-4e14-9d59-e71b95f08ae7-plugins-config-volume\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:46:59.432001 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:59.431969    2564 generic.go:356] "Generic (PLEG): container finished" podID="d78eafd1-ceaa-4e14-9d59-e71b95f08ae7" containerID="f5e28604804ffbe7e03edac3813eaff2e902e5f5b273ea709b08dc24b88afacb" exitCode=0
Sep 10 19:46:59.432101 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:59.432025    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b83810-96d75b4fd-6rmvx"
Sep 10 19:46:59.432154 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:59.432033    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b83810-96d75b4fd-6rmvx" event={"ID":"d78eafd1-ceaa-4e14-9d59-e71b95f08ae7","Type":"ContainerDied","Data":"f5e28604804ffbe7e03edac3813eaff2e902e5f5b273ea709b08dc24b88afacb"}
Sep 10 19:46:59.432190 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:59.432166    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b83810-96d75b4fd-6rmvx" event={"ID":"d78eafd1-ceaa-4e14-9d59-e71b95f08ae7","Type":"ContainerDied","Data":"11f06a06b18b46df9e16be47aa89c85c1007911c1a09f58c07a0ca9afd37d13b"}
Sep 10 19:46:59.432224 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:59.432190    2564 scope.go:117] "RemoveContainer" containerID="f5e28604804ffbe7e03edac3813eaff2e902e5f5b273ea709b08dc24b88afacb"
Sep 10 19:46:59.441916 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:59.441897    2564 scope.go:117] "RemoveContainer" containerID="f5e28604804ffbe7e03edac3813eaff2e902e5f5b273ea709b08dc24b88afacb"
Sep 10 19:46:59.442232 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:46:59.442211    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5e28604804ffbe7e03edac3813eaff2e902e5f5b273ea709b08dc24b88afacb\": container with ID starting with f5e28604804ffbe7e03edac3813eaff2e902e5f5b273ea709b08dc24b88afacb not found: ID does not exist" containerID="f5e28604804ffbe7e03edac3813eaff2e902e5f5b273ea709b08dc24b88afacb"
Sep 10 19:46:59.442306 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:59.442244    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5e28604804ffbe7e03edac3813eaff2e902e5f5b273ea709b08dc24b88afacb"} err="failed to get container status \"f5e28604804ffbe7e03edac3813eaff2e902e5f5b273ea709b08dc24b88afacb\": rpc error: code = NotFound desc = could not find container \"f5e28604804ffbe7e03edac3813eaff2e902e5f5b273ea709b08dc24b88afacb\": container with ID starting with f5e28604804ffbe7e03edac3813eaff2e902e5f5b273ea709b08dc24b88afacb not found: ID does not exist"
Sep 10 19:46:59.455359 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:59.455325    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b83810-96d75b4fd-6rmvx"]
Sep 10 19:46:59.462957 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:59.462921    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b83810-96d75b4fd-6rmvx"]
Sep 10 19:46:59.516827 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:46:59.516792    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:47:00.948815 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:47:00.948781    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d5554e7-bed1-459c-bc8d-fb1d68a1ee93" path="/var/lib/kubelet/pods/4d5554e7-bed1-459c-bc8d-fb1d68a1ee93/volumes"
Sep 10 19:47:00.949185 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:47:00.949159    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d78eafd1-ceaa-4e14-9d59-e71b95f08ae7" path="/var/lib/kubelet/pods/d78eafd1-ceaa-4e14-9d59-e71b95f08ae7/volumes"
Sep 10 19:47:06.016160 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:47:06.016119    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:47:06.116445 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:47:06.116414    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:47:06.288099 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:47:06.288010    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:47:42.849863 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:47:42.849827    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:47:45.369435 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:47:45.369402    2564 scope.go:117] "RemoveContainer" containerID="e838751250a4a3d11b7ad526631bf0a92fad7fa212ca0a4e3ab7ef4e3c7e78a3"
Sep 10 19:47:45.378055 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:47:45.378033    2564 scope.go:117] "RemoveContainer" containerID="83f0295658e020999a0e12059df69d4883c5256620f2fffaf050e364f8a7233a"
Sep 10 19:47:56.632445 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:47:56.632399    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:48:12.107589 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:48:12.107552    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:48:12.209913 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:48:12.209873    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:48:18.313931 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:48:18.313888    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:48:32.668354 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:48:32.668298    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:48:33.890757 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:48:33.890719    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:48:35.321025 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:48:35.320982    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:48:39.634669 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:48:39.634563    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:48:43.749703 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:48:43.749664    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:48:45.411768 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:48:45.411736    2564 scope.go:117] "RemoveContainer" containerID="910fcdbd1f922e9cb0b6ad83d755a51ecbcd4a6e9de78f7cadfaa8e07d536a9d"
Sep 10 19:48:45.419904 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:48:45.419881    2564 scope.go:117] "RemoveContainer" containerID="d153c970861a2a5e59a5a196613776669640cf2cbde2f29bb0854aa0a2f630c9"
Sep 10 19:49:44.738711 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:49:44.738675    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tjnq2"]
Sep 10 19:49:44.741340 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:49:44.738896    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tjnq2" podUID="d352d9a4-e806-48c7-9cba-98af37878cef" containerName="manager" containerID="cri-o://f167490d9b410226d9257e3b4e80f6f6f925a900519ea6b70a7c75158aac2596" gracePeriod=10
Sep 10 19:49:47.094407 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:49:47.094377    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tjnq2"
Sep 10 19:49:47.203456 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:49:47.203373    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d352d9a4-e806-48c7-9cba-98af37878cef-extensions-socket-volume\") pod \"d352d9a4-e806-48c7-9cba-98af37878cef\" (UID: \"d352d9a4-e806-48c7-9cba-98af37878cef\") "
Sep 10 19:49:47.203456 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:49:47.203407    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sx694\" (UniqueName: \"kubernetes.io/projected/d352d9a4-e806-48c7-9cba-98af37878cef-kube-api-access-sx694\") pod \"d352d9a4-e806-48c7-9cba-98af37878cef\" (UID: \"d352d9a4-e806-48c7-9cba-98af37878cef\") "
Sep 10 19:49:47.203760 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:49:47.203736    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d352d9a4-e806-48c7-9cba-98af37878cef-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d352d9a4-e806-48c7-9cba-98af37878cef" (UID: "d352d9a4-e806-48c7-9cba-98af37878cef"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:49:47.205484 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:49:47.205460    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d352d9a4-e806-48c7-9cba-98af37878cef-kube-api-access-sx694" (OuterVolumeSpecName: "kube-api-access-sx694") pod "d352d9a4-e806-48c7-9cba-98af37878cef" (UID: "d352d9a4-e806-48c7-9cba-98af37878cef"). InnerVolumeSpecName "kube-api-access-sx694". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:49:47.304488 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:49:47.304446    2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d352d9a4-e806-48c7-9cba-98af37878cef-extensions-socket-volume\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:49:47.304488 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:49:47.304479    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sx694\" (UniqueName: \"kubernetes.io/projected/d352d9a4-e806-48c7-9cba-98af37878cef-kube-api-access-sx694\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:49:47.355883 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:49:47.355846    2564 generic.go:356] "Generic (PLEG): container finished" podID="d352d9a4-e806-48c7-9cba-98af37878cef" containerID="f167490d9b410226d9257e3b4e80f6f6f925a900519ea6b70a7c75158aac2596" exitCode=0
Sep 10 19:49:47.356074 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:49:47.355895    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tjnq2"
Sep 10 19:49:47.356074 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:49:47.355902    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tjnq2" event={"ID":"d352d9a4-e806-48c7-9cba-98af37878cef","Type":"ContainerDied","Data":"f167490d9b410226d9257e3b4e80f6f6f925a900519ea6b70a7c75158aac2596"}
Sep 10 19:49:47.356074 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:49:47.355954    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tjnq2" event={"ID":"d352d9a4-e806-48c7-9cba-98af37878cef","Type":"ContainerDied","Data":"3335910fdc55bd202ae4170f298724840268a0938c4ee0eddcf6e6341d0f9f52"}
Sep 10 19:49:47.356074 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:49:47.355982    2564 scope.go:117] "RemoveContainer" containerID="f167490d9b410226d9257e3b4e80f6f6f925a900519ea6b70a7c75158aac2596"
Sep 10 19:49:47.365430 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:49:47.365410    2564 scope.go:117] "RemoveContainer" containerID="f167490d9b410226d9257e3b4e80f6f6f925a900519ea6b70a7c75158aac2596"
Sep 10 19:49:47.365760 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:49:47.365736    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f167490d9b410226d9257e3b4e80f6f6f925a900519ea6b70a7c75158aac2596\": container with ID starting with f167490d9b410226d9257e3b4e80f6f6f925a900519ea6b70a7c75158aac2596 not found: ID does not exist" containerID="f167490d9b410226d9257e3b4e80f6f6f925a900519ea6b70a7c75158aac2596"
Sep 10 19:49:47.365834 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:49:47.365767    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f167490d9b410226d9257e3b4e80f6f6f925a900519ea6b70a7c75158aac2596"} err="failed to get container status \"f167490d9b410226d9257e3b4e80f6f6f925a900519ea6b70a7c75158aac2596\": rpc error: code = NotFound desc = could not find container \"f167490d9b410226d9257e3b4e80f6f6f925a900519ea6b70a7c75158aac2596\": container with ID starting with f167490d9b410226d9257e3b4e80f6f6f925a900519ea6b70a7c75158aac2596 not found: ID does not exist"
Sep 10 19:49:47.380789 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:49:47.380765    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tjnq2"]
Sep 10 19:49:47.385096 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:49:47.385073    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tjnq2"]
Sep 10 19:49:48.948502 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:49:48.948460    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d352d9a4-e806-48c7-9cba-98af37878cef" path="/var/lib/kubelet/pods/d352d9a4-e806-48c7-9cba-98af37878cef/volumes"
Sep 10 19:50:45.432694 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:50:45.432658    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-96s67"]
Sep 10 19:50:45.433165 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:50:45.433058    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d5554e7-bed1-459c-bc8d-fb1d68a1ee93" containerName="revoke-keys"
Sep 10 19:50:45.433165 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:50:45.433070    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d5554e7-bed1-459c-bc8d-fb1d68a1ee93" containerName="revoke-keys"
Sep 10 19:50:45.433165 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:50:45.433084    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d352d9a4-e806-48c7-9cba-98af37878cef" containerName="manager"
Sep 10 19:50:45.433165 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:50:45.433089    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d352d9a4-e806-48c7-9cba-98af37878cef" containerName="manager"
Sep 10 19:50:45.433165 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:50:45.433105    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="837f44f6-1b71-41a2-8bcf-364fcfcb8a85" containerName="istio-proxy"
Sep 10 19:50:45.433165 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:50:45.433110    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="837f44f6-1b71-41a2-8bcf-364fcfcb8a85" containerName="istio-proxy"
Sep 10 19:50:45.433165 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:50:45.433118    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="837f44f6-1b71-41a2-8bcf-364fcfcb8a85" containerName="istio-proxy"
Sep 10 19:50:45.433165 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:50:45.433123    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="837f44f6-1b71-41a2-8bcf-364fcfcb8a85" containerName="istio-proxy"
Sep 10 19:50:45.433165 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:50:45.433129    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d78eafd1-ceaa-4e14-9d59-e71b95f08ae7" containerName="payload-pre-processing"
Sep 10 19:50:45.433165 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:50:45.433135    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d78eafd1-ceaa-4e14-9d59-e71b95f08ae7" containerName="payload-pre-processing"
Sep 10 19:50:45.433487 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:50:45.433191    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="837f44f6-1b71-41a2-8bcf-364fcfcb8a85" containerName="istio-proxy"
Sep 10 19:50:45.433487 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:50:45.433199    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d352d9a4-e806-48c7-9cba-98af37878cef" containerName="manager"
Sep 10 19:50:45.433487 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:50:45.433205    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d78eafd1-ceaa-4e14-9d59-e71b95f08ae7" containerName="payload-pre-processing"
Sep 10 19:50:45.433487 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:50:45.433211    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d5554e7-bed1-459c-bc8d-fb1d68a1ee93" containerName="revoke-keys"
Sep 10 19:50:45.436399 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:50:45.436383    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-96s67"
Sep 10 19:50:45.438803 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:50:45.438772    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-qpnj5\""
Sep 10 19:50:45.446599 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:50:45.446573    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-96s67"]
Sep 10 19:50:45.544213 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:50:45.544179    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcgrx\" (UniqueName: \"kubernetes.io/projected/0fb6597a-3c59-4314-a4d4-48366678dd44-kube-api-access-qcgrx\") pod \"kuadrant-operator-controller-manager-5769f99d9f-96s67\" (UID: \"0fb6597a-3c59-4314-a4d4-48366678dd44\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-96s67"
Sep 10 19:50:45.544383 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:50:45.544276    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0fb6597a-3c59-4314-a4d4-48366678dd44-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-96s67\" (UID: \"0fb6597a-3c59-4314-a4d4-48366678dd44\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-96s67"
Sep 10 19:50:45.645506 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:50:45.645466    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0fb6597a-3c59-4314-a4d4-48366678dd44-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-96s67\" (UID: \"0fb6597a-3c59-4314-a4d4-48366678dd44\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-96s67"
Sep 10 19:50:45.645695 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:50:45.645550    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcgrx\" (UniqueName: \"kubernetes.io/projected/0fb6597a-3c59-4314-a4d4-48366678dd44-kube-api-access-qcgrx\") pod \"kuadrant-operator-controller-manager-5769f99d9f-96s67\" (UID: \"0fb6597a-3c59-4314-a4d4-48366678dd44\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-96s67"
Sep 10 19:50:45.645900 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:50:45.645881    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0fb6597a-3c59-4314-a4d4-48366678dd44-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-96s67\" (UID: \"0fb6597a-3c59-4314-a4d4-48366678dd44\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-96s67"
Sep 10 19:50:45.659137 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:50:45.659114    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcgrx\" (UniqueName: \"kubernetes.io/projected/0fb6597a-3c59-4314-a4d4-48366678dd44-kube-api-access-qcgrx\") pod \"kuadrant-operator-controller-manager-5769f99d9f-96s67\" (UID: \"0fb6597a-3c59-4314-a4d4-48366678dd44\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-96s67"
Sep 10 19:50:45.746856 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:50:45.746824    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-96s67"
Sep 10 19:50:46.030755 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:50:46.030728    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-96s67"]
Sep 10 19:50:46.033536 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:50:46.033509    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fb6597a_3c59_4314_a4d4_48366678dd44.slice/crio-256ae3bde4d7126d7717d0f05e942807784be64202dc49ce5bf4df81b3e3335a WatchSource:0}: Error finding container 256ae3bde4d7126d7717d0f05e942807784be64202dc49ce5bf4df81b3e3335a: Status 404 returned error can't find the container with id 256ae3bde4d7126d7717d0f05e942807784be64202dc49ce5bf4df81b3e3335a
Sep 10 19:50:46.670939 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:50:46.670901    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-96s67" event={"ID":"0fb6597a-3c59-4314-a4d4-48366678dd44","Type":"ContainerStarted","Data":"0d1a0a629d2952a464de7454c3c1fa58b05bdd858222b6d3ac5123d2ef7446cb"}
Sep 10 19:50:46.671334 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:50:46.670967    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-96s67"
Sep 10 19:50:46.671334 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:50:46.670989    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-96s67" event={"ID":"0fb6597a-3c59-4314-a4d4-48366678dd44","Type":"ContainerStarted","Data":"256ae3bde4d7126d7717d0f05e942807784be64202dc49ce5bf4df81b3e3335a"}
Sep 10 19:50:46.691360 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:50:46.691306    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-96s67" podStartSLOduration=1.691291341 podStartE2EDuration="1.691291341s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:50:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:50:46.690216587 +0000 UTC m=+2102.291123827" watchObservedRunningTime="2026-09-10 19:50:46.691291341 +0000 UTC m=+2102.292198580"
Sep 10 19:50:57.679500 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:50:57.679464    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-96s67"
Sep 10 19:51:40.455905 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:51:40.455808    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:51:45.464157 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:51:45.464124    2564 scope.go:117] "RemoveContainer" containerID="6af467b62eec005fbe371eaf37ca4afe9f3df9473dbb340a7a538257b2240cea"
Sep 10 19:51:45.473520 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:51:45.473496    2564 scope.go:117] "RemoveContainer" containerID="6025d64d91debe66bdb2f9ff9afb7668f8177d77b012722d6514af2fdcab0143"
Sep 10 19:51:45.481693 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:51:45.481673    2564 scope.go:117] "RemoveContainer" containerID="587a8de717e7d2ed4d0db03ef4efed0d76e4c8e5e3bbbacf8172ca7bbd5a15a9"
Sep 10 19:51:45.490282 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:51:45.490257    2564 scope.go:117] "RemoveContainer" containerID="e29736aedf250506b47a808b1509f0ae1a71c60894ad569e230e093edaf8f2fc"
Sep 10 19:51:52.927637 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:51:52.927581    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-2517a652-kserve-c97fc9c99-2xg8w"]
Sep 10 19:51:52.928435 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:51:52.928414    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="837f44f6-1b71-41a2-8bcf-364fcfcb8a85" containerName="istio-proxy"
Sep 10 19:51:52.931924 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:51:52.931902    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-2517a652-kserve-c97fc9c99-2xg8w"
Sep 10 19:51:52.934317 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:51:52.934291    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-2517a652-kserve-self-signed-certs\""
Sep 10 19:51:52.940736 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:51:52.940714    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-2517a652-kserve-c97fc9c99-2xg8w"]
Sep 10 19:51:52.960083 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:51:52.960034    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:51:52.982575 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:51:52.982455    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a02b380-f2d7-4506-9786-5cea66be0a9f-dshm\") pod \"e2e-same-id-b-2517a652-kserve-c97fc9c99-2xg8w\" (UID: \"6a02b380-f2d7-4506-9786-5cea66be0a9f\") " pod="llm/e2e-same-id-b-2517a652-kserve-c97fc9c99-2xg8w"
Sep 10 19:51:52.982575 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:51:52.982519    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a02b380-f2d7-4506-9786-5cea66be0a9f-tls-certs\") pod \"e2e-same-id-b-2517a652-kserve-c97fc9c99-2xg8w\" (UID: \"6a02b380-f2d7-4506-9786-5cea66be0a9f\") " pod="llm/e2e-same-id-b-2517a652-kserve-c97fc9c99-2xg8w"
Sep 10 19:51:52.982852 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:51:52.982606    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjc4k\" (UniqueName: \"kubernetes.io/projected/6a02b380-f2d7-4506-9786-5cea66be0a9f-kube-api-access-pjc4k\") pod \"e2e-same-id-b-2517a652-kserve-c97fc9c99-2xg8w\" (UID: \"6a02b380-f2d7-4506-9786-5cea66be0a9f\") " pod="llm/e2e-same-id-b-2517a652-kserve-c97fc9c99-2xg8w"
Sep 10 19:51:52.982852 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:51:52.982715    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a02b380-f2d7-4506-9786-5cea66be0a9f-tmp-dir\") pod \"e2e-same-id-b-2517a652-kserve-c97fc9c99-2xg8w\" (UID: \"6a02b380-f2d7-4506-9786-5cea66be0a9f\") " pod="llm/e2e-same-id-b-2517a652-kserve-c97fc9c99-2xg8w"
Sep 10 19:51:52.982852 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:51:52.982750    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6a02b380-f2d7-4506-9786-5cea66be0a9f-home\") pod \"e2e-same-id-b-2517a652-kserve-c97fc9c99-2xg8w\" (UID: \"6a02b380-f2d7-4506-9786-5cea66be0a9f\") " pod="llm/e2e-same-id-b-2517a652-kserve-c97fc9c99-2xg8w"
Sep 10 19:51:52.982852 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:51:52.982803    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a02b380-f2d7-4506-9786-5cea66be0a9f-model-cache\") pod \"e2e-same-id-b-2517a652-kserve-c97fc9c99-2xg8w\" (UID: \"6a02b380-f2d7-4506-9786-5cea66be0a9f\") " pod="llm/e2e-same-id-b-2517a652-kserve-c97fc9c99-2xg8w"
Sep 10 19:51:53.083568 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:51:53.083533    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a02b380-f2d7-4506-9786-5cea66be0a9f-tmp-dir\") pod \"e2e-same-id-b-2517a652-kserve-c97fc9c99-2xg8w\" (UID: \"6a02b380-f2d7-4506-9786-5cea66be0a9f\") " pod="llm/e2e-same-id-b-2517a652-kserve-c97fc9c99-2xg8w"
Sep 10 19:51:53.083769 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:51:53.083581    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6a02b380-f2d7-4506-9786-5cea66be0a9f-home\") pod \"e2e-same-id-b-2517a652-kserve-c97fc9c99-2xg8w\" (UID: \"6a02b380-f2d7-4506-9786-5cea66be0a9f\") " pod="llm/e2e-same-id-b-2517a652-kserve-c97fc9c99-2xg8w"
Sep 10 19:51:53.083769 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:51:53.083618    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a02b380-f2d7-4506-9786-5cea66be0a9f-model-cache\") pod \"e2e-same-id-b-2517a652-kserve-c97fc9c99-2xg8w\" (UID: \"6a02b380-f2d7-4506-9786-5cea66be0a9f\") " pod="llm/e2e-same-id-b-2517a652-kserve-c97fc9c99-2xg8w"
Sep 10 19:51:53.083769 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:51:53.083756    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a02b380-f2d7-4506-9786-5cea66be0a9f-dshm\") pod \"e2e-same-id-b-2517a652-kserve-c97fc9c99-2xg8w\" (UID: \"6a02b380-f2d7-4506-9786-5cea66be0a9f\") " pod="llm/e2e-same-id-b-2517a652-kserve-c97fc9c99-2xg8w"
Sep 10 19:51:53.083954 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:51:53.083788    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a02b380-f2d7-4506-9786-5cea66be0a9f-tls-certs\") pod \"e2e-same-id-b-2517a652-kserve-c97fc9c99-2xg8w\" (UID: \"6a02b380-f2d7-4506-9786-5cea66be0a9f\") " pod="llm/e2e-same-id-b-2517a652-kserve-c97fc9c99-2xg8w"
Sep 10 19:51:53.083954 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:51:53.083819    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjc4k\" (UniqueName: \"kubernetes.io/projected/6a02b380-f2d7-4506-9786-5cea66be0a9f-kube-api-access-pjc4k\") pod \"e2e-same-id-b-2517a652-kserve-c97fc9c99-2xg8w\" (UID: \"6a02b380-f2d7-4506-9786-5cea66be0a9f\") " pod="llm/e2e-same-id-b-2517a652-kserve-c97fc9c99-2xg8w"
Sep 10 19:51:53.084065 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:51:53.083990    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a02b380-f2d7-4506-9786-5cea66be0a9f-tmp-dir\") pod \"e2e-same-id-b-2517a652-kserve-c97fc9c99-2xg8w\" (UID: \"6a02b380-f2d7-4506-9786-5cea66be0a9f\") " pod="llm/e2e-same-id-b-2517a652-kserve-c97fc9c99-2xg8w"
Sep 10 19:51:53.084065 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:51:53.084059    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a02b380-f2d7-4506-9786-5cea66be0a9f-model-cache\") pod \"e2e-same-id-b-2517a652-kserve-c97fc9c99-2xg8w\" (UID: \"6a02b380-f2d7-4506-9786-5cea66be0a9f\") " pod="llm/e2e-same-id-b-2517a652-kserve-c97fc9c99-2xg8w"
Sep 10 19:51:53.084297 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:51:53.084270    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6a02b380-f2d7-4506-9786-5cea66be0a9f-home\") pod \"e2e-same-id-b-2517a652-kserve-c97fc9c99-2xg8w\" (UID: \"6a02b380-f2d7-4506-9786-5cea66be0a9f\") " pod="llm/e2e-same-id-b-2517a652-kserve-c97fc9c99-2xg8w"
Sep 10 19:51:53.085976 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:51:53.085952    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a02b380-f2d7-4506-9786-5cea66be0a9f-dshm\") pod \"e2e-same-id-b-2517a652-kserve-c97fc9c99-2xg8w\" (UID: \"6a02b380-f2d7-4506-9786-5cea66be0a9f\") " pod="llm/e2e-same-id-b-2517a652-kserve-c97fc9c99-2xg8w"
Sep 10 19:51:53.086205 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:51:53.086187    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a02b380-f2d7-4506-9786-5cea66be0a9f-tls-certs\") pod \"e2e-same-id-b-2517a652-kserve-c97fc9c99-2xg8w\" (UID: \"6a02b380-f2d7-4506-9786-5cea66be0a9f\") " pod="llm/e2e-same-id-b-2517a652-kserve-c97fc9c99-2xg8w"
Sep 10 19:51:53.093531 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:51:53.093507    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjc4k\" (UniqueName: \"kubernetes.io/projected/6a02b380-f2d7-4506-9786-5cea66be0a9f-kube-api-access-pjc4k\") pod \"e2e-same-id-b-2517a652-kserve-c97fc9c99-2xg8w\" (UID: \"6a02b380-f2d7-4506-9786-5cea66be0a9f\") " pod="llm/e2e-same-id-b-2517a652-kserve-c97fc9c99-2xg8w"
Sep 10 19:51:53.242196 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:51:53.242104    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-2517a652-kserve-c97fc9c99-2xg8w"
Sep 10 19:51:53.514391 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:51:53.514367    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-2517a652-kserve-c97fc9c99-2xg8w"]
Sep 10 19:51:53.516253 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:51:53.516231    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6a02b380_f2d7_4506_9786_5cea66be0a9f.slice/crio-c8f26968a996cb9a240e41ad6fd83b686044cb4d14232638658e02c4e6594fb6 WatchSource:0}: Error finding container c8f26968a996cb9a240e41ad6fd83b686044cb4d14232638658e02c4e6594fb6: Status 404 returned error can't find the container with id c8f26968a996cb9a240e41ad6fd83b686044cb4d14232638658e02c4e6594fb6
Sep 10 19:51:54.039976 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:51:54.039941    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-2517a652-kserve-c97fc9c99-2xg8w" event={"ID":"6a02b380-f2d7-4506-9786-5cea66be0a9f","Type":"ContainerStarted","Data":"4d256402df6da73f8e5d3675607c69e35412acce5d9326a63f79a58f23db7d36"}
Sep 10 19:51:54.040370 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:51:54.039989    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-2517a652-kserve-c97fc9c99-2xg8w" event={"ID":"6a02b380-f2d7-4506-9786-5cea66be0a9f","Type":"ContainerStarted","Data":"c8f26968a996cb9a240e41ad6fd83b686044cb4d14232638658e02c4e6594fb6"}
Sep 10 19:51:54.062350 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:51:54.062290    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-2517a652-kserve-c97fc9c99-2xg8w" podStartSLOduration=2.062269204 podStartE2EDuration="2.062269204s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:51:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:51:54.059835603 +0000 UTC m=+2169.660742844" watchObservedRunningTime="2026-09-10 19:51:54.062269204 +0000 UTC m=+2169.663176447"
Sep 10 19:52:03.242956 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:03.242918    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-2517a652-kserve-c97fc9c99-2xg8w"
Sep 10 19:52:03.243427 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:03.243066    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-2517a652-kserve-c97fc9c99-2xg8w"
Sep 10 19:52:03.255812 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:03.255782    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-2517a652-kserve-c97fc9c99-2xg8w"
Sep 10 19:52:04.105859 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:04.105823    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-2517a652-kserve-c97fc9c99-2xg8w"
Sep 10 19:52:28.558303 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:28.558263    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:52:29.416948 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:29.416914    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:52:30.159642 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:30.159594    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-2517a652-kserve-c97fc9c99-2xg8w"]
Sep 10 19:52:30.200131 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:30.200098    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:52:45.180652 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:45.180590    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-2517a652-kserve-c97fc9c99-2xg8w" podUID="6a02b380-f2d7-4506-9786-5cea66be0a9f" containerName="main" containerID="cri-o://4d256402df6da73f8e5d3675607c69e35412acce5d9326a63f79a58f23db7d36" gracePeriod=45
Sep 10 19:52:45.309137 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:45.309107    2564 generic.go:356] "Generic (PLEG): container finished" podID="6a02b380-f2d7-4506-9786-5cea66be0a9f" containerID="4d256402df6da73f8e5d3675607c69e35412acce5d9326a63f79a58f23db7d36" exitCode=0
Sep 10 19:52:45.309137 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:45.309144    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-2517a652-kserve-c97fc9c99-2xg8w" event={"ID":"6a02b380-f2d7-4506-9786-5cea66be0a9f","Type":"ContainerDied","Data":"4d256402df6da73f8e5d3675607c69e35412acce5d9326a63f79a58f23db7d36"}
Sep 10 19:52:45.428934 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:45.428902    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-2517a652-kserve-c97fc9c99-2xg8w"
Sep 10 19:52:45.476281 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:45.476204    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a02b380-f2d7-4506-9786-5cea66be0a9f-dshm\") pod \"6a02b380-f2d7-4506-9786-5cea66be0a9f\" (UID: \"6a02b380-f2d7-4506-9786-5cea66be0a9f\") "
Sep 10 19:52:45.476281 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:45.476251    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6a02b380-f2d7-4506-9786-5cea66be0a9f-home\") pod \"6a02b380-f2d7-4506-9786-5cea66be0a9f\" (UID: \"6a02b380-f2d7-4506-9786-5cea66be0a9f\") "
Sep 10 19:52:45.476281 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:45.476279    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a02b380-f2d7-4506-9786-5cea66be0a9f-tls-certs\") pod \"6a02b380-f2d7-4506-9786-5cea66be0a9f\" (UID: \"6a02b380-f2d7-4506-9786-5cea66be0a9f\") "
Sep 10 19:52:45.476545 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:45.476300    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a02b380-f2d7-4506-9786-5cea66be0a9f-model-cache\") pod \"6a02b380-f2d7-4506-9786-5cea66be0a9f\" (UID: \"6a02b380-f2d7-4506-9786-5cea66be0a9f\") "
Sep 10 19:52:45.476545 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:45.476402    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pjc4k\" (UniqueName: \"kubernetes.io/projected/6a02b380-f2d7-4506-9786-5cea66be0a9f-kube-api-access-pjc4k\") pod \"6a02b380-f2d7-4506-9786-5cea66be0a9f\" (UID: \"6a02b380-f2d7-4506-9786-5cea66be0a9f\") "
Sep 10 19:52:45.476545 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:45.476457    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a02b380-f2d7-4506-9786-5cea66be0a9f-tmp-dir\") pod \"6a02b380-f2d7-4506-9786-5cea66be0a9f\" (UID: \"6a02b380-f2d7-4506-9786-5cea66be0a9f\") "
Sep 10 19:52:45.476726 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:45.476569    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a02b380-f2d7-4506-9786-5cea66be0a9f-home" (OuterVolumeSpecName: "home") pod "6a02b380-f2d7-4506-9786-5cea66be0a9f" (UID: "6a02b380-f2d7-4506-9786-5cea66be0a9f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:52:45.476726 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:45.476697    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a02b380-f2d7-4506-9786-5cea66be0a9f-model-cache" (OuterVolumeSpecName: "model-cache") pod "6a02b380-f2d7-4506-9786-5cea66be0a9f" (UID: "6a02b380-f2d7-4506-9786-5cea66be0a9f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:52:45.476831 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:45.476739    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a02b380-f2d7-4506-9786-5cea66be0a9f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6a02b380-f2d7-4506-9786-5cea66be0a9f" (UID: "6a02b380-f2d7-4506-9786-5cea66be0a9f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:52:45.477019 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:45.476973    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a02b380-f2d7-4506-9786-5cea66be0a9f-tmp-dir\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:52:45.477019 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:45.476997    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6a02b380-f2d7-4506-9786-5cea66be0a9f-home\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:52:45.477019 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:45.477008    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a02b380-f2d7-4506-9786-5cea66be0a9f-model-cache\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:52:45.478547 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:45.478527    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a02b380-f2d7-4506-9786-5cea66be0a9f-kube-api-access-pjc4k" (OuterVolumeSpecName: "kube-api-access-pjc4k") pod "6a02b380-f2d7-4506-9786-5cea66be0a9f" (UID: "6a02b380-f2d7-4506-9786-5cea66be0a9f"). InnerVolumeSpecName "kube-api-access-pjc4k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:52:45.478686 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:45.478578    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a02b380-f2d7-4506-9786-5cea66be0a9f-dshm" (OuterVolumeSpecName: "dshm") pod "6a02b380-f2d7-4506-9786-5cea66be0a9f" (UID: "6a02b380-f2d7-4506-9786-5cea66be0a9f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:52:45.478769 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:45.478719    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a02b380-f2d7-4506-9786-5cea66be0a9f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6a02b380-f2d7-4506-9786-5cea66be0a9f" (UID: "6a02b380-f2d7-4506-9786-5cea66be0a9f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:52:45.522421 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:45.522390    2564 scope.go:117] "RemoveContainer" containerID="4ae38d4fdddb58bcfa5ca609b5ddc8f1445024f689ce6c33cf4464b05191003c"
Sep 10 19:52:45.532446 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:45.532424    2564 scope.go:117] "RemoveContainer" containerID="f101030fb3affd3faae4bc9daced684d89b049a405f7d31d33ef3ae54b6d6891"
Sep 10 19:52:45.578031 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:45.578003    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a02b380-f2d7-4506-9786-5cea66be0a9f-dshm\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:52:45.578031 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:45.578029    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a02b380-f2d7-4506-9786-5cea66be0a9f-tls-certs\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:52:45.578223 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:45.578040    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pjc4k\" (UniqueName: \"kubernetes.io/projected/6a02b380-f2d7-4506-9786-5cea66be0a9f-kube-api-access-pjc4k\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:52:46.315678 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:46.315646    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-2517a652-kserve-c97fc9c99-2xg8w"
Sep 10 19:52:46.315678 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:46.315671    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-2517a652-kserve-c97fc9c99-2xg8w" event={"ID":"6a02b380-f2d7-4506-9786-5cea66be0a9f","Type":"ContainerDied","Data":"c8f26968a996cb9a240e41ad6fd83b686044cb4d14232638658e02c4e6594fb6"}
Sep 10 19:52:46.316163 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:46.315721    2564 scope.go:117] "RemoveContainer" containerID="4d256402df6da73f8e5d3675607c69e35412acce5d9326a63f79a58f23db7d36"
Sep 10 19:52:46.345304 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:46.345269    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-2517a652-kserve-c97fc9c99-2xg8w"]
Sep 10 19:52:46.349896 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:46.349869    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-2517a652-kserve-c97fc9c99-2xg8w"]
Sep 10 19:52:46.950354 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:46.950320    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a02b380-f2d7-4506-9786-5cea66be0a9f" path="/var/lib/kubelet/pods/6a02b380-f2d7-4506-9786-5cea66be0a9f/volumes"
Sep 10 19:52:49.132573 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:49.132537    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7d4dc54f4c-8jpcm"]
Sep 10 19:52:49.132983 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:49.132943    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a02b380-f2d7-4506-9786-5cea66be0a9f" containerName="main"
Sep 10 19:52:49.132983 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:49.132956    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a02b380-f2d7-4506-9786-5cea66be0a9f" containerName="main"
Sep 10 19:52:49.133058 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:49.133036    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a02b380-f2d7-4506-9786-5cea66be0a9f" containerName="main"
Sep 10 19:52:49.137927 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:49.137906    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-8jpcm"
Sep 10 19:52:49.141463 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:49.141431    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-gr54c\""
Sep 10 19:52:49.141463 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:49.141449    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 10 19:52:49.141700 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:49.141477    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 19:52:49.141700 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:49.141489    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 10 19:52:49.141700 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:49.141501    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 10 19:52:49.145603 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:49.145580    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7d4dc54f4c-8jpcm"]
Sep 10 19:52:49.210492 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:49.210452    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2d32d7d-0a0c-4f5c-b8ba-195ecc48802b-openshift-service-ca\") pod \"maas-api-7d4dc54f4c-8jpcm\" (UID: \"a2d32d7d-0a0c-4f5c-b8ba-195ecc48802b\") " pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-8jpcm"
Sep 10 19:52:49.210727 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:49.210503    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2d32d7d-0a0c-4f5c-b8ba-195ecc48802b-maas-api-tls\") pod \"maas-api-7d4dc54f4c-8jpcm\" (UID: \"a2d32d7d-0a0c-4f5c-b8ba-195ecc48802b\") " pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-8jpcm"
Sep 10 19:52:49.210727 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:49.210529    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g66lg\" (UniqueName: \"kubernetes.io/projected/a2d32d7d-0a0c-4f5c-b8ba-195ecc48802b-kube-api-access-g66lg\") pod \"maas-api-7d4dc54f4c-8jpcm\" (UID: \"a2d32d7d-0a0c-4f5c-b8ba-195ecc48802b\") " pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-8jpcm"
Sep 10 19:52:49.210727 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:49.210608    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2d32d7d-0a0c-4f5c-b8ba-195ecc48802b-metrics-certs\") pod \"maas-api-7d4dc54f4c-8jpcm\" (UID: \"a2d32d7d-0a0c-4f5c-b8ba-195ecc48802b\") " pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-8jpcm"
Sep 10 19:52:49.312141 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:49.312106    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2d32d7d-0a0c-4f5c-b8ba-195ecc48802b-openshift-service-ca\") pod \"maas-api-7d4dc54f4c-8jpcm\" (UID: \"a2d32d7d-0a0c-4f5c-b8ba-195ecc48802b\") " pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-8jpcm"
Sep 10 19:52:49.312330 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:49.312161    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2d32d7d-0a0c-4f5c-b8ba-195ecc48802b-maas-api-tls\") pod \"maas-api-7d4dc54f4c-8jpcm\" (UID: \"a2d32d7d-0a0c-4f5c-b8ba-195ecc48802b\") " pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-8jpcm"
Sep 10 19:52:49.312330 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:49.312196    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g66lg\" (UniqueName: \"kubernetes.io/projected/a2d32d7d-0a0c-4f5c-b8ba-195ecc48802b-kube-api-access-g66lg\") pod \"maas-api-7d4dc54f4c-8jpcm\" (UID: \"a2d32d7d-0a0c-4f5c-b8ba-195ecc48802b\") " pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-8jpcm"
Sep 10 19:52:49.312330 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:49.312222    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2d32d7d-0a0c-4f5c-b8ba-195ecc48802b-metrics-certs\") pod \"maas-api-7d4dc54f4c-8jpcm\" (UID: \"a2d32d7d-0a0c-4f5c-b8ba-195ecc48802b\") " pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-8jpcm"
Sep 10 19:52:49.312929 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:49.312901    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2d32d7d-0a0c-4f5c-b8ba-195ecc48802b-openshift-service-ca\") pod \"maas-api-7d4dc54f4c-8jpcm\" (UID: \"a2d32d7d-0a0c-4f5c-b8ba-195ecc48802b\") " pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-8jpcm"
Sep 10 19:52:49.314732 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:49.314706    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2d32d7d-0a0c-4f5c-b8ba-195ecc48802b-maas-api-tls\") pod \"maas-api-7d4dc54f4c-8jpcm\" (UID: \"a2d32d7d-0a0c-4f5c-b8ba-195ecc48802b\") " pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-8jpcm"
Sep 10 19:52:49.314839 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:49.314817    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2d32d7d-0a0c-4f5c-b8ba-195ecc48802b-metrics-certs\") pod \"maas-api-7d4dc54f4c-8jpcm\" (UID: \"a2d32d7d-0a0c-4f5c-b8ba-195ecc48802b\") " pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-8jpcm"
Sep 10 19:52:49.322781 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:49.322755    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g66lg\" (UniqueName: \"kubernetes.io/projected/a2d32d7d-0a0c-4f5c-b8ba-195ecc48802b-kube-api-access-g66lg\") pod \"maas-api-7d4dc54f4c-8jpcm\" (UID: \"a2d32d7d-0a0c-4f5c-b8ba-195ecc48802b\") " pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-8jpcm"
Sep 10 19:52:49.450644 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:49.450513    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-8jpcm"
Sep 10 19:52:49.733991 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:52:49.733959    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2d32d7d_0a0c_4f5c_b8ba_195ecc48802b.slice/crio-2588a2ac793409ebc0f045c19e2d0a9b64cfd134a98f32ef1f476b9899bfc7fd WatchSource:0}: Error finding container 2588a2ac793409ebc0f045c19e2d0a9b64cfd134a98f32ef1f476b9899bfc7fd: Status 404 returned error can't find the container with id 2588a2ac793409ebc0f045c19e2d0a9b64cfd134a98f32ef1f476b9899bfc7fd
Sep 10 19:52:49.735420 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:49.735389    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7d4dc54f4c-8jpcm"]
Sep 10 19:52:49.735767 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:49.735745    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 19:52:50.341457 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:50.341400    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-8jpcm" event={"ID":"a2d32d7d-0a0c-4f5c-b8ba-195ecc48802b","Type":"ContainerStarted","Data":"2588a2ac793409ebc0f045c19e2d0a9b64cfd134a98f32ef1f476b9899bfc7fd"}
Sep 10 19:52:52.353131 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:52.353084    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-8jpcm" event={"ID":"a2d32d7d-0a0c-4f5c-b8ba-195ecc48802b","Type":"ContainerStarted","Data":"65754b3e1dab83c5a02c3a4e6423d845aa70fca51b10a06379a6d6439cb58f3f"}
Sep 10 19:52:52.353531 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:52.353173    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-8jpcm"
Sep 10 19:52:52.372988 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:52.372934    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-8jpcm" podStartSLOduration=1.707540017 podStartE2EDuration="3.372918452s" totalImagesPullingTime="1.665378435s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:52:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:52:52.371066802 +0000 UTC m=+2227.971974042" watchObservedRunningTime="2026-09-10 19:52:52.372918452 +0000 UTC m=+2227.973825692"
Sep 10 19:52:56.730528 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:56.730479    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-7d4dc54f4c-8jpcm"]
Sep 10 19:52:56.731183 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:56.731150    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-8jpcm" podUID="a2d32d7d-0a0c-4f5c-b8ba-195ecc48802b" containerName="maas-api" containerID="cri-o://65754b3e1dab83c5a02c3a4e6423d845aa70fca51b10a06379a6d6439cb58f3f" gracePeriod=30
Sep 10 19:52:56.736311 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:56.736288    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-8jpcm"
Sep 10 19:52:56.989839 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:56.989776    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-8jpcm"
Sep 10 19:52:57.092991 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:57.092951    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2d32d7d-0a0c-4f5c-b8ba-195ecc48802b-openshift-service-ca\") pod \"a2d32d7d-0a0c-4f5c-b8ba-195ecc48802b\" (UID: \"a2d32d7d-0a0c-4f5c-b8ba-195ecc48802b\") "
Sep 10 19:52:57.093199 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:57.093036    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g66lg\" (UniqueName: \"kubernetes.io/projected/a2d32d7d-0a0c-4f5c-b8ba-195ecc48802b-kube-api-access-g66lg\") pod \"a2d32d7d-0a0c-4f5c-b8ba-195ecc48802b\" (UID: \"a2d32d7d-0a0c-4f5c-b8ba-195ecc48802b\") "
Sep 10 19:52:57.093199 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:57.093065    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2d32d7d-0a0c-4f5c-b8ba-195ecc48802b-maas-api-tls\") pod \"a2d32d7d-0a0c-4f5c-b8ba-195ecc48802b\" (UID: \"a2d32d7d-0a0c-4f5c-b8ba-195ecc48802b\") "
Sep 10 19:52:57.093199 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:57.093143    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2d32d7d-0a0c-4f5c-b8ba-195ecc48802b-metrics-certs\") pod \"a2d32d7d-0a0c-4f5c-b8ba-195ecc48802b\" (UID: \"a2d32d7d-0a0c-4f5c-b8ba-195ecc48802b\") "
Sep 10 19:52:57.093405 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:57.093373    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2d32d7d-0a0c-4f5c-b8ba-195ecc48802b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a2d32d7d-0a0c-4f5c-b8ba-195ecc48802b" (UID: "a2d32d7d-0a0c-4f5c-b8ba-195ecc48802b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:52:57.095351 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:57.095318    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2d32d7d-0a0c-4f5c-b8ba-195ecc48802b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a2d32d7d-0a0c-4f5c-b8ba-195ecc48802b" (UID: "a2d32d7d-0a0c-4f5c-b8ba-195ecc48802b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:52:57.095453 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:57.095381    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2d32d7d-0a0c-4f5c-b8ba-195ecc48802b-kube-api-access-g66lg" (OuterVolumeSpecName: "kube-api-access-g66lg") pod "a2d32d7d-0a0c-4f5c-b8ba-195ecc48802b" (UID: "a2d32d7d-0a0c-4f5c-b8ba-195ecc48802b"). InnerVolumeSpecName "kube-api-access-g66lg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:52:57.095453 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:57.095397    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2d32d7d-0a0c-4f5c-b8ba-195ecc48802b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a2d32d7d-0a0c-4f5c-b8ba-195ecc48802b" (UID: "a2d32d7d-0a0c-4f5c-b8ba-195ecc48802b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:52:57.194183 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:57.194124    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2d32d7d-0a0c-4f5c-b8ba-195ecc48802b-openshift-service-ca\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:52:57.194183 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:57.194155    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g66lg\" (UniqueName: \"kubernetes.io/projected/a2d32d7d-0a0c-4f5c-b8ba-195ecc48802b-kube-api-access-g66lg\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:52:57.194183 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:57.194167    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2d32d7d-0a0c-4f5c-b8ba-195ecc48802b-maas-api-tls\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:52:57.194183 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:57.194175    2564 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2d32d7d-0a0c-4f5c-b8ba-195ecc48802b-metrics-certs\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 10 19:52:57.386129 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:57.386089    2564 generic.go:356] "Generic (PLEG): container finished" podID="a2d32d7d-0a0c-4f5c-b8ba-195ecc48802b" containerID="65754b3e1dab83c5a02c3a4e6423d845aa70fca51b10a06379a6d6439cb58f3f" exitCode=0
Sep 10 19:52:57.386129 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:57.386131    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-8jpcm"
Sep 10 19:52:57.386404 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:57.386142    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-8jpcm" event={"ID":"a2d32d7d-0a0c-4f5c-b8ba-195ecc48802b","Type":"ContainerDied","Data":"65754b3e1dab83c5a02c3a4e6423d845aa70fca51b10a06379a6d6439cb58f3f"}
Sep 10 19:52:57.386404 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:57.386193    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-8jpcm" event={"ID":"a2d32d7d-0a0c-4f5c-b8ba-195ecc48802b","Type":"ContainerDied","Data":"2588a2ac793409ebc0f045c19e2d0a9b64cfd134a98f32ef1f476b9899bfc7fd"}
Sep 10 19:52:57.386404 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:57.386215    2564 scope.go:117] "RemoveContainer" containerID="65754b3e1dab83c5a02c3a4e6423d845aa70fca51b10a06379a6d6439cb58f3f"
Sep 10 19:52:57.397924 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:57.397894    2564 scope.go:117] "RemoveContainer" containerID="65754b3e1dab83c5a02c3a4e6423d845aa70fca51b10a06379a6d6439cb58f3f"
Sep 10 19:52:57.398289 ip-10-0-129-230 kubenswrapper[2564]: E0910 19:52:57.398268    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65754b3e1dab83c5a02c3a4e6423d845aa70fca51b10a06379a6d6439cb58f3f\": container with ID starting with 65754b3e1dab83c5a02c3a4e6423d845aa70fca51b10a06379a6d6439cb58f3f not found: ID does not exist" containerID="65754b3e1dab83c5a02c3a4e6423d845aa70fca51b10a06379a6d6439cb58f3f"
Sep 10 19:52:57.398370 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:57.398299    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65754b3e1dab83c5a02c3a4e6423d845aa70fca51b10a06379a6d6439cb58f3f"} err="failed to get container status \"65754b3e1dab83c5a02c3a4e6423d845aa70fca51b10a06379a6d6439cb58f3f\": rpc error: code = NotFound desc = could not find container \"65754b3e1dab83c5a02c3a4e6423d845aa70fca51b10a06379a6d6439cb58f3f\": container with ID starting with 65754b3e1dab83c5a02c3a4e6423d845aa70fca51b10a06379a6d6439cb58f3f not found: ID does not exist"
Sep 10 19:52:57.412302 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:57.412257    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-7d4dc54f4c-8jpcm"]
Sep 10 19:52:57.414540 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:57.414511    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-7d4dc54f4c-8jpcm"]
Sep 10 19:52:58.949136 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:52:58.949105    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2d32d7d-0a0c-4f5c-b8ba-195ecc48802b" path="/var/lib/kubelet/pods/a2d32d7d-0a0c-4f5c-b8ba-195ecc48802b/volumes"
Sep 10 19:53:28.514760 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:28.514725    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-tv6rf_b046a9c4-10fd-4387-b304-a57f379d8314/manager/0.log"
Sep 10 19:53:28.634048 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:28.634014    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-k55hx_437ff782-378c-4a6e-a7ce-3ec34f48a472/manager/0.log"
Sep 10 19:53:28.905583 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:28.905505    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-96fp6_6f4e21f1-9f35-4dda-9b62-f32d6b4a23c2/server/0.log"
Sep 10 19:53:29.023316 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:29.023285    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-48fv6_5c5b1c30-dce1-463a-9739-3cc9b306dfe5/manager/0.log"
Sep 10 19:53:29.281311 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:29.281287    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-55556678b8-vpjt8_99764670-0b23-4b19-b3fe-cffb58fd825b/copy-manifests/0.log"
Sep 10 19:53:29.289998 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:29.289974    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-55556678b8-vpjt8_99764670-0b23-4b19-b3fe-cffb58fd825b/manager/0.log"
Sep 10 19:53:29.982915 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:29.982877    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29817810-svwf6_50e21465-bf07-4a6c-89f0-b3214b0b62f9/cleanup/0.log"
Sep 10 19:53:30.094799 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:30.094771    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29817825-sb2x8_04e90788-4814-4c5e-9d46-8df1cb15563e/cleanup/0.log"
Sep 10 19:53:31.126985 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:31.126937    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-zgt99_e66da5a2-eef7-4e40-b4db-46f69585732e/manager/0.log"
Sep 10 19:53:31.246021 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:31.245985    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-cbqrk_834ae1eb-c324-493f-9105-a3d5c10d3c1d/manager/0.log"
Sep 10 19:53:31.489987 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:31.489912    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-96s67_0fb6597a-3c59-4314-a4d4-48366678dd44/manager/0.log"
Sep 10 19:53:31.724479 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:31.724448    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-psz7d_9be52691-ec26-45cf-80e9-fa75c08b47fb/manager/0.log"
Sep 10 19:53:32.090058 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:32.090028    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j_b34759e7-f5de-46c2-9f9f-9d3b3dd3200f/istio-proxy/0.log"
Sep 10 19:53:32.454441 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:32.454356    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-64954c75b6-htzd4_518a84e9-05e6-43d2-b478-1a2885a29dce/kube-auth-proxy/0.log"
Sep 10 19:53:32.570808 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:32.570776    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-8r27k_c5fe5daa-4a4f-45c8-9aaf-e83382f78471/istio-proxy/1.log"
Sep 10 19:53:32.688822 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:32.688789    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-plw9s_6428c980-c9b3-46f4-b29d-d11aa11fca18/payload-pre-processing/0.log"
Sep 10 19:53:32.815752 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:32.815726    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-ll4jr_f80fdb52-770e-4a98-904f-556d9a3d9b28/payload-processing/0.log"
Sep 10 19:53:33.411873 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:33.411837    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-nc2pk_8612f429-9013-4747-a3a5-b680fedc3021/main/0.log"
Sep 10 19:53:33.638220 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:33.638188    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-t7fwh_f937b290-1298-4e30-b917-0b8e83243444/main/0.log"
Sep 10 19:53:40.017164 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:40.017132    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:53:40.118684 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:40.118607    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log" limit=1024
Sep 10 19:53:43.125848 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:43.125820    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-vcs72_80a423f3-63d3-42fa-a233-166ab675777e/global-pull-secret-syncer/0.log"
Sep 10 19:53:43.190513 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:43.190479    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-lhhr2_6e38a77e-c0f7-43c1-a915-fd6a036cbb0e/konnectivity-agent/0.log"
Sep 10 19:53:43.305117 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:43.305078    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-230.ec2.internal_099d282a8bec5dd8dd074b5f725a2d0c/haproxy/0.log"
Sep 10 19:53:45.557831 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:45.557803    2564 scope.go:117] "RemoveContainer" containerID="d4b10eb3d9bde5a96f5947f05636069aab77f16cce0b5bd1e59cd6ffdabb256f"
Sep 10 19:53:48.060002 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:48.059975    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-zgt99_e66da5a2-eef7-4e40-b4db-46f69585732e/manager/0.log"
Sep 10 19:53:48.091179 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:48.091152    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-cbqrk_834ae1eb-c324-493f-9105-a3d5c10d3c1d/manager/0.log"
Sep 10 19:53:48.163190 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:48.163156    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-96s67_0fb6597a-3c59-4314-a4d4-48366678dd44/manager/0.log"
Sep 10 19:53:48.250916 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:48.250894    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-psz7d_9be52691-ec26-45cf-80e9-fa75c08b47fb/manager/0.log"
Sep 10 19:53:50.130818 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:50.130790    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-qgvtq_eeec0ca3-75b9-4590-a29e-609fa17b7e7e/kube-state-metrics/0.log"
Sep 10 19:53:50.156344 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:50.156313    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-qgvtq_eeec0ca3-75b9-4590-a29e-609fa17b7e7e/kube-rbac-proxy-main/0.log"
Sep 10 19:53:50.179557 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:50.179532    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-qgvtq_eeec0ca3-75b9-4590-a29e-609fa17b7e7e/kube-rbac-proxy-self/0.log"
Sep 10 19:53:50.210286 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:50.210255    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-7578c5ccf6-wpp2l_69415242-9c1a-4b21-bcf2-a4afb82cdbd2/metrics-server/0.log"
Sep 10 19:53:50.450522 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:50.450497    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-k72xf_ee547a46-3813-4faa-a7bd-7e05be04e865/node-exporter/0.log"
Sep 10 19:53:50.474431 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:50.474406    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-k72xf_ee547a46-3813-4faa-a7bd-7e05be04e865/kube-rbac-proxy/0.log"
Sep 10 19:53:50.503812 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:50.503789    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-k72xf_ee547a46-3813-4faa-a7bd-7e05be04e865/init-textfile/0.log"
Sep 10 19:53:50.635569 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:50.635486    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c121a589-1848-4341-a3a0-5a4627dee824/prometheus/0.log"
Sep 10 19:53:50.663284 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:50.663262    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c121a589-1848-4341-a3a0-5a4627dee824/config-reloader/0.log"
Sep 10 19:53:50.693871 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:50.693846    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c121a589-1848-4341-a3a0-5a4627dee824/thanos-sidecar/0.log"
Sep 10 19:53:50.718491 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:50.718469    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c121a589-1848-4341-a3a0-5a4627dee824/kube-rbac-proxy-web/0.log"
Sep 10 19:53:50.741894 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:50.741865    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c121a589-1848-4341-a3a0-5a4627dee824/kube-rbac-proxy/0.log"
Sep 10 19:53:50.766508 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:50.766488    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c121a589-1848-4341-a3a0-5a4627dee824/kube-rbac-proxy-thanos/0.log"
Sep 10 19:53:50.793770 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:50.793737    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c121a589-1848-4341-a3a0-5a4627dee824/init-config-reloader/0.log"
Sep 10 19:53:50.836459 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:50.836438    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-5ctrq_86305da4-90a4-46c4-89c2-57af64a862fd/prometheus-operator/0.log"
Sep 10 19:53:50.872561 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:50.872541    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-5ctrq_86305da4-90a4-46c4-89c2-57af64a862fd/kube-rbac-proxy/0.log"
Sep 10 19:53:54.010391 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:54.010359    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-85d98c998f-ssx24_ec666f15-107e-4574-9d6c-cc73b6840c2e/download-server/0.log"
Sep 10 19:53:54.026785 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:54.026756    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8bt8w/perf-node-gather-daemonset-w4vk5"]
Sep 10 19:53:54.027295 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:54.027256    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2d32d7d-0a0c-4f5c-b8ba-195ecc48802b" containerName="maas-api"
Sep 10 19:53:54.027295 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:54.027278    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2d32d7d-0a0c-4f5c-b8ba-195ecc48802b" containerName="maas-api"
Sep 10 19:53:54.027487 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:54.027355    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2d32d7d-0a0c-4f5c-b8ba-195ecc48802b" containerName="maas-api"
Sep 10 19:53:54.030426 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:54.030410    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8bt8w/perf-node-gather-daemonset-w4vk5"
Sep 10 19:53:54.032743 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:54.032725    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8bt8w\"/\"kube-root-ca.crt\""
Sep 10 19:53:54.033670 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:54.033654    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-8bt8w\"/\"default-dockercfg-ss8rz\""
Sep 10 19:53:54.033779 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:54.033675    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8bt8w\"/\"openshift-service-ca.crt\""
Sep 10 19:53:54.039765 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:54.039744    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8bt8w/perf-node-gather-daemonset-w4vk5"]
Sep 10 19:53:54.215610 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:54.215569    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4456f65c-0be1-4777-88cb-6c6205e242c5-podres\") pod \"perf-node-gather-daemonset-w4vk5\" (UID: \"4456f65c-0be1-4777-88cb-6c6205e242c5\") " pod="openshift-must-gather-8bt8w/perf-node-gather-daemonset-w4vk5"
Sep 10 19:53:54.215610 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:54.215617    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4456f65c-0be1-4777-88cb-6c6205e242c5-lib-modules\") pod \"perf-node-gather-daemonset-w4vk5\" (UID: \"4456f65c-0be1-4777-88cb-6c6205e242c5\") " pod="openshift-must-gather-8bt8w/perf-node-gather-daemonset-w4vk5"
Sep 10 19:53:54.215896 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:54.215685    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7z5qd\" (UniqueName: \"kubernetes.io/projected/4456f65c-0be1-4777-88cb-6c6205e242c5-kube-api-access-7z5qd\") pod \"perf-node-gather-daemonset-w4vk5\" (UID: \"4456f65c-0be1-4777-88cb-6c6205e242c5\") " pod="openshift-must-gather-8bt8w/perf-node-gather-daemonset-w4vk5"
Sep 10 19:53:54.215896 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:54.215713    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4456f65c-0be1-4777-88cb-6c6205e242c5-proc\") pod \"perf-node-gather-daemonset-w4vk5\" (UID: \"4456f65c-0be1-4777-88cb-6c6205e242c5\") " pod="openshift-must-gather-8bt8w/perf-node-gather-daemonset-w4vk5"
Sep 10 19:53:54.215896 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:54.215768    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4456f65c-0be1-4777-88cb-6c6205e242c5-sys\") pod \"perf-node-gather-daemonset-w4vk5\" (UID: \"4456f65c-0be1-4777-88cb-6c6205e242c5\") " pod="openshift-must-gather-8bt8w/perf-node-gather-daemonset-w4vk5"
Sep 10 19:53:54.316759 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:54.316662    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4456f65c-0be1-4777-88cb-6c6205e242c5-podres\") pod \"perf-node-gather-daemonset-w4vk5\" (UID: \"4456f65c-0be1-4777-88cb-6c6205e242c5\") " pod="openshift-must-gather-8bt8w/perf-node-gather-daemonset-w4vk5"
Sep 10 19:53:54.316759 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:54.316708    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4456f65c-0be1-4777-88cb-6c6205e242c5-lib-modules\") pod \"perf-node-gather-daemonset-w4vk5\" (UID: \"4456f65c-0be1-4777-88cb-6c6205e242c5\") " pod="openshift-must-gather-8bt8w/perf-node-gather-daemonset-w4vk5"
Sep 10 19:53:54.316991 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:54.316830    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4456f65c-0be1-4777-88cb-6c6205e242c5-lib-modules\") pod \"perf-node-gather-daemonset-w4vk5\" (UID: \"4456f65c-0be1-4777-88cb-6c6205e242c5\") " pod="openshift-must-gather-8bt8w/perf-node-gather-daemonset-w4vk5"
Sep 10 19:53:54.316991 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:54.316835    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4456f65c-0be1-4777-88cb-6c6205e242c5-podres\") pod \"perf-node-gather-daemonset-w4vk5\" (UID: \"4456f65c-0be1-4777-88cb-6c6205e242c5\") " pod="openshift-must-gather-8bt8w/perf-node-gather-daemonset-w4vk5"
Sep 10 19:53:54.316991 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:54.316834    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7z5qd\" (UniqueName: \"kubernetes.io/projected/4456f65c-0be1-4777-88cb-6c6205e242c5-kube-api-access-7z5qd\") pod \"perf-node-gather-daemonset-w4vk5\" (UID: \"4456f65c-0be1-4777-88cb-6c6205e242c5\") " pod="openshift-must-gather-8bt8w/perf-node-gather-daemonset-w4vk5"
Sep 10 19:53:54.316991 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:54.316888    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4456f65c-0be1-4777-88cb-6c6205e242c5-proc\") pod \"perf-node-gather-daemonset-w4vk5\" (UID: \"4456f65c-0be1-4777-88cb-6c6205e242c5\") " pod="openshift-must-gather-8bt8w/perf-node-gather-daemonset-w4vk5"
Sep 10 19:53:54.316991 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:54.316921    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4456f65c-0be1-4777-88cb-6c6205e242c5-sys\") pod \"perf-node-gather-daemonset-w4vk5\" (UID: \"4456f65c-0be1-4777-88cb-6c6205e242c5\") " pod="openshift-must-gather-8bt8w/perf-node-gather-daemonset-w4vk5"
Sep 10 19:53:54.317224 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:54.316999    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4456f65c-0be1-4777-88cb-6c6205e242c5-proc\") pod \"perf-node-gather-daemonset-w4vk5\" (UID: \"4456f65c-0be1-4777-88cb-6c6205e242c5\") " pod="openshift-must-gather-8bt8w/perf-node-gather-daemonset-w4vk5"
Sep 10 19:53:54.317224 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:54.317013    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4456f65c-0be1-4777-88cb-6c6205e242c5-sys\") pod \"perf-node-gather-daemonset-w4vk5\" (UID: \"4456f65c-0be1-4777-88cb-6c6205e242c5\") " pod="openshift-must-gather-8bt8w/perf-node-gather-daemonset-w4vk5"
Sep 10 19:53:54.325786 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:54.325760    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7z5qd\" (UniqueName: \"kubernetes.io/projected/4456f65c-0be1-4777-88cb-6c6205e242c5-kube-api-access-7z5qd\") pod \"perf-node-gather-daemonset-w4vk5\" (UID: \"4456f65c-0be1-4777-88cb-6c6205e242c5\") " pod="openshift-must-gather-8bt8w/perf-node-gather-daemonset-w4vk5"
Sep 10 19:53:54.340714 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:54.340689    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8bt8w/perf-node-gather-daemonset-w4vk5"
Sep 10 19:53:54.610259 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:54.610232    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8bt8w/perf-node-gather-daemonset-w4vk5"]
Sep 10 19:53:54.613037 ip-10-0-129-230 kubenswrapper[2564]: W0910 19:53:54.613008    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod4456f65c_0be1_4777_88cb_6c6205e242c5.slice/crio-0d685740258cd156fee961c0c7de4ed750ee7cb359b784e2448b01cb82f69346 WatchSource:0}: Error finding container 0d685740258cd156fee961c0c7de4ed750ee7cb359b784e2448b01cb82f69346: Status 404 returned error can't find the container with id 0d685740258cd156fee961c0c7de4ed750ee7cb359b784e2448b01cb82f69346
Sep 10 19:53:54.688225 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:54.688192    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8bt8w/perf-node-gather-daemonset-w4vk5" event={"ID":"4456f65c-0be1-4777-88cb-6c6205e242c5","Type":"ContainerStarted","Data":"42097d5ad08151e1f22163078e32b0f1c145c0c7541613fea0805dbb158ff63d"}
Sep 10 19:53:54.688373 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:54.688255    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-8bt8w/perf-node-gather-daemonset-w4vk5"
Sep 10 19:53:54.688373 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:54.688273    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8bt8w/perf-node-gather-daemonset-w4vk5" event={"ID":"4456f65c-0be1-4777-88cb-6c6205e242c5","Type":"ContainerStarted","Data":"0d685740258cd156fee961c0c7de4ed750ee7cb359b784e2448b01cb82f69346"}
Sep 10 19:53:54.705962 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:54.705918    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-8bt8w/perf-node-gather-daemonset-w4vk5" podStartSLOduration=0.705903444 podStartE2EDuration="705.903444ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:53:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:53:54.703679834 +0000 UTC m=+2290.304587076" watchObservedRunningTime="2026-09-10 19:53:54.705903444 +0000 UTC m=+2290.306810689"
Sep 10 19:53:55.902403 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:55.902371    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-67nsx_28f7bb38-4380-48b2-8881-38f1b188b5c8/dns/0.log"
Sep 10 19:53:55.925008 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:55.924983    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-67nsx_28f7bb38-4380-48b2-8881-38f1b188b5c8/kube-rbac-proxy/0.log"
Sep 10 19:53:56.108422 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:56.108396    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-l8b8c_e08684c6-1cc0-4f4c-a4cd-1fea1e46339a/dns-node-resolver/0.log"
Sep 10 19:53:56.691964 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:56.691931    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-54895dc98-2h4rn_815cb529-d322-4a64-a0c3-35de02c583ee/registry/0.log"
Sep 10 19:53:56.741057 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:56.741031    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7s7hj_ae8c5b13-1925-4df6-b3a2-d4ace9a51801/node-ca/0.log"
Sep 10 19:53:57.827669 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:57.827639    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j_b34759e7-f5de-46c2-9f9f-9d3b3dd3200f/istio-proxy/0.log"
Sep 10 19:53:58.315960 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:58.315933    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-64954c75b6-htzd4_518a84e9-05e6-43d2-b478-1a2885a29dce/kube-auth-proxy/0.log"
Sep 10 19:53:58.344605 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:58.344575    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-8r27k_c5fe5daa-4a4f-45c8-9aaf-e83382f78471/istio-proxy/0.log"
Sep 10 19:53:58.352826 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:58.352801    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-8r27k_c5fe5daa-4a4f-45c8-9aaf-e83382f78471/istio-proxy/1.log"
Sep 10 19:53:58.391355 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:58.391325    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-plw9s_6428c980-c9b3-46f4-b29d-d11aa11fca18/payload-pre-processing/0.log"
Sep 10 19:53:58.426236 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:58.426208    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-ll4jr_f80fdb52-770e-4a98-904f-556d9a3d9b28/payload-processing/0.log"
Sep 10 19:53:59.388068 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:59.388037    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tc2c7_90e5b1e2-b5f3-4306-8e86-363010cceb60/serve-healthcheck-canary/0.log"
Sep 10 19:53:59.971178 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:59.971151    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5v4m7_a66669df-07a2-4835-ac05-17a37bcfda7c/kube-rbac-proxy/0.log"
Sep 10 19:53:59.994528 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:53:59.994503    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5v4m7_a66669df-07a2-4835-ac05-17a37bcfda7c/exporter/0.log"
Sep 10 19:54:00.016480 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:54:00.016453    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5v4m7_a66669df-07a2-4835-ac05-17a37bcfda7c/extractor/0.log"
Sep 10 19:54:00.702381 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:54:00.702349    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-8bt8w/perf-node-gather-daemonset-w4vk5"
Sep 10 19:54:02.357033 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:54:02.357005    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-tv6rf_b046a9c4-10fd-4387-b304-a57f379d8314/manager/0.log"
Sep 10 19:54:02.524800 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:54:02.524768    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-k55hx_437ff782-378c-4a6e-a7ce-3ec34f48a472/manager/0.log"
Sep 10 19:54:02.626007 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:54:02.625925    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-96fp6_6f4e21f1-9f35-4dda-9b62-f32d6b4a23c2/server/0.log"
Sep 10 19:54:02.712086 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:54:02.712058    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-48fv6_5c5b1c30-dce1-463a-9739-3cc9b306dfe5/manager/0.log"
Sep 10 19:54:02.880057 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:54:02.879981    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-55556678b8-vpjt8_99764670-0b23-4b19-b3fe-cffb58fd825b/manager/0.log"
Sep 10 19:54:02.911005 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:54:02.910975    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-55556678b8-vpjt8_99764670-0b23-4b19-b3fe-cffb58fd825b/copy-manifests/0.log"
Sep 10 19:54:04.522817 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:54:04.522790    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-s25nx_ac59f2f5-eeae-4b22-a3e5-4d83179e9f0c/openshift-lws-operator/0.log"
Sep 10 19:54:04.575999 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:54:04.575970    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j_b34759e7-f5de-46c2-9f9f-9d3b3dd3200f/istio-proxy/0.log"
Sep 10 19:54:05.022072 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:54:05.022038    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-64954c75b6-htzd4_518a84e9-05e6-43d2-b478-1a2885a29dce/kube-auth-proxy/0.log"
Sep 10 19:54:05.042700 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:54:05.042670    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-8r27k_c5fe5daa-4a4f-45c8-9aaf-e83382f78471/istio-proxy/0.log"
Sep 10 19:54:05.050012 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:54:05.049989    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-8r27k_c5fe5daa-4a4f-45c8-9aaf-e83382f78471/istio-proxy/1.log"
Sep 10 19:54:05.079142 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:54:05.079113    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-plw9s_6428c980-c9b3-46f4-b29d-d11aa11fca18/payload-pre-processing/0.log"
Sep 10 19:54:05.107740 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:54:05.107694    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-ll4jr_f80fdb52-770e-4a98-904f-556d9a3d9b28/payload-processing/0.log"
Sep 10 19:54:06.689634 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:54:06.689583    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-pbh5j_b34759e7-f5de-46c2-9f9f-9d3b3dd3200f/istio-proxy/0.log"
Sep 10 19:54:07.152343 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:54:07.152313    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-64954c75b6-htzd4_518a84e9-05e6-43d2-b478-1a2885a29dce/kube-auth-proxy/0.log"
Sep 10 19:54:07.178865 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:54:07.178833    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-8r27k_c5fe5daa-4a4f-45c8-9aaf-e83382f78471/istio-proxy/0.log"
Sep 10 19:54:07.186558 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:54:07.186529    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-8r27k_c5fe5daa-4a4f-45c8-9aaf-e83382f78471/istio-proxy/1.log"
Sep 10 19:54:07.212028 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:54:07.211997    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-plw9s_6428c980-c9b3-46f4-b29d-d11aa11fca18/payload-pre-processing/0.log"
Sep 10 19:54:07.238911 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:54:07.238879    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-ll4jr_f80fdb52-770e-4a98-904f-556d9a3d9b28/payload-processing/0.log"
Sep 10 19:54:11.445838 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:54:11.445811    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mzwh9_ac0ef973-45eb-4f42-8c5d-d86695e32507/kube-multus-additional-cni-plugins/0.log"
Sep 10 19:54:11.468772 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:54:11.468747    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mzwh9_ac0ef973-45eb-4f42-8c5d-d86695e32507/egress-router-binary-copy/0.log"
Sep 10 19:54:11.490039 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:54:11.490017    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mzwh9_ac0ef973-45eb-4f42-8c5d-d86695e32507/cni-plugins/0.log"
Sep 10 19:54:11.511568 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:54:11.511544    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mzwh9_ac0ef973-45eb-4f42-8c5d-d86695e32507/bond-cni-plugin/0.log"
Sep 10 19:54:11.533616 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:54:11.533597    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mzwh9_ac0ef973-45eb-4f42-8c5d-d86695e32507/routeoverride-cni/0.log"
Sep 10 19:54:11.556796 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:54:11.556774    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mzwh9_ac0ef973-45eb-4f42-8c5d-d86695e32507/whereabouts-cni-bincopy/0.log"
Sep 10 19:54:11.577776 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:54:11.577749    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mzwh9_ac0ef973-45eb-4f42-8c5d-d86695e32507/whereabouts-cni/0.log"
Sep 10 19:54:11.898876 ip-10-0-129-230 kubenswrapper[2564]: I0910 19:54:11.898844    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-pvhzx_c89c4c0a-726d-4d2d-ab72-d3cfc4c907c2/kube-multus/0.log"