Jul 27 19:26:25.976273 ip-10-0-129-235 systemd[1]: Starting Kubernetes Kubelet...
Jul 27 19:26:26.357749 ip-10-0-129-235 kubenswrapper[2558]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 27 19:26:26.357749 ip-10-0-129-235 kubenswrapper[2558]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 27 19:26:26.357749 ip-10-0-129-235 kubenswrapper[2558]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 27 19:26:26.357749 ip-10-0-129-235 kubenswrapper[2558]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 27 19:26:26.357749 ip-10-0-129-235 kubenswrapper[2558]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 27 19:26:26.359379 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.359276    2558 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 27 19:26:26.363412 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363386    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 19:26:26.363412 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363412    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 19:26:26.363492 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363418    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 19:26:26.363492 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363422    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 19:26:26.363492 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363426    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 19:26:26.363492 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363429    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 19:26:26.363492 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363433    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 19:26:26.363492 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363435    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 19:26:26.363492 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363438    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 19:26:26.363492 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363441    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 19:26:26.363492 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363443    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 19:26:26.363492 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363463    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 19:26:26.363492 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363466    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 19:26:26.363492 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363469    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 19:26:26.363492 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363471    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 19:26:26.363492 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363474    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 19:26:26.363492 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363477    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 19:26:26.363492 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363479    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 19:26:26.363492 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363483    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 19:26:26.363492 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363485    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 19:26:26.363492 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363488    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 19:26:26.363492 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363491    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 19:26:26.363994 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363494    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 19:26:26.363994 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363497    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 19:26:26.363994 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363500    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 19:26:26.363994 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363503    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 19:26:26.363994 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363506    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 19:26:26.363994 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363511    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 19:26:26.363994 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363513    2558 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 19:26:26.363994 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363516    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 19:26:26.363994 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363518    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 19:26:26.363994 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363521    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 19:26:26.363994 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363523    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 19:26:26.363994 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363526    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 19:26:26.363994 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363528    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 19:26:26.363994 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363531    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 19:26:26.363994 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363534    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 19:26:26.363994 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363536    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 19:26:26.363994 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363539    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 19:26:26.363994 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363541    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 19:26:26.363994 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363543    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 19:26:26.363994 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363546    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 19:26:26.364473 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363548    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 19:26:26.364473 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363550    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 19:26:26.364473 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363553    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 19:26:26.364473 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363555    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 19:26:26.364473 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363558    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 19:26:26.364473 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363560    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 19:26:26.364473 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363562    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 19:26:26.364473 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363565    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 19:26:26.364473 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363567    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 19:26:26.364473 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363570    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 19:26:26.364473 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363572    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 19:26:26.364473 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363575    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 19:26:26.364473 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363577    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 19:26:26.364473 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363581    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 19:26:26.364473 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363584    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 19:26:26.364473 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363586    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 19:26:26.364473 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363589    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 19:26:26.364473 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363591    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 19:26:26.364473 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363594    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 19:26:26.364473 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363597    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 19:26:26.365025 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363599    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 19:26:26.365025 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363601    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 19:26:26.365025 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363604    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 19:26:26.365025 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363606    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 19:26:26.365025 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363609    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 19:26:26.365025 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363611    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 19:26:26.365025 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363614    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 19:26:26.365025 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363617    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 19:26:26.365025 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363619    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 19:26:26.365025 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363621    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 19:26:26.365025 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363624    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 19:26:26.365025 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363626    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 19:26:26.365025 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363629    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 19:26:26.365025 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363631    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 19:26:26.365025 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363634    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 19:26:26.365025 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363636    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 19:26:26.365025 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363639    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 19:26:26.365025 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363643    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 19:26:26.365025 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363645    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 19:26:26.365789 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363648    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 19:26:26.365789 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363650    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 19:26:26.365789 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363653    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 19:26:26.365789 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363655    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 19:26:26.365789 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363657    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 19:26:26.365789 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363660    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 19:26:26.365789 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363665    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 19:26:26.365789 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363667    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 19:26:26.365789 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363670    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 19:26:26.365789 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363673    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 19:26:26.365789 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363675    2558 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 19:26:26.365789 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363678    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 19:26:26.365789 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363681    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 19:26:26.365789 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363684    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 19:26:26.365789 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363686    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 19:26:26.365789 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363691    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 19:26:26.365789 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363694    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 19:26:26.365789 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.363698    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 19:26:26.365789 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364478    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 19:26:26.366388 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364487    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 19:26:26.366388 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364490    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 19:26:26.366388 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364494    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 19:26:26.366388 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364497    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 19:26:26.366388 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364500    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 19:26:26.366388 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364503    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 19:26:26.366388 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364506    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 19:26:26.366388 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364508    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 19:26:26.366388 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364511    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 19:26:26.366388 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364514    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 19:26:26.366388 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364516    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 19:26:26.366388 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364519    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 19:26:26.366388 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364521    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 19:26:26.366388 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364524    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 19:26:26.366388 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364526    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 19:26:26.366388 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364529    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 19:26:26.366388 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364531    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 19:26:26.366388 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364533    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 19:26:26.366388 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364536    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 19:26:26.366388 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364539    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 19:26:26.366962 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364542    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 19:26:26.366962 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364544    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 19:26:26.366962 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364547    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 19:26:26.366962 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364550    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 19:26:26.366962 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364552    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 19:26:26.366962 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364554    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 19:26:26.366962 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364557    2558 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 19:26:26.366962 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364560    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 19:26:26.366962 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364562    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 19:26:26.366962 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364565    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 19:26:26.366962 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364567    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 19:26:26.366962 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364569    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 19:26:26.366962 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364573    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 19:26:26.366962 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364577    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 19:26:26.366962 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364580    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 19:26:26.366962 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364583    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 19:26:26.366962 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364586    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 19:26:26.366962 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364588    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 19:26:26.366962 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364591    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 19:26:26.367461 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364593    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 19:26:26.367461 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364596    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 19:26:26.367461 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364598    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 19:26:26.367461 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364602    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 19:26:26.367461 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364604    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 19:26:26.367461 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364607    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 19:26:26.367461 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364611    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 19:26:26.367461 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364615    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 19:26:26.367461 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364618    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 19:26:26.367461 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364620    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 19:26:26.367461 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364623    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 19:26:26.367461 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364626    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 19:26:26.367461 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364629    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 19:26:26.367461 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364631    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 19:26:26.367461 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364634    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 19:26:26.367461 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364636    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 19:26:26.367461 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364639    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 19:26:26.367461 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364641    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 19:26:26.367461 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364644    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 19:26:26.367951 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364646    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 19:26:26.367951 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364649    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 19:26:26.367951 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364651    2558 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 19:26:26.367951 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364653    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 19:26:26.367951 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364656    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 19:26:26.367951 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364658    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 19:26:26.367951 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364661    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 19:26:26.367951 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364663    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 19:26:26.367951 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364666    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 19:26:26.367951 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364668    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 19:26:26.367951 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364671    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 19:26:26.367951 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364674    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 19:26:26.367951 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364676    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 19:26:26.367951 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364678    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 19:26:26.367951 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364681    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 19:26:26.367951 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364684    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 19:26:26.367951 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364686    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 19:26:26.367951 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364688    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 19:26:26.367951 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364692    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 19:26:26.367951 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364694    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 19:26:26.368438 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364696    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 19:26:26.368438 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364699    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 19:26:26.368438 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364701    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 19:26:26.368438 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364704    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 19:26:26.368438 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364707    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 19:26:26.368438 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364710    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 19:26:26.368438 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364712    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 19:26:26.368438 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364715    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 19:26:26.368438 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364717    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 19:26:26.368438 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364720    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 19:26:26.368438 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364724    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 19:26:26.368438 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364727    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 19:26:26.368438 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364730    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 19:26:26.368438 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364733    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 19:26:26.368438 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364736    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 19:26:26.368438 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364738    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 19:26:26.368438 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364741    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 19:26:26.368438 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364744    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 19:26:26.368438 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364746    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 19:26:26.368438 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.364749    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 19:26:26.368969 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366201    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 19:26:26.368969 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366217    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 19:26:26.368969 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366222    2558 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 19:26:26.368969 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366225    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 19:26:26.368969 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366228    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 19:26:26.368969 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366231    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 19:26:26.368969 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366234    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 19:26:26.368969 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366237    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 19:26:26.368969 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366240    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 19:26:26.368969 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366242    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 19:26:26.368969 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366245    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 19:26:26.368969 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366248    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 19:26:26.368969 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366250    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 19:26:26.368969 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366253    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 19:26:26.368969 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366255    2558 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 19:26:26.368969 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366258    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 19:26:26.368969 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366260    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 19:26:26.368969 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366263    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 19:26:26.368969 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366266    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 19:26:26.368969 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366269    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 19:26:26.369555 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366271    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 19:26:26.369555 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366274    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 19:26:26.369555 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366276    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 19:26:26.369555 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366279    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 19:26:26.369555 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366282    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 19:26:26.369555 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366285    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 19:26:26.369555 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366287    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 19:26:26.369555 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366290    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 19:26:26.369555 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366292    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 19:26:26.369555 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366294    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 19:26:26.369555 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366297    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 19:26:26.369555 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366299    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 19:26:26.369555 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366305    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 19:26:26.369555 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366309    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 19:26:26.369555 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366311    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 19:26:26.369555 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366314    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 19:26:26.369555 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366317    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 19:26:26.369555 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366319    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 19:26:26.369555 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366322    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 19:26:26.369555 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366325    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 19:26:26.370060 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366327    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 19:26:26.370060 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366330    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 19:26:26.370060 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366333    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 19:26:26.370060 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366335    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 19:26:26.370060 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366338    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 19:26:26.370060 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366340    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 19:26:26.370060 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366343    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 19:26:26.370060 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366345    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 19:26:26.370060 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366348    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 19:26:26.370060 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366350    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 19:26:26.370060 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366352    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 19:26:26.370060 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366355    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 19:26:26.370060 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366357    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 19:26:26.370060 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366360    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 19:26:26.370060 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366363    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 19:26:26.370060 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366365    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 19:26:26.370060 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366367    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 19:26:26.370060 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366370    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 19:26:26.370060 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366373    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 19:26:26.370539 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366376    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 19:26:26.370539 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366378    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 19:26:26.370539 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366380    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 19:26:26.370539 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366383    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 19:26:26.370539 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366385    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 19:26:26.370539 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366388    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 19:26:26.370539 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366391    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 19:26:26.370539 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366394    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 19:26:26.370539 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366400    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 19:26:26.370539 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366403    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 19:26:26.370539 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366405    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 19:26:26.370539 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366410    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 19:26:26.370539 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366414    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 19:26:26.370539 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366417    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 19:26:26.370539 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366420    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 19:26:26.370539 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366423    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 19:26:26.370539 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366425    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 19:26:26.370539 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366429    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 19:26:26.370539 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366432    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 19:26:26.371019 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366435    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 19:26:26.371019 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366437    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 19:26:26.371019 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366440    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 19:26:26.371019 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366444    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 19:26:26.371019 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366447    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 19:26:26.371019 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366450    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 19:26:26.371019 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366453    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 19:26:26.371019 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366456    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 19:26:26.371019 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366459    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 19:26:26.371019 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366462    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 19:26:26.371019 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366467    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 19:26:26.371019 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366470    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 19:26:26.371019 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366472    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 19:26:26.371019 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366475    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 19:26:26.371019 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366477    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 19:26:26.371019 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366480    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 19:26:26.371019 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366482    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 19:26:26.371019 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366484    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 19:26:26.371019 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366487    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 19:26:26.371019 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366491    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 19:26:26.371497 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.366493    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 19:26:26.371497 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366569    2558 flags.go:64] FLAG: --address="0.0.0.0"
Jul 27 19:26:26.371497 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366578    2558 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 27 19:26:26.371497 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366585    2558 flags.go:64] FLAG: --anonymous-auth="true"
Jul 27 19:26:26.371497 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366590    2558 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 27 19:26:26.371497 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366594    2558 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 27 19:26:26.371497 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366597    2558 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 27 19:26:26.371497 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366602    2558 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 27 19:26:26.371497 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366606    2558 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 27 19:26:26.371497 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366609    2558 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 27 19:26:26.371497 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366612    2558 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 27 19:26:26.371497 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366616    2558 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 27 19:26:26.371497 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366619    2558 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 27 19:26:26.371497 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366622    2558 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 27 19:26:26.371497 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366625    2558 flags.go:64] FLAG: --cgroup-root=""
Jul 27 19:26:26.371497 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366628    2558 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 27 19:26:26.371497 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366631    2558 flags.go:64] FLAG: --client-ca-file=""
Jul 27 19:26:26.371497 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366635    2558 flags.go:64] FLAG: --cloud-provider="external"
Jul 27 19:26:26.371497 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366638    2558 flags.go:64] FLAG: --cluster-dns="[]"
Jul 27 19:26:26.371497 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366643    2558 flags.go:64] FLAG: --cluster-domain=""
Jul 27 19:26:26.371497 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366646    2558 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 27 19:26:26.371497 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366649    2558 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 27 19:26:26.371497 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366652    2558 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 27 19:26:26.371497 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366656    2558 flags.go:64] FLAG: --container-log-max-files="5"
Jul 27 19:26:26.372087 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366660    2558 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 27 19:26:26.372087 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366662    2558 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 27 19:26:26.372087 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366666    2558 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 27 19:26:26.372087 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366669    2558 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 27 19:26:26.372087 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366672    2558 flags.go:64] FLAG: --contention-profiling="false"
Jul 27 19:26:26.372087 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366675    2558 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 27 19:26:26.372087 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366678    2558 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 27 19:26:26.372087 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366682    2558 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 27 19:26:26.372087 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366685    2558 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 27 19:26:26.372087 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366690    2558 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 27 19:26:26.372087 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366694    2558 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 27 19:26:26.372087 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366697    2558 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 27 19:26:26.372087 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366700    2558 flags.go:64] FLAG: --enable-load-reader="false"
Jul 27 19:26:26.372087 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366703    2558 flags.go:64] FLAG: --enable-server="true"
Jul 27 19:26:26.372087 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366705    2558 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 27 19:26:26.372087 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366711    2558 flags.go:64] FLAG: --event-burst="100"
Jul 27 19:26:26.372087 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366714    2558 flags.go:64] FLAG: --event-qps="50"
Jul 27 19:26:26.372087 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366717    2558 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 27 19:26:26.372087 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366720    2558 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 27 19:26:26.372087 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366723    2558 flags.go:64] FLAG: --eviction-hard=""
Jul 27 19:26:26.372087 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366730    2558 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 27 19:26:26.372087 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366733    2558 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 27 19:26:26.372087 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366736    2558 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 27 19:26:26.372087 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366739    2558 flags.go:64] FLAG: --eviction-soft=""
Jul 27 19:26:26.372087 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366742    2558 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 27 19:26:26.372679 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366745    2558 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 27 19:26:26.372679 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366749    2558 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 27 19:26:26.372679 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366752    2558 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 27 19:26:26.372679 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366755    2558 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 27 19:26:26.372679 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366758    2558 flags.go:64] FLAG: --fail-swap-on="true"
Jul 27 19:26:26.372679 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366761    2558 flags.go:64] FLAG: --feature-gates=""
Jul 27 19:26:26.372679 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366764    2558 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 27 19:26:26.372679 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366767    2558 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 27 19:26:26.372679 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366770    2558 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 27 19:26:26.372679 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366774    2558 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 27 19:26:26.372679 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366777    2558 flags.go:64] FLAG: --healthz-port="10248"
Jul 27 19:26:26.372679 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366781    2558 flags.go:64] FLAG: --help="false"
Jul 27 19:26:26.372679 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366784    2558 flags.go:64] FLAG: --hostname-override="ip-10-0-129-235.ec2.internal"
Jul 27 19:26:26.372679 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366787    2558 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 27 19:26:26.372679 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366790    2558 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 27 19:26:26.372679 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366793    2558 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 27 19:26:26.372679 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366796    2558 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 27 19:26:26.372679 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366800    2558 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 27 19:26:26.372679 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366803    2558 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 27 19:26:26.372679 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366806    2558 flags.go:64] FLAG: --image-service-endpoint=""
Jul 27 19:26:26.372679 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366809    2558 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 27 19:26:26.372679 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366812    2558 flags.go:64] FLAG: --kube-api-burst="100"
Jul 27 19:26:26.372679 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366814    2558 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 27 19:26:26.373428 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366818    2558 flags.go:64] FLAG: --kube-api-qps="50"
Jul 27 19:26:26.373428 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366821    2558 flags.go:64] FLAG: --kube-reserved=""
Jul 27 19:26:26.373428 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366824    2558 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 27 19:26:26.373428 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366826    2558 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 27 19:26:26.373428 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366829    2558 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 27 19:26:26.373428 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366832    2558 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 27 19:26:26.373428 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366835    2558 flags.go:64] FLAG: --lock-file=""
Jul 27 19:26:26.373428 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366838    2558 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 27 19:26:26.373428 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366840    2558 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 27 19:26:26.373428 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366860    2558 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 27 19:26:26.373428 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366871    2558 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 27 19:26:26.373428 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366875    2558 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 27 19:26:26.373428 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366878    2558 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 27 19:26:26.373428 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366881    2558 flags.go:64] FLAG: --logging-format="text"
Jul 27 19:26:26.373428 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366884    2558 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 27 19:26:26.373428 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366887    2558 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 27 19:26:26.373428 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366890    2558 flags.go:64] FLAG: --manifest-url=""
Jul 27 19:26:26.373428 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366893    2558 flags.go:64] FLAG: --manifest-url-header=""
Jul 27 19:26:26.373428 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366898    2558 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 27 19:26:26.373428 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366901    2558 flags.go:64] FLAG: --max-open-files="1000000"
Jul 27 19:26:26.373428 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366905    2558 flags.go:64] FLAG: --max-pods="110"
Jul 27 19:26:26.373428 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366908    2558 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 27 19:26:26.373428 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366911    2558 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 27 19:26:26.373428 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366914    2558 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 27 19:26:26.373428 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366918    2558 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 27 19:26:26.374079 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366921    2558 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 27 19:26:26.374079 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366924    2558 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 27 19:26:26.374079 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366927    2558 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 27 19:26:26.374079 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366946    2558 flags.go:64] FLAG: --node-status-max-images="50"
Jul 27 19:26:26.374079 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366949    2558 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 27 19:26:26.374079 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366952    2558 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 27 19:26:26.374079 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366956    2558 flags.go:64] FLAG: --pod-cidr=""
Jul 27 19:26:26.374079 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366959    2558 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 27 19:26:26.374079 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366965    2558 flags.go:64] FLAG: --pod-manifest-path=""
Jul 27 19:26:26.374079 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366968    2558 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 27 19:26:26.374079 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366972    2558 flags.go:64] FLAG: --pods-per-core="0"
Jul 27 19:26:26.374079 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366975    2558 flags.go:64] FLAG: --port="10250"
Jul 27 19:26:26.374079 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366978    2558 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 27 19:26:26.374079 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366981    2558 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-09456bf62ecf66413"
Jul 27 19:26:26.374079 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366984    2558 flags.go:64] FLAG: --qos-reserved=""
Jul 27 19:26:26.374079 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366987    2558 flags.go:64] FLAG: --read-only-port="10255"
Jul 27 19:26:26.374079 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366989    2558 flags.go:64] FLAG: --register-node="true"
Jul 27 19:26:26.374079 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.366992    2558 flags.go:64] FLAG: --register-with-taints=""
Jul 27 19:26:26.374079 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.367000    2558 flags.go:64] FLAG: --registry-burst="10"
Jul 27 19:26:26.374079 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.367004    2558 flags.go:64] FLAG: --registry-qps="5"
Jul 27 19:26:26.374079 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.367007    2558 flags.go:64] FLAG: --reserved-cpus=""
Jul 27 19:26:26.374079 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.367010    2558 flags.go:64] FLAG: --reserved-memory=""
Jul 27 19:26:26.374079 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.367014    2558 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 27 19:26:26.374079 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.367017    2558 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 27 19:26:26.374079 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.367020    2558 flags.go:64] FLAG: --rotate-certificates="false"
Jul 27 19:26:26.374704 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.367023    2558 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 27 19:26:26.374704 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.367026    2558 flags.go:64] FLAG: --runonce="false"
Jul 27 19:26:26.374704 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.367028    2558 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 27 19:26:26.374704 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.367031    2558 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 27 19:26:26.374704 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.367034    2558 flags.go:64] FLAG: --seccomp-default="false"
Jul 27 19:26:26.374704 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.367043    2558 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 27 19:26:26.374704 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.367046    2558 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 27 19:26:26.374704 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.367049    2558 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 27 19:26:26.374704 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.367052    2558 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 27 19:26:26.374704 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.367055    2558 flags.go:64] FLAG: --storage-driver-password="root"
Jul 27 19:26:26.374704 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.367058    2558 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 27 19:26:26.374704 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.367061    2558 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 27 19:26:26.374704 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.367064    2558 flags.go:64] FLAG: --storage-driver-user="root"
Jul 27 19:26:26.374704 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.367067    2558 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 27 19:26:26.374704 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.367069    2558 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 27 19:26:26.374704 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.367072    2558 flags.go:64] FLAG: --system-cgroups=""
Jul 27 19:26:26.374704 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.367075    2558 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 27 19:26:26.374704 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.367081    2558 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 27 19:26:26.374704 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.367084    2558 flags.go:64] FLAG: --tls-cert-file=""
Jul 27 19:26:26.374704 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.367087    2558 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 27 19:26:26.374704 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.367091    2558 flags.go:64] FLAG: --tls-min-version=""
Jul 27 19:26:26.374704 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.367094    2558 flags.go:64] FLAG: --tls-private-key-file=""
Jul 27 19:26:26.374704 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.367097    2558 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 27 19:26:26.374704 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.367100    2558 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 27 19:26:26.374704 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.367103    2558 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 27 19:26:26.375358 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.367106    2558 flags.go:64] FLAG: --v="2"
Jul 27 19:26:26.375358 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.367110    2558 flags.go:64] FLAG: --version="false"
Jul 27 19:26:26.375358 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.367115    2558 flags.go:64] FLAG: --vmodule=""
Jul 27 19:26:26.375358 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.367118    2558 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 27 19:26:26.375358 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.367122    2558 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 27 19:26:26.375358 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367254    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 19:26:26.375358 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367257    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 19:26:26.375358 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367261    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 19:26:26.375358 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367264    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 19:26:26.375358 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367269    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 19:26:26.375358 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367273    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 19:26:26.375358 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367275    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 19:26:26.375358 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367281    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 19:26:26.375358 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367285    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 19:26:26.375358 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367287    2558 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 19:26:26.375358 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367290    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 19:26:26.375358 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367292    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 19:26:26.375358 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367295    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 19:26:26.375358 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367298    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 19:26:26.375358 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367301    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 19:26:26.375877 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367303    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 19:26:26.375877 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367306    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 19:26:26.375877 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367308    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 19:26:26.375877 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367311    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 19:26:26.375877 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367314    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 19:26:26.375877 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367316    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 19:26:26.375877 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367319    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 19:26:26.375877 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367321    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 19:26:26.375877 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367324    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 19:26:26.375877 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367327    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 19:26:26.375877 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367329    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 19:26:26.375877 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367332    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 19:26:26.375877 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367334    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 19:26:26.375877 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367337    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 19:26:26.375877 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367340    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 19:26:26.375877 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367343    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 19:26:26.375877 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367346    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 19:26:26.375877 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367349    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 19:26:26.375877 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367351    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 19:26:26.375877 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367354    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 19:26:26.376372 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367357    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 19:26:26.376372 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367359    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 19:26:26.376372 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367362    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 19:26:26.376372 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367364    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 19:26:26.376372 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367370    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 19:26:26.376372 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367372    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 19:26:26.376372 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367375    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 19:26:26.376372 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367378    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 19:26:26.376372 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367381    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 19:26:26.376372 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367383    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 19:26:26.376372 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367387    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 19:26:26.376372 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367389    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 19:26:26.376372 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367392    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 19:26:26.376372 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367395    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 19:26:26.376372 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367397    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 19:26:26.376372 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367400    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 19:26:26.376372 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367403    2558 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 19:26:26.376372 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367405    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 19:26:26.376372 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367408    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 19:26:26.376372 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367410    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 19:26:26.377051 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367413    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 19:26:26.377051 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367415    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 19:26:26.377051 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367418    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 19:26:26.377051 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367421    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 19:26:26.377051 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367423    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 19:26:26.377051 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367425    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 19:26:26.377051 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367428    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 19:26:26.377051 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367431    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 19:26:26.377051 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367434    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 19:26:26.377051 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367437    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 19:26:26.377051 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367439    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 19:26:26.377051 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367442    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 19:26:26.377051 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367445    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 19:26:26.377051 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367447    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 19:26:26.377051 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367450    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 19:26:26.377051 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367452    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 19:26:26.377051 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367458    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 19:26:26.377051 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367461    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 19:26:26.377051 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367464    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 19:26:26.377051 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367467    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 19:26:26.377554 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367469    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 19:26:26.377554 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367472    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 19:26:26.377554 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367475    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 19:26:26.377554 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367478    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 19:26:26.377554 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367480    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 19:26:26.377554 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367483    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 19:26:26.377554 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367485    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 19:26:26.377554 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367488    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 19:26:26.377554 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367491    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 19:26:26.377554 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367493    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 19:26:26.377554 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367496    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 19:26:26.377554 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367498    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 19:26:26.377554 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367501    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 19:26:26.377554 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367504    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 19:26:26.377554 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367507    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 19:26:26.377554 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367511    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 19:26:26.377554 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367514    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 19:26:26.377554 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367517    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 19:26:26.377554 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367520    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 19:26:26.377554 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367522    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 19:26:26.378087 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367525    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 19:26:26.378087 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367528    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 19:26:26.378087 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367532    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 19:26:26.378087 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.367535    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 19:26:26.378087 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.367544    2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 27 19:26:26.378087 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.374894    2558 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 27 19:26:26.378087 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.374917    2558 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 27 19:26:26.378087 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.374950    2558 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 27 19:26:26.378087 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.374957    2558 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 27 19:26:26.378087 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375030    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 19:26:26.378087 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375036    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 19:26:26.378087 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375039    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 19:26:26.378087 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375042    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 19:26:26.378087 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375045    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 19:26:26.378438 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375048    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 19:26:26.378438 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375051    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 19:26:26.378438 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375054    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 19:26:26.378438 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375057    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 19:26:26.378438 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375060    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 19:26:26.378438 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375062    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 19:26:26.378438 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375065    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 19:26:26.378438 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375068    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 19:26:26.378438 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375070    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 19:26:26.378438 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375075    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 19:26:26.378438 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375080    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 19:26:26.378438 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375083    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 19:26:26.378438 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375086    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 19:26:26.378438 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375089    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 19:26:26.378438 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375092    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 19:26:26.378438 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375095    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 19:26:26.378438 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375097    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 19:26:26.378438 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375100    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 19:26:26.378438 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375103    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 19:26:26.378959 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375106    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 19:26:26.378959 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375109    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 19:26:26.378959 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375111    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 19:26:26.378959 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375114    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 19:26:26.378959 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375116    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 19:26:26.378959 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375121    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 19:26:26.378959 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375124    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 19:26:26.378959 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375126    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 19:26:26.378959 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375129    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 19:26:26.378959 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375131    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 19:26:26.378959 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375134    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 19:26:26.378959 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375137    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 19:26:26.378959 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375139    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 19:26:26.378959 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375141    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 19:26:26.378959 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375144    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 19:26:26.378959 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375147    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 19:26:26.378959 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375150    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 19:26:26.378959 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375153    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 19:26:26.378959 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375155    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 19:26:26.378959 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375158    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 19:26:26.379457 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375161    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 19:26:26.379457 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375164    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 19:26:26.379457 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375166    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 19:26:26.379457 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375169    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 19:26:26.379457 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375171    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 19:26:26.379457 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375174    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 19:26:26.379457 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375176    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 19:26:26.379457 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375179    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 19:26:26.379457 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375181    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 19:26:26.379457 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375184    2558 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 19:26:26.379457 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375186    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 19:26:26.379457 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375189    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 19:26:26.379457 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375191    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 19:26:26.379457 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375194    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 19:26:26.379457 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375196    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 19:26:26.379457 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375198    2558 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 19:26:26.379457 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375201    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 19:26:26.379457 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375204    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 19:26:26.379457 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375208    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 19:26:26.379457 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375210    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 19:26:26.379980 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375213    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 19:26:26.379980 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375216    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 19:26:26.379980 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375218    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 19:26:26.379980 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375221    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 19:26:26.379980 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375224    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 19:26:26.379980 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375226    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 19:26:26.379980 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375229    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 19:26:26.379980 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375231    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 19:26:26.379980 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375234    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 19:26:26.379980 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375237    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 19:26:26.379980 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375241    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 19:26:26.379980 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375244    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 19:26:26.379980 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375247    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 19:26:26.379980 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375250    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 19:26:26.379980 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375252    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 19:26:26.379980 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375255    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 19:26:26.379980 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375257    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 19:26:26.379980 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375260    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 19:26:26.379980 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375262    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 19:26:26.380500 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375265    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 19:26:26.380500 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375268    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 19:26:26.380500 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375272    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 19:26:26.380500 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375275    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 19:26:26.380500 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375278    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 19:26:26.380500 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375281    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 19:26:26.380500 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375284    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 19:26:26.380500 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375286    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 19:26:26.380500 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375289    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 19:26:26.380500 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375292    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 19:26:26.380500 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375296    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 19:26:26.380500 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375298    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 19:26:26.380500 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375302    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 19:26:26.380500 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375304    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 19:26:26.380500 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375307    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 19:26:26.380500 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375309    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 19:26:26.381000 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.375315    2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 27 19:26:26.381000 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375443    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 19:26:26.381000 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375448    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 19:26:26.381000 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375452    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 19:26:26.381000 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375455    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 19:26:26.381000 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375457    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 19:26:26.381000 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375460    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 19:26:26.381000 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375463    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 19:26:26.381000 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375466    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 19:26:26.381000 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375468    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 19:26:26.381000 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375471    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 19:26:26.381000 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375473    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 19:26:26.381000 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375476    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 19:26:26.381000 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375478    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 19:26:26.381000 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375481    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 19:26:26.381373 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375483    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 19:26:26.381373 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375486    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 19:26:26.381373 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375488    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 19:26:26.381373 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375490    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 19:26:26.381373 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375493    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 19:26:26.381373 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375495    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 19:26:26.381373 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375498    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 19:26:26.381373 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375501    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 19:26:26.381373 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375503    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 19:26:26.381373 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375506    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 19:26:26.381373 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375509    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 19:26:26.381373 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375511    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 19:26:26.381373 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375514    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 19:26:26.381373 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375517    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 19:26:26.381373 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375520    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 19:26:26.381373 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375541    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 19:26:26.381373 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375546    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 19:26:26.381373 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375549    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 19:26:26.381373 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375552    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 19:26:26.381861 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375555    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 19:26:26.381861 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375558    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 19:26:26.381861 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375560    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 19:26:26.381861 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375563    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 19:26:26.381861 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375566    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 19:26:26.381861 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375568    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 19:26:26.381861 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375571    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 19:26:26.381861 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375574    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 19:26:26.381861 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375577    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 19:26:26.381861 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375579    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 19:26:26.381861 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375582    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 19:26:26.381861 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375585    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 19:26:26.381861 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375587    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 19:26:26.381861 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375591    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 19:26:26.381861 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375595    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 19:26:26.381861 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375598    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 19:26:26.381861 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375601    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 19:26:26.381861 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375604    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 19:26:26.381861 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375606    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 19:26:26.382369 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375609    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 19:26:26.382369 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375611    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 19:26:26.382369 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375614    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 19:26:26.382369 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375616    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 19:26:26.382369 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375620    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 19:26:26.382369 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375622    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 19:26:26.382369 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375625    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 19:26:26.382369 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375628    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 19:26:26.382369 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375631    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 19:26:26.382369 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375634    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 19:26:26.382369 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375637    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 19:26:26.382369 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375640    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 19:26:26.382369 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375642    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 19:26:26.382369 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375645    2558 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 19:26:26.382369 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375647    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 19:26:26.382369 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375650    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 19:26:26.382369 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375652    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 19:26:26.382369 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375655    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 19:26:26.382369 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375657    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 19:26:26.382369 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375659    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 19:26:26.383047 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375662    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 19:26:26.383047 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375665    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 19:26:26.383047 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375667    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 19:26:26.383047 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375670    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 19:26:26.383047 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375672    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 19:26:26.383047 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375676    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 19:26:26.383047 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375679    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 19:26:26.383047 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375682    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 19:26:26.383047 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375685    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 19:26:26.383047 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375688    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 19:26:26.383047 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375690    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 19:26:26.383047 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375693    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 19:26:26.383047 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375696    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 19:26:26.383047 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375699    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 19:26:26.383047 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375702    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 19:26:26.383047 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375704    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 19:26:26.383047 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375707    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 19:26:26.383047 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375710    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 19:26:26.383047 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375713    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 19:26:26.383047 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375716    2558 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 19:26:26.383572 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375718    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 19:26:26.383572 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375721    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 19:26:26.383572 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375724    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 19:26:26.383572 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375727    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 19:26:26.383572 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375731    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 19:26:26.383572 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375734    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 19:26:26.383572 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:26.375737    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 19:26:26.383572 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.375742    2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 27 19:26:26.383572 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.375918    2558 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 27 19:26:26.383572 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.377991    2558 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 27 19:26:26.383572 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.379329    2558 server.go:1019] "Starting client certificate rotation"
Jul 27 19:26:26.383572 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.379438    2558 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 27 19:26:26.383572 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.379473    2558 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 27 19:26:26.383969 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.381975    2558 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 27 19:26:26.383969 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.383185    2558 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 27 19:26:26.396043 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.396012    2558 log.go:25] "Validated CRI v1 runtime API"
Jul 27 19:26:26.398200 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.398181    2558 log.go:25] "Validated CRI v1 image API"
Jul 27 19:26:26.399119 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.399104    2558 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 27 19:26:26.401778 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.401756    2558 fs.go:135] Filesystem UUIDs: map[33aef376-eb9b-462c-a397-b0e2989c691c:/dev/nvme0n1p4 76391d54-8a99-47ec-a4e8-15dac15ea362:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Jul 27 19:26:26.401859 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.401779    2558 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 27 19:26:26.407821 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.407781    2558 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 27 19:26:26.409387 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.409262    2558 manager.go:216] Machine: {Timestamp:2026-07-27 19:26:26.407906967 +0000 UTC m=+0.313663377 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3103652 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec22477a8dd5a87fa65c5e0ccb3a894b SystemUUID:ec22477a-8dd5-a87f-a65c-5e0ccb3a894b BootID:ddcf4278-91ee-4755-90dc-38f59d187f06 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:d0:88:f9:e8:1f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:d0:88:f9:e8:1f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:9e:04:35:16:5a:c2 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 27 19:26:26.409387 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.409378    2558 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 27 19:26:26.409588 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.409519    2558 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 27 19:26:26.412487 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.412442    2558 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 27 19:26:26.412681 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.412489    2558 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-235.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 27 19:26:26.413331 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.413317    2558 topology_manager.go:138] "Creating topology manager with none policy"
Jul 27 19:26:26.413402 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.413340    2558 container_manager_linux.go:309] "Creating device plugin manager"
Jul 27 19:26:26.413402 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.413352    2558 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 27 19:26:26.413948 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.413935    2558 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 27 19:26:26.415043 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.415027    2558 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 27 19:26:26.416952 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.416934    2558 state_mem.go:36] "Initialized new in-memory state store"
Jul 27 19:26:26.417110 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.417098    2558 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 27 19:26:26.417183 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.417142    2558 kubelet.go:486] "Attempting to sync node with API server"
Jul 27 19:26:26.417183 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.417158    2558 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 27 19:26:26.417183 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.417179    2558 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 27 19:26:26.417296 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.417193    2558 kubelet.go:398] "Adding apiserver pod source"
Jul 27 19:26:26.417296 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.417208    2558 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 27 19:26:26.417645 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.417617    2558 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 27 19:26:26.417645 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.417639    2558 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 27 19:26:26.419669 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.419648    2558 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 27 19:26:26.421940 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.421923    2558 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 27 19:26:26.422027 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.421962    2558 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 27 19:26:26.423428 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.423411    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 27 19:26:26.423501 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.423433    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 27 19:26:26.423501 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.423442    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 27 19:26:26.423501 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.423451    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 27 19:26:26.423501 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.423464    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 27 19:26:26.423501 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.423472    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 27 19:26:26.423501 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.423484    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 27 19:26:26.423501 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.423493    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 27 19:26:26.423727 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.423516    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 27 19:26:26.423727 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.423525    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 27 19:26:26.423727 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.423539    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 27 19:26:26.423727 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.423550    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 27 19:26:26.424198 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.424174    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 27 19:26:26.424198 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.424194    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 27 19:26:26.426445 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:26.426412    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 27 19:26:26.426445 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:26.426428    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-235.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 27 19:26:26.426579 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.426470    2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-235.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:26.427863 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.427836    2558 server.go:1268] "Started kubelet"
Jul 27 19:26:26.427988 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.427940    2558 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 27 19:26:26.428077 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.427974    2558 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 27 19:26:26.428077 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.428052    2558 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 27 19:26:26.428907 ip-10-0-129-235 systemd[1]: Started Kubernetes Kubelet.
Jul 27 19:26:26.429923 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.429798    2558 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 27 19:26:26.430770 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.430753    2558 server.go:310] "Adding debug handlers to kubelet server"
Jul 27 19:26:26.434052 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:26.434027    2558 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 27 19:26:26.437524 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.437507    2558 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 27 19:26:26.437611 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.437581    2558 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 27 19:26:26.437683 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:26.437665    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:26:26.438300 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.438282    2558 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 27 19:26:26.438419 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.438286    2558 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 27 19:26:26.438419 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.438349    2558 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 27 19:26:26.438520 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.438432    2558 reconstruct.go:97] "Volume reconstruction finished"
Jul 27 19:26:26.438520 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.438442    2558 reconciler.go:29] "Reconciler: start to sync state"
Jul 27 19:26:26.438838 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.438822    2558 factory.go:55] Registering systemd factory
Jul 27 19:26:26.438995 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.438980    2558 factory.go:223] Registration of the systemd container factory successfully
Jul 27 19:26:26.439285 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.439268    2558 factory.go:153] Registering CRI-O factory
Jul 27 19:26:26.439366 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.439290    2558 factory.go:223] Registration of the crio container factory successfully
Jul 27 19:26:26.439411 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.439370    2558 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 27 19:26:26.439411 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.439399    2558 factory.go:103] Registering Raw factory
Jul 27 19:26:26.439411 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.439410    2558 manager.go:1196] Started watching for new ooms in manager
Jul 27 19:26:26.439953 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.439941    2558 manager.go:349] Starting recovery of all containers
Jul 27 19:26:26.441920 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:26.439300    2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-235.ec2.internal.18c63c1a6906046e  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-235.ec2.internal,UID:ip-10-0-129-235.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-235.ec2.internal,},FirstTimestamp:2026-07-27 19:26:26.427798638 +0000 UTC m=+0.333555044,LastTimestamp:2026-07-27 19:26:26.427798638 +0000 UTC m=+0.333555044,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-235.ec2.internal,}"
Jul 27 19:26:26.450432 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:26.450403    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 27 19:26:26.450693 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:26.450648    2558 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-235.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 27 19:26:26.454389 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.454248    2558 manager.go:354] Recovery completed
Jul 27 19:26:26.458971 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.458955    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:26:26.460992 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.460959    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:26:26.461124 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.461002    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:26:26.461124 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.461012    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:26:26.461704 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.461685    2558 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 27 19:26:26.461704 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.461700    2558 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 27 19:26:26.461868 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.461717    2558 state_mem.go:36] "Initialized new in-memory state store"
Jul 27 19:26:26.461868 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.461720    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:26:26.462523 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.462508    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:26:26.462586 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.462560    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:26:26.462586 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.462575    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:26:26.463862 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.463832    2558 policy_none.go:49] "None policy: Start"
Jul 27 19:26:26.463917 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.463866    2558 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 27 19:26:26.463917 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.463877    2558 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 27 19:26:26.465053 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.465039    2558 policy_none.go:47] "Start"
Jul 27 19:26:26.470487 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:26.470389    2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-235.ec2.internal.18c63c1a6b006ebf  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-235.ec2.internal,UID:ip-10-0-129-235.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-235.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-235.ec2.internal,},FirstTimestamp:2026-07-27 19:26:26.460987071 +0000 UTC m=+0.366743475,LastTimestamp:2026-07-27 19:26:26.460987071 +0000 UTC m=+0.366743475,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-235.ec2.internal,}"
Jul 27 19:26:26.479743 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:26.479659    2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-235.ec2.internal.18c63c1a6b00bbd0  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-235.ec2.internal,UID:ip-10-0-129-235.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-129-235.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-129-235.ec2.internal,},FirstTimestamp:2026-07-27 19:26:26.4610068 +0000 UTC m=+0.366763205,LastTimestamp:2026-07-27 19:26:26.4610068 +0000 UTC m=+0.366763205,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-235.ec2.internal,}"
Jul 27 19:26:26.489479 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:26.489319    2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-235.ec2.internal.18c63c1a6b00e095  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-235.ec2.internal,UID:ip-10-0-129-235.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-129-235.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-129-235.ec2.internal,},FirstTimestamp:2026-07-27 19:26:26.461016213 +0000 UTC m=+0.366772618,LastTimestamp:2026-07-27 19:26:26.461016213 +0000 UTC m=+0.366772618,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-235.ec2.internal,}"
Jul 27 19:26:26.500622 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.500592    2558 manager.go:337] "Starting Device Plugin manager"
Jul 27 19:26:26.500781 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:26.500654    2558 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 27 19:26:26.500781 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.500668    2558 server.go:87] "Starting device plugin registration server"
Jul 27 19:26:26.501024 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.501004    2558 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 27 19:26:26.501136 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.501038    2558 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 27 19:26:26.501187 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.501162    2558 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 27 19:26:26.501337 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.501265    2558 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 27 19:26:26.501337 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.501279    2558 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 27 19:26:26.501534 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:26.501269    2558 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-129-235.ec2.internal.18c63c1a6b006ebf\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-235.ec2.internal.18c63c1a6b006ebf  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-235.ec2.internal,UID:ip-10-0-129-235.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-235.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-235.ec2.internal,},FirstTimestamp:2026-07-27 19:26:26.460987071 +0000 UTC m=+0.366743475,LastTimestamp:2026-07-27 19:26:26.462522019 +0000 UTC m=+0.368278425,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-235.ec2.internal,}"
Jul 27 19:26:26.503178 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:26.502275    2558 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 27 19:26:26.503178 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:26.502330    2558 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:26:26.511060 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:26.510972    2558 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-129-235.ec2.internal.18c63c1a6b00bbd0\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-235.ec2.internal.18c63c1a6b00bbd0  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-235.ec2.internal,UID:ip-10-0-129-235.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-129-235.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-129-235.ec2.internal,},FirstTimestamp:2026-07-27 19:26:26.4610068 +0000 UTC m=+0.366763205,LastTimestamp:2026-07-27 19:26:26.462567575 +0000 UTC m=+0.368323980,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-235.ec2.internal,}"
Jul 27 19:26:26.519101 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:26.519018    2558 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-129-235.ec2.internal.18c63c1a6b00e095\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-235.ec2.internal.18c63c1a6b00e095  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-235.ec2.internal,UID:ip-10-0-129-235.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-129-235.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-129-235.ec2.internal,},FirstTimestamp:2026-07-27 19:26:26.461016213 +0000 UTC m=+0.366772618,LastTimestamp:2026-07-27 19:26:26.462578951 +0000 UTC m=+0.368335356,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-235.ec2.internal,}"
Jul 27 19:26:26.527235 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:26.527149    2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-235.ec2.internal.18c63c1a6d898d52  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-235.ec2.internal,UID:ip-10-0-129-235.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated Node Allocatable limit across pods,Source:EventSource{Component:kubelet,Host:ip-10-0-129-235.ec2.internal,},FirstTimestamp:2026-07-27 19:26:26.503527762 +0000 UTC m=+0.409284154,LastTimestamp:2026-07-27 19:26:26.503527762 +0000 UTC m=+0.409284154,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-235.ec2.internal,}"
Jul 27 19:26:26.602198 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.602154    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:26:26.603275 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.603246    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:26:26.603356 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.603287    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:26:26.603356 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.603302    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:26:26.603356 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.603335    2558 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-235.ec2.internal"
Jul 27 19:26:26.611644 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:26.611486    2558 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-129-235.ec2.internal.18c63c1a6b006ebf\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-235.ec2.internal.18c63c1a6b006ebf  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-235.ec2.internal,UID:ip-10-0-129-235.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-235.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-235.ec2.internal,},FirstTimestamp:2026-07-27 19:26:26.460987071 +0000 UTC m=+0.366743475,LastTimestamp:2026-07-27 19:26:26.603267654 +0000 UTC m=+0.509024061,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-235.ec2.internal,}"
Jul 27 19:26:26.619652 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:26.619612    2558 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-129-235.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-129-235.ec2.internal"
Jul 27 19:26:26.619794 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:26.619599    2558 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-129-235.ec2.internal.18c63c1a6b00bbd0\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-235.ec2.internal.18c63c1a6b00bbd0  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-235.ec2.internal,UID:ip-10-0-129-235.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-129-235.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-129-235.ec2.internal,},FirstTimestamp:2026-07-27 19:26:26.4610068 +0000 UTC m=+0.366763205,LastTimestamp:2026-07-27 19:26:26.603294317 +0000 UTC m=+0.509050721,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-235.ec2.internal,}"
Jul 27 19:26:26.631195 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:26.631102    2558 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-129-235.ec2.internal.18c63c1a6b00e095\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-235.ec2.internal.18c63c1a6b00e095  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-235.ec2.internal,UID:ip-10-0-129-235.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-129-235.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-129-235.ec2.internal,},FirstTimestamp:2026-07-27 19:26:26.461016213 +0000 UTC m=+0.366772618,LastTimestamp:2026-07-27 19:26:26.603307061 +0000 UTC m=+0.509063470,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-235.ec2.internal,}"
Jul 27 19:26:26.632892 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.632866    2558 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 27 19:26:26.634351 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.634319    2558 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 27 19:26:26.634486 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.634362    2558 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 27 19:26:26.634486 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.634391    2558 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 27 19:26:26.634486 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:26.634438    2558 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 27 19:26:26.643822 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:26.643790    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass"
Jul 27 19:26:26.659507 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:26.659474    2558 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-235.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="400ms"
Jul 27 19:26:26.734782 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.734737    2558 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-235.ec2.internal"]
Jul 27 19:26:26.734920 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.734829    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:26:26.735890 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.735871    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:26:26.735987 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.735902    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:26:26.735987 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.735913    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:26:26.737431 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.737417    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:26:26.737539 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.737523    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal"
Jul 27 19:26:26.737574 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.737556    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:26:26.738219 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.738197    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:26:26.738320 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.738197    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:26:26.738320 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.738261    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:26:26.738320 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.738274    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:26:26.738320 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.738234    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:26:26.738320 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.738301    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:26:26.739560 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.739544    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-235.ec2.internal"
Jul 27 19:26:26.739650 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.739571    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:26:26.739703 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.739687    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3c4ec7bc2375d832f57ca8b11d2ae551-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal\" (UID: \"3c4ec7bc2375d832f57ca8b11d2ae551\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal"
Jul 27 19:26:26.739760 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.739737    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/eaacc6d8a8d0349d7d1ac6d641f2511f-config\") pod \"kube-apiserver-proxy-ip-10-0-129-235.ec2.internal\" (UID: \"eaacc6d8a8d0349d7d1ac6d641f2511f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-235.ec2.internal"
Jul 27 19:26:26.739798 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.739777    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3c4ec7bc2375d832f57ca8b11d2ae551-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal\" (UID: \"3c4ec7bc2375d832f57ca8b11d2ae551\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal"
Jul 27 19:26:26.740452 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.740434    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:26:26.740519 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.740468    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:26:26.740519 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.740481    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:26:26.745469 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:26.745396    2558 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-129-235.ec2.internal.18c63c1a6b006ebf\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-235.ec2.internal.18c63c1a6b006ebf  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-235.ec2.internal,UID:ip-10-0-129-235.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-235.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-235.ec2.internal,},FirstTimestamp:2026-07-27 19:26:26.460987071 +0000 UTC m=+0.366743475,LastTimestamp:2026-07-27 19:26:26.735888081 +0000 UTC m=+0.641644486,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-235.ec2.internal,}"
Jul 27 19:26:26.753864 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:26.753773    2558 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-129-235.ec2.internal.18c63c1a6b00bbd0\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-235.ec2.internal.18c63c1a6b00bbd0  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-235.ec2.internal,UID:ip-10-0-129-235.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-129-235.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-129-235.ec2.internal,},FirstTimestamp:2026-07-27 19:26:26.4610068 +0000 UTC m=+0.366763205,LastTimestamp:2026-07-27 19:26:26.735907152 +0000 UTC m=+0.641663556,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-235.ec2.internal,}"
Jul 27 19:26:26.762471 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:26.762386    2558 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-129-235.ec2.internal.18c63c1a6b00e095\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-235.ec2.internal.18c63c1a6b00e095  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-235.ec2.internal,UID:ip-10-0-129-235.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-129-235.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-129-235.ec2.internal,},FirstTimestamp:2026-07-27 19:26:26.461016213 +0000 UTC m=+0.366772618,LastTimestamp:2026-07-27 19:26:26.735916369 +0000 UTC m=+0.641672775,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-235.ec2.internal,}"
Jul 27 19:26:26.770559 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:26.770475    2558 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-129-235.ec2.internal.18c63c1a6b006ebf\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-235.ec2.internal.18c63c1a6b006ebf  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-235.ec2.internal,UID:ip-10-0-129-235.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-235.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-235.ec2.internal,},FirstTimestamp:2026-07-27 19:26:26.460987071 +0000 UTC m=+0.366743475,LastTimestamp:2026-07-27 19:26:26.738222408 +0000 UTC m=+0.643978817,Count:5,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-235.ec2.internal,}"
Jul 27 19:26:26.778035 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:26.777954    2558 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-129-235.ec2.internal.18c63c1a6b006ebf\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-235.ec2.internal.18c63c1a6b006ebf  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-235.ec2.internal,UID:ip-10-0-129-235.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-235.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-235.ec2.internal,},FirstTimestamp:2026-07-27 19:26:26.460987071 +0000 UTC m=+0.366743475,LastTimestamp:2026-07-27 19:26:26.738243429 +0000 UTC m=+0.643999834,Count:6,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-235.ec2.internal,}"
Jul 27 19:26:26.784950 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:26.784917    2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-235.ec2.internal\" not found" node="ip-10-0-129-235.ec2.internal"
Jul 27 19:26:26.786921 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:26.786829    2558 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-129-235.ec2.internal.18c63c1a6b00bbd0\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-235.ec2.internal.18c63c1a6b00bbd0  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-235.ec2.internal,UID:ip-10-0-129-235.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-129-235.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-129-235.ec2.internal,},FirstTimestamp:2026-07-27 19:26:26.4610068 +0000 UTC m=+0.366763205,LastTimestamp:2026-07-27 19:26:26.73826825 +0000 UTC m=+0.644024654,Count:5,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-235.ec2.internal,}"
Jul 27 19:26:26.789714 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:26.789691    2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-235.ec2.internal\" not found" node="ip-10-0-129-235.ec2.internal"
Jul 27 19:26:26.796525 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:26.796448    2558 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-129-235.ec2.internal.18c63c1a6b00e095\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-235.ec2.internal.18c63c1a6b00e095  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-235.ec2.internal,UID:ip-10-0-129-235.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-129-235.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-129-235.ec2.internal,},FirstTimestamp:2026-07-27 19:26:26.461016213 +0000 UTC m=+0.366772618,LastTimestamp:2026-07-27 19:26:26.738278018 +0000 UTC m=+0.644034422,Count:5,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-235.ec2.internal,}"
Jul 27 19:26:26.805644 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:26.805562    2558 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-129-235.ec2.internal.18c63c1a6b00bbd0\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-235.ec2.internal.18c63c1a6b00bbd0  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-235.ec2.internal,UID:ip-10-0-129-235.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-129-235.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-129-235.ec2.internal,},FirstTimestamp:2026-07-27 19:26:26.4610068 +0000 UTC m=+0.366763205,LastTimestamp:2026-07-27 19:26:26.73829211 +0000 UTC m=+0.644048514,Count:6,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-235.ec2.internal,}"
Jul 27 19:26:26.814168 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:26.814077    2558 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-129-235.ec2.internal.18c63c1a6b00e095\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-235.ec2.internal.18c63c1a6b00e095  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-235.ec2.internal,UID:ip-10-0-129-235.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-129-235.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-129-235.ec2.internal,},FirstTimestamp:2026-07-27 19:26:26.461016213 +0000 UTC m=+0.366772618,LastTimestamp:2026-07-27 19:26:26.738308074 +0000 UTC m=+0.644064492,Count:6,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-235.ec2.internal,}"
Jul 27 19:26:26.820694 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.820669    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:26:26.821635 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.821614    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:26:26.821731 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.821653    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:26:26.821731 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.821669    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:26:26.821731 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.821705    2558 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-235.ec2.internal"
Jul 27 19:26:26.822571 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:26.822494    2558 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-129-235.ec2.internal.18c63c1a6b006ebf\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-235.ec2.internal.18c63c1a6b006ebf  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-235.ec2.internal,UID:ip-10-0-129-235.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-235.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-235.ec2.internal,},FirstTimestamp:2026-07-27 19:26:26.460987071 +0000 UTC m=+0.366743475,LastTimestamp:2026-07-27 19:26:26.740452845 +0000 UTC m=+0.646209250,Count:7,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-235.ec2.internal,}"
Jul 27 19:26:26.831061 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:26.830964    2558 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-129-235.ec2.internal.18c63c1a6b00bbd0\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-235.ec2.internal.18c63c1a6b00bbd0  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-235.ec2.internal,UID:ip-10-0-129-235.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-129-235.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-129-235.ec2.internal,},FirstTimestamp:2026-07-27 19:26:26.4610068 +0000 UTC m=+0.366763205,LastTimestamp:2026-07-27 19:26:26.74047545 +0000 UTC m=+0.646231855,Count:7,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-235.ec2.internal,}"
Jul 27 19:26:26.839923 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.839897    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3c4ec7bc2375d832f57ca8b11d2ae551-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal\" (UID: \"3c4ec7bc2375d832f57ca8b11d2ae551\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal"
Jul 27 19:26:26.840101 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.839930    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3c4ec7bc2375d832f57ca8b11d2ae551-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal\" (UID: \"3c4ec7bc2375d832f57ca8b11d2ae551\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal"
Jul 27 19:26:26.840101 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:26.839945    2558 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-129-235.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-129-235.ec2.internal"
Jul 27 19:26:26.840101 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:26.839831    2558 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-129-235.ec2.internal.18c63c1a6b00e095\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-235.ec2.internal.18c63c1a6b00e095  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-235.ec2.internal,UID:ip-10-0-129-235.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-129-235.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-129-235.ec2.internal,},FirstTimestamp:2026-07-27 19:26:26.461016213 +0000 UTC m=+0.366772618,LastTimestamp:2026-07-27 19:26:26.740485073 +0000 UTC m=+0.646241477,Count:7,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-235.ec2.internal,}"
Jul 27 19:26:26.840101 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.840001    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3c4ec7bc2375d832f57ca8b11d2ae551-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal\" (UID: \"3c4ec7bc2375d832f57ca8b11d2ae551\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal"
Jul 27 19:26:26.840101 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.840005    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3c4ec7bc2375d832f57ca8b11d2ae551-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal\" (UID: \"3c4ec7bc2375d832f57ca8b11d2ae551\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal"
Jul 27 19:26:26.840101 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.840005    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/eaacc6d8a8d0349d7d1ac6d641f2511f-config\") pod \"kube-apiserver-proxy-ip-10-0-129-235.ec2.internal\" (UID: \"eaacc6d8a8d0349d7d1ac6d641f2511f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-235.ec2.internal"
Jul 27 19:26:26.840101 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:26.839964    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/eaacc6d8a8d0349d7d1ac6d641f2511f-config\") pod \"kube-apiserver-proxy-ip-10-0-129-235.ec2.internal\" (UID: \"eaacc6d8a8d0349d7d1ac6d641f2511f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-235.ec2.internal"
Jul 27 19:26:26.848993 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:26.848906    2558 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-129-235.ec2.internal.18c63c1a6b006ebf\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-235.ec2.internal.18c63c1a6b006ebf  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-235.ec2.internal,UID:ip-10-0-129-235.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-235.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-235.ec2.internal,},FirstTimestamp:2026-07-27 19:26:26.460987071 +0000 UTC m=+0.366743475,LastTimestamp:2026-07-27 19:26:26.821635722 +0000 UTC m=+0.727392127,Count:8,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-235.ec2.internal,}"
Jul 27 19:26:26.856610 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:26.856521    2558 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-129-235.ec2.internal.18c63c1a6b00bbd0\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-235.ec2.internal.18c63c1a6b00bbd0  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-235.ec2.internal,UID:ip-10-0-129-235.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-129-235.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-129-235.ec2.internal,},FirstTimestamp:2026-07-27 19:26:26.4610068 +0000 UTC m=+0.366763205,LastTimestamp:2026-07-27 19:26:26.821660762 +0000 UTC m=+0.727417169,Count:8,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-235.ec2.internal,}"
Jul 27 19:26:27.069034 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:27.068945    2558 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-235.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="800ms"
Jul 27 19:26:28.121643 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:28.121603    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal"
Jul 27 19:26:28.127508 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:28.127486    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-235.ec2.internal"
Jul 27 19:26:28.274656 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:28.274619    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:26:28.275778 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:28.275751    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:26:28.275890 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:28.275786    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:26:28.275890 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:28.275797    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:26:28.275890 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:28.275824    2558 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-235.ec2.internal"
Jul 27 19:26:28.294548 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:28.294518    2558 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-129-235.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-129-235.ec2.internal"
Jul 27 19:26:28.416152 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:28.416069    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-235.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 27 19:26:28.468838 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:28.468808    2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-235.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:28.644871 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:28.644847    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:26:28.646717 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:28.646696    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:26:28.646830 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:28.646730    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:26:28.646830 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:28.646742    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:26:28.652910 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:28.652870    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c4ec7bc2375d832f57ca8b11d2ae551.slice/crio-6223ac60670451ea8d93063d83f9fe6fa7a3fac77a3fb51a2b25d10225fcd2f7 WatchSource:0}: Error finding container 6223ac60670451ea8d93063d83f9fe6fa7a3fac77a3fb51a2b25d10225fcd2f7: Status 404 returned error can't find the container with id 6223ac60670451ea8d93063d83f9fe6fa7a3fac77a3fb51a2b25d10225fcd2f7
Jul 27 19:26:28.657865 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:28.657847    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 19:26:28.667716 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:28.667590    2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal.18c63c1aedf60fb5  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal,UID:3c4ec7bc2375d832f57ca8b11d2ae551,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Pulling,Message:Pulling image \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\",Source:EventSource{Component:kubelet,Host:ip-10-0-129-235.ec2.internal,},FirstTimestamp:2026-07-27 19:26:28.658122677 +0000 UTC m=+1.530314643,LastTimestamp:2026-07-27 19:26:28.658122677 +0000 UTC m=+1.530314643,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-235.ec2.internal,}"
Jul 27 19:26:28.671960 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:28.671906    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal" event={"ID":"3c4ec7bc2375d832f57ca8b11d2ae551","Type":"ContainerStarted","Data":"6223ac60670451ea8d93063d83f9fe6fa7a3fac77a3fb51a2b25d10225fcd2f7"}
Jul 27 19:26:28.684205 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:26:28.684173    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeaacc6d8a8d0349d7d1ac6d641f2511f.slice/crio-4e8d04a3f827527adb7d6f3fcaabadc76cc3de3e0535a04e43dde193a911d333 WatchSource:0}: Error finding container 4e8d04a3f827527adb7d6f3fcaabadc76cc3de3e0535a04e43dde193a911d333: Status 404 returned error can't find the container with id 4e8d04a3f827527adb7d6f3fcaabadc76cc3de3e0535a04e43dde193a911d333
Jul 27 19:26:28.695427 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:28.695335    2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"kube-system\"" event="&Event{ObjectMeta:{kube-apiserver-proxy-ip-10-0-129-235.ec2.internal.18c63c1aef9d02e6  kube-system    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:kube-system,Name:kube-apiserver-proxy-ip-10-0-129-235.ec2.internal,UID:eaacc6d8a8d0349d7d1ac6d641f2511f,APIVersion:v1,ResourceVersion:,FieldPath:spec.containers{haproxy},},Reason:Pulling,Message:Pulling image \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:924ddd9185690e32284b189b3d0c7f005f10029b713c963227fee4da98829559\",Source:EventSource{Component:kubelet,Host:ip-10-0-129-235.ec2.internal,},FirstTimestamp:2026-07-27 19:26:28.685841126 +0000 UTC m=+1.558033081,LastTimestamp:2026-07-27 19:26:28.685841126 +0000 UTC m=+1.558033081,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-235.ec2.internal,}"
Jul 27 19:26:28.772271 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:28.772223    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 27 19:26:28.907471 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:28.907434    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 27 19:26:28.907641 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:28.907476    2558 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-235.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="1.6s"
Jul 27 19:26:29.095684 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:29.095596    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:26:29.096646 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:29.096624    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:26:29.096740 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:29.096666    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:26:29.096740 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:29.096681    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:26:29.096740 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:29.096717    2558 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-235.ec2.internal"
Jul 27 19:26:29.107023 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:29.106986    2558 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-129-235.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-129-235.ec2.internal"
Jul 27 19:26:29.242663 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:29.242620    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass"
Jul 27 19:26:29.470937 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:29.470852    2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-235.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:29.674448 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:29.674390    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-235.ec2.internal" event={"ID":"eaacc6d8a8d0349d7d1ac6d641f2511f","Type":"ContainerStarted","Data":"4e8d04a3f827527adb7d6f3fcaabadc76cc3de3e0535a04e43dde193a911d333"}
Jul 27 19:26:30.472664 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:30.472627    2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-235.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:30.516705 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:30.516672    2558 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-235.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="3.2s"
Jul 27 19:26:30.576940 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:30.576852    2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal.18c63c1b5fc09eab  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal,UID:3c4ec7bc2375d832f57ca8b11d2ae551,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Pulled,Message:Successfully pulled image \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\" in 1.909s (1.909s including waiting). Image size: 278280461 bytes.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-235.ec2.internal,},FirstTimestamp:2026-07-27 19:26:30.567222955 +0000 UTC m=+3.439414912,LastTimestamp:2026-07-27 19:26:30.567222955 +0000 UTC m=+3.439414912,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-235.ec2.internal,}"
Jul 27 19:26:30.600847 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:30.600730    2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"kube-system\"" event="&Event{ObjectMeta:{kube-apiserver-proxy-ip-10-0-129-235.ec2.internal.18c63c1b6137099e  kube-system    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:kube-system,Name:kube-apiserver-proxy-ip-10-0-129-235.ec2.internal,UID:eaacc6d8a8d0349d7d1ac6d641f2511f,APIVersion:v1,ResourceVersion:,FieldPath:spec.containers{haproxy},},Reason:Pulled,Message:Successfully pulled image \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:924ddd9185690e32284b189b3d0c7f005f10029b713c963227fee4da98829559\" in 1.905s (1.905s including waiting). Image size: 317186379 bytes.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-235.ec2.internal,},FirstTimestamp:2026-07-27 19:26:30.591760798 +0000 UTC m=+3.463952767,LastTimestamp:2026-07-27 19:26:30.591760798 +0000 UTC m=+3.463952767,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-235.ec2.internal,}"
Jul 27 19:26:30.666003 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:30.665723    2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"kube-system\"" event="&Event{ObjectMeta:{kube-apiserver-proxy-ip-10-0-129-235.ec2.internal.18c63c1b64f9797b  kube-system    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:kube-system,Name:kube-apiserver-proxy-ip-10-0-129-235.ec2.internal,UID:eaacc6d8a8d0349d7d1ac6d641f2511f,APIVersion:v1,ResourceVersion:,FieldPath:spec.containers{haproxy},},Reason:Created,Message:Created container: haproxy,Source:EventSource{Component:kubelet,Host:ip-10-0-129-235.ec2.internal,},FirstTimestamp:2026-07-27 19:26:30.654835067 +0000 UTC m=+3.527027037,LastTimestamp:2026-07-27 19:26:30.654835067 +0000 UTC m=+3.527027037,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-235.ec2.internal,}"
Jul 27 19:26:30.674902 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:30.674788    2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"kube-system\"" event="&Event{ObjectMeta:{kube-apiserver-proxy-ip-10-0-129-235.ec2.internal.18c63c1b657d7ae4  kube-system    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:kube-system,Name:kube-apiserver-proxy-ip-10-0-129-235.ec2.internal,UID:eaacc6d8a8d0349d7d1ac6d641f2511f,APIVersion:v1,ResourceVersion:,FieldPath:spec.containers{haproxy},},Reason:Started,Message:Started container haproxy,Source:EventSource{Component:kubelet,Host:ip-10-0-129-235.ec2.internal,},FirstTimestamp:2026-07-27 19:26:30.66348618 +0000 UTC m=+3.535678150,LastTimestamp:2026-07-27 19:26:30.66348618 +0000 UTC m=+3.535678150,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-235.ec2.internal,}"
Jul 27 19:26:30.678010 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:30.677973    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-235.ec2.internal" event={"ID":"eaacc6d8a8d0349d7d1ac6d641f2511f","Type":"ContainerStarted","Data":"bb986965bb0f163b46fb926923541e21349b77ac28f734f7e7c228be347f5aea"}
Jul 27 19:26:30.678171 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:30.678038    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:26:30.678910 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:30.678889    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:26:30.679036 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:30.678928    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:26:30.679036 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:30.678945    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:26:30.679170 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:30.679156    2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-235.ec2.internal\" not found" node="ip-10-0-129-235.ec2.internal"
Jul 27 19:26:30.707487 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:30.707459    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:26:30.708657 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:30.708632    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:26:30.708779 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:30.708677    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:26:30.708779 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:30.708694    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:26:30.708779 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:30.708735    2558 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-235.ec2.internal"
Jul 27 19:26:30.725200 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:30.725170    2558 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-129-235.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-129-235.ec2.internal"
Jul 27 19:26:30.779352 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:30.779256    2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal.18c63c1b6be49919  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal,UID:3c4ec7bc2375d832f57ca8b11d2ae551,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Created,Message:Created container: setup,Source:EventSource{Component:kubelet,Host:ip-10-0-129-235.ec2.internal,},FirstTimestamp:2026-07-27 19:26:30.770907417 +0000 UTC m=+3.643099369,LastTimestamp:2026-07-27 19:26:30.770907417 +0000 UTC m=+3.643099369,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-235.ec2.internal,}"
Jul 27 19:26:30.788396 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:30.788305    2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal.18c63c1b6c6fa04f  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal,UID:3c4ec7bc2375d832f57ca8b11d2ae551,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Started,Message:Started container setup,Source:EventSource{Component:kubelet,Host:ip-10-0-129-235.ec2.internal,},FirstTimestamp:2026-07-27 19:26:30.780018767 +0000 UTC m=+3.652210722,LastTimestamp:2026-07-27 19:26:30.780018767 +0000 UTC m=+3.652210722,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-235.ec2.internal,}"
Jul 27 19:26:30.867521 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:30.867477    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:26:30.868584 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:30.868563    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:26:30.868708 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:30.868598    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:26:30.868708 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:30.868613    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:26:31.091579 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:31.091466    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 27 19:26:31.091579 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:31.091521    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-235.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 27 19:26:31.284568 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:31.284533    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 27 19:26:31.394700 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:31.394623    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass"
Jul 27 19:26:31.473871 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:31.473836    2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-235.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:31.681572 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:31.681493    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:26:31.681733 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:31.681485    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal" event={"ID":"3c4ec7bc2375d832f57ca8b11d2ae551","Type":"ContainerStarted","Data":"d993fdb4462bc3071975a7bda993a437eab3494fe04cb78efb35e2b3c69f9fdf"}
Jul 27 19:26:31.681733 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:31.681499    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:26:31.682339 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:31.682317    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:26:31.682447 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:31.682348    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:26:31.682447 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:31.682360    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:26:31.682447 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:31.682377    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:26:31.682447 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:31.682388    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:26:31.682447 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:31.682363    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:26:31.682704 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:31.682680    2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-235.ec2.internal\" not found" node="ip-10-0-129-235.ec2.internal"
Jul 27 19:26:31.682801 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:31.682681    2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-235.ec2.internal\" not found" node="ip-10-0-129-235.ec2.internal"
Jul 27 19:26:32.469832 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:32.469798    2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-235.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:32.682973 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:32.682941    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:26:32.683862 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:32.683843    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:26:32.683958 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:32.683879    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:26:32.683958 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:32.683892    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:26:32.684168 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:32.684153    2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-235.ec2.internal\" not found" node="ip-10-0-129-235.ec2.internal"
Jul 27 19:26:33.479637 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:33.479606    2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-235.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:33.727119 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:33.727081    2558 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-235.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="6.4s"
Jul 27 19:26:33.925386 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:33.925286    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:26:33.926215 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:33.926192    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:26:33.926361 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:33.926231    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:26:33.926361 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:33.926245    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:26:33.926361 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:33.926277    2558 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-235.ec2.internal"
Jul 27 19:26:33.943836 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:33.943803    2558 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-129-235.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-129-235.ec2.internal"
Jul 27 19:26:34.468161 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:34.468126    2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-235.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:34.769311 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:34.769212    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass"
Jul 27 19:26:34.886040 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:34.885992    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:26:34.886991 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:34.886972    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:26:34.887087 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:34.887006    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:26:34.887087 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:34.887020    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:26:35.016343 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:35.016279    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 27 19:26:35.469071 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:35.469039    2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-235.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:35.735441 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:35.735332    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 27 19:26:36.034647 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:36.034563    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-235.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 27 19:26:36.470579 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:36.470492    2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-235.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:37.473450 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:37.473399    2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-235.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:37.536011 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:37.535984    2558 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:26:38.470027 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:38.469995    2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-235.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:39.469936 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:39.469904    2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-235.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:40.136531 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:40.136501    2558 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-235.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="7s"
Jul 27 19:26:40.344427 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:40.344358    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:26:40.346719 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:40.346702    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:26:40.346822 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:40.346732    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:26:40.346822 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:40.346748    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:26:40.346822 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:40.346782    2558 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-235.ec2.internal"
Jul 27 19:26:40.364189 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:40.364148    2558 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-129-235.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-129-235.ec2.internal"
Jul 27 19:26:40.468874 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:40.468805    2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-235.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:41.468801 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:41.468768    2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-235.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:42.017700 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:42.017668    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass"
Jul 27 19:26:42.469359 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:42.469333    2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-235.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:42.698861 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:42.698832    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal_3c4ec7bc2375d832f57ca8b11d2ae551/setup/0.log"
Jul 27 19:26:42.699013 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:42.698869    2558 generic.go:356] "Generic (PLEG): container finished" podID="3c4ec7bc2375d832f57ca8b11d2ae551" containerID="d993fdb4462bc3071975a7bda993a437eab3494fe04cb78efb35e2b3c69f9fdf" exitCode=1
Jul 27 19:26:42.699013 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:42.698900    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal" event={"ID":"3c4ec7bc2375d832f57ca8b11d2ae551","Type":"ContainerDied","Data":"d993fdb4462bc3071975a7bda993a437eab3494fe04cb78efb35e2b3c69f9fdf"}
Jul 27 19:26:42.699076 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:42.699033    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:26:42.699806 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:42.699790    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:26:42.699885 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:42.699821    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:26:42.699885 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:42.699832    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:26:42.700008 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:42.699997    2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-235.ec2.internal\" not found" node="ip-10-0-129-235.ec2.internal"
Jul 27 19:26:42.712190 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:42.712112    2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal.18c63c1e331b6f4f  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal,UID:3c4ec7bc2375d832f57ca8b11d2ae551,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Pulled,Message:Container image \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\" already present on machine,Source:EventSource{Component:kubelet,Host:ip-10-0-129-235.ec2.internal,},FirstTimestamp:2026-07-27 19:26:42.703101775 +0000 UTC m=+15.575293728,LastTimestamp:2026-07-27 19:26:42.703101775 +0000 UTC m=+15.575293728,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-235.ec2.internal,}"
Jul 27 19:26:42.783949 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:42.783871    2558 event.go:359] "Server rejected event (will not retry!)" err="events \"kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal.18c63c1b6be49919\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal.18c63c1b6be49919  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal,UID:3c4ec7bc2375d832f57ca8b11d2ae551,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Created,Message:Created container: setup,Source:EventSource{Component:kubelet,Host:ip-10-0-129-235.ec2.internal,},FirstTimestamp:2026-07-27 19:26:30.770907417 +0000 UTC m=+3.643099369,LastTimestamp:2026-07-27 19:26:42.775678308 +0000 UTC m=+15.647870263,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-235.ec2.internal,}"
Jul 27 19:26:42.792946 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:42.792871    2558 event.go:359] "Server rejected event (will not retry!)" err="events \"kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal.18c63c1b6c6fa04f\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal.18c63c1b6c6fa04f  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal,UID:3c4ec7bc2375d832f57ca8b11d2ae551,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Started,Message:Started container setup,Source:EventSource{Component:kubelet,Host:ip-10-0-129-235.ec2.internal,},FirstTimestamp:2026-07-27 19:26:30.780018767 +0000 UTC m=+3.652210722,LastTimestamp:2026-07-27 19:26:42.783551125 +0000 UTC m=+15.655743078,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-235.ec2.internal,}"
Jul 27 19:26:43.470394 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:43.470363    2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-235.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:43.593922 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:43.593889    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-235.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 27 19:26:43.702141 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:43.702112    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal_3c4ec7bc2375d832f57ca8b11d2ae551/setup/0.log"
Jul 27 19:26:43.702317 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:43.702167    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal" event={"ID":"3c4ec7bc2375d832f57ca8b11d2ae551","Type":"ContainerStarted","Data":"7a7dda71efa9a135ee150d5c7e060ac392d7f7b2a9aec95c5dd8f57fd9b89fce"}
Jul 27 19:26:43.702317 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:43.702303    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:26:43.703151 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:43.703128    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:26:43.703268 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:43.703160    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:26:43.703268 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:43.703172    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:26:43.703363 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:43.703350    2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-235.ec2.internal\" not found" node="ip-10-0-129-235.ec2.internal"
Jul 27 19:26:43.793053 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:43.792975    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:26:43.794053 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:43.794037    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:26:43.794124 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:43.794067    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:26:43.794124 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:43.794076    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:26:44.468290 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:44.468259    2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-235.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:45.468780 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:45.468749    2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-235.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:45.775209 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:45.775126    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 27 19:26:45.820860 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:45.820824    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 27 19:26:46.474324 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:46.474299    2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-235.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:47.146305 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:47.146273    2558 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-235.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="7s"
Jul 27 19:26:47.365123 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:47.365074    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:26:47.366029 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:47.366013    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:26:47.366107 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:47.366052    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:26:47.366107 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:47.366066    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:26:47.366107 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:47.366094    2558 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-235.ec2.internal"
Jul 27 19:26:47.382064 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:47.382031    2558 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-129-235.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-129-235.ec2.internal"
Jul 27 19:26:47.469510 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:47.469457    2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-235.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:47.537106 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:47.537076    2558 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:26:48.467820 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:48.467790    2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-235.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:49.468754 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:49.468720    2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-235.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:50.472560 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:50.472529    2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-235.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:51.469463 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:51.469431    2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-235.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:52.469933 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:52.469906    2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-235.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:53.470051 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:53.470021    2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-235.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:54.155696 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:54.155614    2558 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-235.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="7s"
Jul 27 19:26:54.382804 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:54.382760    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:26:54.384158 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:54.384141    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:26:54.384241 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:54.384174    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:26:54.384241 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:54.384185    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:26:54.384241 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:54.384213    2558 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-235.ec2.internal"
Jul 27 19:26:54.400887 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:54.400861    2558 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-129-235.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-129-235.ec2.internal"
Jul 27 19:26:54.470149 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:54.470092    2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-235.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:54.719832 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:54.719806    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal_3c4ec7bc2375d832f57ca8b11d2ae551/setup/1.log"
Jul 27 19:26:54.720214 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:54.720166    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal_3c4ec7bc2375d832f57ca8b11d2ae551/setup/0.log"
Jul 27 19:26:54.720214 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:54.720203    2558 generic.go:356] "Generic (PLEG): container finished" podID="3c4ec7bc2375d832f57ca8b11d2ae551" containerID="7a7dda71efa9a135ee150d5c7e060ac392d7f7b2a9aec95c5dd8f57fd9b89fce" exitCode=1
Jul 27 19:26:54.720346 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:54.720237    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal" event={"ID":"3c4ec7bc2375d832f57ca8b11d2ae551","Type":"ContainerDied","Data":"7a7dda71efa9a135ee150d5c7e060ac392d7f7b2a9aec95c5dd8f57fd9b89fce"}
Jul 27 19:26:54.720346 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:54.720329    2558 scope.go:117] "RemoveContainer" containerID="d993fdb4462bc3071975a7bda993a437eab3494fe04cb78efb35e2b3c69f9fdf"
Jul 27 19:26:54.720495 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:54.720481    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:26:54.721208 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:54.721189    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:26:54.721284 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:54.721223    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:26:54.721284 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:54.721235    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:26:54.721634 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:54.721438    2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-235.ec2.internal\" not found" node="ip-10-0-129-235.ec2.internal"
Jul 27 19:26:54.721634 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:54.721486    2558 scope.go:117] "RemoveContainer" containerID="d993fdb4462bc3071975a7bda993a437eab3494fe04cb78efb35e2b3c69f9fdf"
Jul 27 19:26:54.729227 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:54.729204    2558 log.go:32] "RemoveContainer from runtime service failed" err="rpc error: code = Unknown desc = failed to delete container k8s_setup_kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal_openshift-machine-config-operator_3c4ec7bc2375d832f57ca8b11d2ae551_0 in pod sandbox 6223ac60670451ea8d93063d83f9fe6fa7a3fac77a3fb51a2b25d10225fcd2f7 from index: no such id: 'd993fdb4462bc3071975a7bda993a437eab3494fe04cb78efb35e2b3c69f9fdf'" containerID="d993fdb4462bc3071975a7bda993a437eab3494fe04cb78efb35e2b3c69f9fdf"
Jul 27 19:26:54.729295 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:54.729245    2558 kuberuntime_container.go:962] "Unhandled Error" err="failed to remove pod init container \"setup\": rpc error: code = Unknown desc = failed to delete container k8s_setup_kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal_openshift-machine-config-operator_3c4ec7bc2375d832f57ca8b11d2ae551_0 in pod sandbox 6223ac60670451ea8d93063d83f9fe6fa7a3fac77a3fb51a2b25d10225fcd2f7 from index: no such id: 'd993fdb4462bc3071975a7bda993a437eab3494fe04cb78efb35e2b3c69f9fdf'; Skipping pod \"kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal_openshift-machine-config-operator(3c4ec7bc2375d832f57ca8b11d2ae551)\"" logger="UnhandledError"
Jul 27 19:26:54.729406 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:54.729390    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"setup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=setup pod=kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal_openshift-machine-config-operator(3c4ec7bc2375d832f57ca8b11d2ae551)\"" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal" podUID="3c4ec7bc2375d832f57ca8b11d2ae551"
Jul 27 19:26:54.737649 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:54.737575    2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal.18c63c20ffed8da9  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal,UID:3c4ec7bc2375d832f57ca8b11d2ae551,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:BackOff,Message:Back-off restarting failed container setup in pod kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal_openshift-machine-config-operator(3c4ec7bc2375d832f57ca8b11d2ae551),Source:EventSource{Component:kubelet,Host:ip-10-0-129-235.ec2.internal,},FirstTimestamp:2026-07-27 19:26:54.729358761 +0000 UTC m=+27.601550726,LastTimestamp:2026-07-27 19:26:54.729358761 +0000 UTC m=+27.601550726,Count:1,Type:Warning,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-235.ec2.internal,}"
Jul 27 19:26:55.468784 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:55.468757    2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-235.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:55.722547 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:55.722471    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal_3c4ec7bc2375d832f57ca8b11d2ae551/setup/1.log"
Jul 27 19:26:56.468980 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:56.468953    2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-235.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:57.469298 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:57.469269    2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-235.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:57.537823 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:57.537760    2558 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:26:58.469883 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:58.469854    2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-235.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:59.469794 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:26:59.469760    2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-235.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:59.735965 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:26:59.735887    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass"
Jul 27 19:27:00.469753 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:00.469724    2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-235.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:27:01.088559 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:01.088518    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:27:01.089517 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:01.089498    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:27:01.089616 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:01.089531    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:27:01.089616 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:01.089542    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:27:01.166223 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:01.166190    2558 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-235.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="7s"
Jul 27 19:27:01.401685 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:01.401580    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:27:01.404052 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:01.404034    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:27:01.404159 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:01.404062    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:27:01.404159 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:01.404072    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:27:01.404159 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:01.404107    2558 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-235.ec2.internal"
Jul 27 19:27:01.422244 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:01.422219    2558 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-129-235.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-129-235.ec2.internal"
Jul 27 19:27:01.468973 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:01.468948    2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-235.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:27:02.238094 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:02.238064    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-235.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 27 19:27:02.469498 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:02.469471    2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-235.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:27:03.469574 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:03.469543    2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-235.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:27:04.469206 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:04.469172    2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-235.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:27:05.468030 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:05.468000    2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-235.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:27:06.475534 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:06.475497    2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-235.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:27:06.668512 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:06.668480    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:27:06.669513 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:06.669491    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:27:06.669640 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:06.669521    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:27:06.669640 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:06.669534    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:27:06.669747 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:06.669720    2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-235.ec2.internal\" not found" node="ip-10-0-129-235.ec2.internal"
Jul 27 19:27:06.680347 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:06.680275    2558 event.go:359] "Server rejected event (will not retry!)" err="events \"kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal.18c63c1e331b6f4f\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal.18c63c1e331b6f4f  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal,UID:3c4ec7bc2375d832f57ca8b11d2ae551,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Pulled,Message:Container image \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\" already present on machine,Source:EventSource{Component:kubelet,Host:ip-10-0-129-235.ec2.internal,},FirstTimestamp:2026-07-27 19:26:42.703101775 +0000 UTC m=+15.575293728,LastTimestamp:2026-07-27 19:27:06.67040072 +0000 UTC m=+39.542592675,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-235.ec2.internal,}"
Jul 27 19:27:06.740045 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:06.740024    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal_3c4ec7bc2375d832f57ca8b11d2ae551/setup/1.log"
Jul 27 19:27:06.746923 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:06.746851    2558 event.go:359] "Server rejected event (will not retry!)" err="events \"kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal.18c63c1b6be49919\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal.18c63c1b6be49919  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal,UID:3c4ec7bc2375d832f57ca8b11d2ae551,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Created,Message:Created container: setup,Source:EventSource{Component:kubelet,Host:ip-10-0-129-235.ec2.internal,},FirstTimestamp:2026-07-27 19:26:30.770907417 +0000 UTC m=+3.643099369,LastTimestamp:2026-07-27 19:27:06.73757786 +0000 UTC m=+39.609769813,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-235.ec2.internal,}"
Jul 27 19:27:06.757845 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:06.757780    2558 event.go:359] "Server rejected event (will not retry!)" err="events \"kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal.18c63c1b6c6fa04f\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal.18c63c1b6c6fa04f  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal,UID:3c4ec7bc2375d832f57ca8b11d2ae551,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Started,Message:Started container setup,Source:EventSource{Component:kubelet,Host:ip-10-0-129-235.ec2.internal,},FirstTimestamp:2026-07-27 19:26:30.780018767 +0000 UTC m=+3.652210722,LastTimestamp:2026-07-27 19:27:06.746424889 +0000 UTC m=+39.618616859,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-235.ec2.internal,}"
Jul 27 19:27:07.472577 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:07.472549    2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-235.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:27:07.538891 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:07.538855    2558 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:07.690812 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:07.690780    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 27 19:27:07.743508 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:07.743444    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal_3c4ec7bc2375d832f57ca8b11d2ae551/setup/1.log"
Jul 27 19:27:07.743508 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:07.743483    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal" event={"ID":"3c4ec7bc2375d832f57ca8b11d2ae551","Type":"ContainerStarted","Data":"71e65ca902a5f8d6923443b1c9a99ec5e16e561aa1f7acfbab194c31fa90884e"}
Jul 27 19:27:07.743660 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:07.743616    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:27:07.744497 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:07.744479    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:27:07.744637 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:07.744509    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:27:07.744637 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:07.744520    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:27:07.744750 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:07.744706    2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-235.ec2.internal\" not found" node="ip-10-0-129-235.ec2.internal"
Jul 27 19:27:08.176372 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:08.176275    2558 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-235.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="7s"
Jul 27 19:27:08.422472 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:08.422432    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:27:08.423568 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:08.423539    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:27:08.423568 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:08.423574    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:27:08.423712 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:08.423585    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:27:08.423712 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:08.423611    2558 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-235.ec2.internal"
Jul 27 19:27:08.441459 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:08.441385    2558 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-129-235.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-129-235.ec2.internal"
Jul 27 19:27:08.468606 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:08.468583    2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-235.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:27:09.469200 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:09.469164    2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-235.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:27:10.474832 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:10.474797    2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-235.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:27:10.535504 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:10.535474    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 27 19:27:11.469203 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:11.469169    2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-235.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:27:11.791380 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:11.791312    2558 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bf6f8"
Jul 27 19:27:11.800011 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:11.799993    2558 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bf6f8"
Jul 27 19:27:12.412938 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:12.412894    2558 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 27 19:27:12.478899 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:12.478868    2558 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-129-235.ec2.internal" not found
Jul 27 19:27:12.494798 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:12.494774    2558 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-129-235.ec2.internal" not found
Jul 27 19:27:12.554222 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:12.554197    2558 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-129-235.ec2.internal" not found
Jul 27 19:27:12.804583 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:12.804507    2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-26 19:22:11 +0000 UTC" deadline="2028-01-10 09:30:59.643867874 +0000 UTC"
Jul 27 19:27:12.804583 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:12.804535    2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12758h3m46.839336158s"
Jul 27 19:27:12.815609 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:12.815586    2558 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-129-235.ec2.internal" not found
Jul 27 19:27:12.815609 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:12.815604    2558 csi_plugin.go:399] Failed to initialize CSINode: error updating CSINode annotation: timed out waiting for the condition; caused by: nodes "ip-10-0-129-235.ec2.internal" not found
Jul 27 19:27:12.852543 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:12.852525    2558 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-129-235.ec2.internal" not found
Jul 27 19:27:12.868109 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:12.868092    2558 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-129-235.ec2.internal" not found
Jul 27 19:27:12.923976 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:12.923952    2558 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-129-235.ec2.internal" not found
Jul 27 19:27:13.191064 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:13.191041    2558 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-129-235.ec2.internal" not found
Jul 27 19:27:13.191064 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:13.191062    2558 csi_plugin.go:399] Failed to initialize CSINode: error updating CSINode annotation: timed out waiting for the condition; caused by: nodes "ip-10-0-129-235.ec2.internal" not found
Jul 27 19:27:13.454684 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:13.454612    2558 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-129-235.ec2.internal" not found
Jul 27 19:27:13.470080 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:13.470059    2558 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-129-235.ec2.internal" not found
Jul 27 19:27:13.529557 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:13.529535    2558 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-129-235.ec2.internal" not found
Jul 27 19:27:13.800392 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:13.800310    2558 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-129-235.ec2.internal" not found
Jul 27 19:27:13.800392 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:13.800334    2558 csi_plugin.go:399] Failed to initialize CSINode: error updating CSINode annotation: timed out waiting for the condition; caused by: nodes "ip-10-0-129-235.ec2.internal" not found
Jul 27 19:27:15.182516 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:15.182483    2558 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-129-235.ec2.internal\" not found" node="ip-10-0-129-235.ec2.internal"
Jul 27 19:27:15.441719 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:15.441648    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:27:15.442643 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:15.442629    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:27:15.442708 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:15.442659    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:27:15.442708 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:15.442696    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:27:15.442773 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:15.442721    2558 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-235.ec2.internal"
Jul 27 19:27:15.452862 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:15.452844    2558 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-235.ec2.internal"
Jul 27 19:27:15.452923 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:15.452866    2558 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-235.ec2.internal\": node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:15.494658 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:15.494633    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:15.595169 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:15.595147    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:15.695883 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:15.695812    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:15.796030 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:15.795995    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:15.896205 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:15.896170    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:15.996507 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:15.996447    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:16.097006 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:16.096969    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:16.197902 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:16.197876    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:16.298400 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:16.298325    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:16.398885 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:16.398854    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:16.499057 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:16.499027    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:16.599742 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:16.599688    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:16.700301 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:16.700266    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:16.800693 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:16.800666    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:16.901093 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:16.901072    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:17.002099 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:17.002076    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:17.102557 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:17.102530    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:17.203130 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:17.203076    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:17.303617 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:17.303580    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:17.403794 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:17.403770    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:17.504080 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:17.504022    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:17.539351 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:17.539329    2558 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:17.604292 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:17.604261    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:17.704972 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:17.704939    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:17.805349 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:17.805319    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:17.905857 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:17.905820    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:18.006604 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:18.006569    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:18.107117 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:18.107057    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:18.207876 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:18.207852    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:18.308296 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:18.308259    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:18.409368 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:18.409343    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:18.509473 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:18.509439    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:18.610092 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:18.610062    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:18.710765 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:18.710708    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:18.763342 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:18.763321    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal_3c4ec7bc2375d832f57ca8b11d2ae551/setup/2.log"
Jul 27 19:27:18.763672 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:18.763658    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal_3c4ec7bc2375d832f57ca8b11d2ae551/setup/1.log"
Jul 27 19:27:18.763721 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:18.763690    2558 generic.go:356] "Generic (PLEG): container finished" podID="3c4ec7bc2375d832f57ca8b11d2ae551" containerID="71e65ca902a5f8d6923443b1c9a99ec5e16e561aa1f7acfbab194c31fa90884e" exitCode=1
Jul 27 19:27:18.763758 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:18.763717    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal" event={"ID":"3c4ec7bc2375d832f57ca8b11d2ae551","Type":"ContainerDied","Data":"71e65ca902a5f8d6923443b1c9a99ec5e16e561aa1f7acfbab194c31fa90884e"}
Jul 27 19:27:18.763790 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:18.763757    2558 scope.go:117] "RemoveContainer" containerID="7a7dda71efa9a135ee150d5c7e060ac392d7f7b2a9aec95c5dd8f57fd9b89fce"
Jul 27 19:27:18.763841 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:18.763829    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:27:18.764882 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:18.764707    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:27:18.764882 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:18.764733    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:27:18.764882 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:18.764745    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:27:18.764976 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:18.764948    2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-235.ec2.internal\" not found" node="ip-10-0-129-235.ec2.internal"
Jul 27 19:27:18.765019 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:18.765007    2558 scope.go:117] "RemoveContainer" containerID="7a7dda71efa9a135ee150d5c7e060ac392d7f7b2a9aec95c5dd8f57fd9b89fce"
Jul 27 19:27:18.773742 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:18.773717    2558 log.go:32] "RemoveContainer from runtime service failed" err="rpc error: code = Unknown desc = failed to delete container k8s_setup_kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal_openshift-machine-config-operator_3c4ec7bc2375d832f57ca8b11d2ae551_1 in pod sandbox 6223ac60670451ea8d93063d83f9fe6fa7a3fac77a3fb51a2b25d10225fcd2f7 from index: no such id: '7a7dda71efa9a135ee150d5c7e060ac392d7f7b2a9aec95c5dd8f57fd9b89fce'" containerID="7a7dda71efa9a135ee150d5c7e060ac392d7f7b2a9aec95c5dd8f57fd9b89fce"
Jul 27 19:27:18.773827 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:18.773759    2558 kuberuntime_container.go:962] "Unhandled Error" err="failed to remove pod init container \"setup\": rpc error: code = Unknown desc = failed to delete container k8s_setup_kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal_openshift-machine-config-operator_3c4ec7bc2375d832f57ca8b11d2ae551_1 in pod sandbox 6223ac60670451ea8d93063d83f9fe6fa7a3fac77a3fb51a2b25d10225fcd2f7 from index: no such id: '7a7dda71efa9a135ee150d5c7e060ac392d7f7b2a9aec95c5dd8f57fd9b89fce'; Skipping pod \"kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal_openshift-machine-config-operator(3c4ec7bc2375d832f57ca8b11d2ae551)\"" logger="UnhandledError"
Jul 27 19:27:18.773893 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:18.773878    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"setup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=setup pod=kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal_openshift-machine-config-operator(3c4ec7bc2375d832f57ca8b11d2ae551)\"" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal" podUID="3c4ec7bc2375d832f57ca8b11d2ae551"
Jul 27 19:27:18.811807 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:18.811776    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:18.912236 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:18.912207    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:19.013083 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:19.013020    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:19.113473 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:19.113439    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:19.214028 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:19.213985    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:19.314511 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:19.314441    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:19.414992 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:19.414955    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:19.515077 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:19.515037    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:19.616089 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:19.616060    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:19.716733 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:19.716708    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:19.766230 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:19.766206    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal_3c4ec7bc2375d832f57ca8b11d2ae551/setup/2.log"
Jul 27 19:27:19.817826 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:19.817806    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:19.918833 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:19.918780    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:20.019387 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:20.019360    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:20.119944 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:20.119914    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:20.220397 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:20.220342    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:20.320906 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:20.320867    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:20.421475 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:20.421448    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:20.522581 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:20.522520    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:20.623091 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:20.623061    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:20.723149 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:20.723125    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:20.823658 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:20.823565    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:20.924253 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:20.924231    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:21.024936 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:21.024895    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:21.125466 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:21.125434    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:21.225986 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:21.225964    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:21.326447 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:21.326401    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:21.427139 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:21.427067    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:21.528133 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:21.528098    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:21.628843 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:21.628806    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:21.729358 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:21.729312    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:21.829755 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:21.829719    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:21.930351 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:21.930317    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:22.031060 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:22.030987    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:22.131538 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:22.131509    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:22.232461 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:22.232438    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:22.332981 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:22.332920    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:22.433687 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:22.433649    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:22.534676 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:22.534635    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:22.634788 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:22.634764    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:22.735373 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:22.735344    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:22.835813 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:22.835784    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:22.936512 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:22.936434    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:23.036791 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:23.036759    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:23.137881 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:23.137850    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:23.238847 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:23.238799    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:23.339267 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:23.339237    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:23.439943 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:23.439914    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:23.540986 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:23.540930    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:23.641576 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:23.641549    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:23.742171 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:23.742147    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:23.842859 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:23.842769    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:23.943895 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:23.943860    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:24.044680 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:24.044644    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:24.145089 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:24.145072    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:24.245767 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:24.245752    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:24.346008 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:24.345985    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:24.446998 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:24.446922    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:24.547953 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:24.547920    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:24.648548 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:24.648520    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:24.749333 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:24.749266    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:24.849590 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:24.849548    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:24.950238 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:24.950198    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:25.050947 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:25.050882    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:25.151376 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:25.151348    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:25.252151 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:25.252125    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:25.353149 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:25.353064    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:25.453187 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:25.453159    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:25.528987 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:25.528942    2558 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-235.ec2.internal\": node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:25.553834 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:25.553811    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:25.654279 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:25.654258    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:25.754903 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:25.754883    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:25.855171 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:25.855136    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:25.955853 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:25.955792    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:26.056230 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:26.056203    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:26.156673 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:26.156641    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:26.257084 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:26.257040    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:26.357865 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:26.357827    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:26.458510 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:26.458479    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:26.559394 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:26.559331    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:26.660488 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:26.660450    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:26.761090 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:26.761066    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:26.861446 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:26.861359    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:26.962221 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:26.962182    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:27.062676 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:27.062640    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:27.163676 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:27.163654    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:27.264230 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:27.264196    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:27.364690 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:27.364652    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:27.464846 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:27.464787    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:27.540343 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:27.540312    2558 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:27.564985 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:27.564963    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:27.665640 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:27.665607    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:27.765999 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:27.765909    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:27.866327 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:27.866285    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:27.967162 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:27.967121    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:28.068186 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:28.068117    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:28.168613 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:28.168580    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:28.269168 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:28.269137    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:28.370049 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:28.370023    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:28.470835 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:28.470789    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:28.571584 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:28.571549    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:28.672503 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:28.672446    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:28.772911 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:28.772877    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:28.873301 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:28.873256    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:28.974065 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:28.974008    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:29.075056 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:29.075032    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:29.175554 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:29.175522    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:29.276070 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:29.276021    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:29.376483 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:29.376451    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:29.477378 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:29.477348    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:29.578265 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:29.578207    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:29.678916 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:29.678887    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:29.778955 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:29.778931    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:29.879817 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:29.879799    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:29.980308 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:29.980288    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:30.080891 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:30.080854    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:30.181357 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:30.181293    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:30.281895 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:30.281860    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:30.382913 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:30.382878    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:30.483805 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:30.483737    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:30.584536 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:30.584503    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:30.685236 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:30.685206    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:30.785793 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:30.785725    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:30.886193 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:30.886167    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:30.987034 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:30.986992    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:31.087509 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:31.087445    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:31.188249 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:31.188214    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:31.288820 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:31.288780    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:31.389757 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:31.389724    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:31.490223 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:31.490198    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:31.590871 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:31.590835    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:31.669123 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:31.669077    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:27:31.673322 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:31.673297    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:27:31.673427 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:31.673338    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:27:31.673427 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:31.673348    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:27:31.673562 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:31.673549    2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-235.ec2.internal\" not found" node="ip-10-0-129-235.ec2.internal"
Jul 27 19:27:31.673700 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:31.673684    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"setup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=setup pod=kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal_openshift-machine-config-operator(3c4ec7bc2375d832f57ca8b11d2ae551)\"" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal" podUID="3c4ec7bc2375d832f57ca8b11d2ae551"
Jul 27 19:27:31.690914 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:31.690892    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:31.791790 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:31.791764    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:31.892262 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:31.892238    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:31.993109 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:31.993043    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:32.093500 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:32.093471    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:32.194019 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:32.193987    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:32.294135 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:32.294089    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:32.394487 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:32.394453    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:32.495444 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:32.495398    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:32.596519 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:32.596459    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:32.697061 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:32.697023    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:32.797379 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:32.797353    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:32.897873 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:32.897846    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:32.998769 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:32.998730    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:33.099226 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:33.099190    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:33.199759 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:33.199699    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:33.300131 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:33.300105    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:33.400596 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:33.400572    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:33.501596 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:33.501536    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:33.602249 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:33.602217    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:33.702296 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:33.702269    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:33.802831 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:33.802767    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:33.903305 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:33.903267    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:34.004012 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:34.003986    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:34.104469 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:34.104405    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:34.204939 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:34.204902    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:34.305376 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:34.305343    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:34.406328 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:34.406302    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:34.460093 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:34.460075    2558 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 19:27:34.486152 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:34.486136    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:27:34.487142 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:34.487125    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:27:34.487202 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:34.487156    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:27:34.487202 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:34.487169    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:27:34.506981 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:34.506962    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:34.607521 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:34.607496    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:34.708144 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:34.708074    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:34.808564 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:34.808539    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:34.909112 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:34.909079    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:35.009918 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:35.009853    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:35.110358 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:35.110334    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-235.ec2.internal\" not found"
Jul 27 19:27:35.153319 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.153296    2558 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 19:27:35.172375 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.172355    2558 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-235.ec2.internal"
Jul 27 19:27:35.181999 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.181983    2558 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 27 19:27:35.183538 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.183525    2558 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal"
Jul 27 19:27:35.195209 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.195192    2558 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 27 19:27:35.487060 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.487034    2558 apiserver.go:52] "Watching apiserver"
Jul 27 19:27:35.494307 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.494282    2558 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 27 19:27:35.494676 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.494654    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-bnrsg","openshift-image-registry/node-ca-qljqz","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal","openshift-multus/network-metrics-daemon-zbjtr","kube-system/global-pull-secret-syncer-2g8q7","openshift-multus/multus-2fqrl","openshift-multus/multus-additional-cni-plugins-2xk7d","openshift-network-diagnostics/network-check-target-2vtnn","openshift-network-operator/iptables-alerter-6bsx5","openshift-ovn-kubernetes/ovnkube-node-5dfc2","kube-system/konnectivity-agent-5zsvw","kube-system/kube-apiserver-proxy-ip-10-0-129-235.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4dkh","openshift-cluster-node-tuning-operator/tuned-2cw79"]
Jul 27 19:27:35.498457 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.498434    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2xk7d"
Jul 27 19:27:35.501142 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.500885    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 27 19:27:35.501340 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.501324    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 27 19:27:35.501735 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.501704    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 27 19:27:35.501833 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.501710    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 27 19:27:35.501833 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.501813    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4dkh"
Jul 27 19:27:35.502597 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.502580    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 27 19:27:35.503324 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.503299    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-f2lp5\""
Jul 27 19:27:35.504066 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.504044    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-2cw79"
Jul 27 19:27:35.504167 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.504087    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 27 19:27:35.504229 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.504162    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-dvtqg\""
Jul 27 19:27:35.504288 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.504253    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 27 19:27:35.504453 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.504438    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 27 19:27:35.506247 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.506092    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-lrdkg\""
Jul 27 19:27:35.506340 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.506311    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 27 19:27:35.506402 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.506355    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 27 19:27:35.506559 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.506544    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zbjtr"
Jul 27 19:27:35.506639 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:35.506622    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zbjtr" podUID="229036e0-7551-4987-98d3-0ccef9c511e5"
Jul 27 19:27:35.507124 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.507105    2558 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 27 19:27:35.508740 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.508720    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2g8q7"
Jul 27 19:27:35.508808 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:35.508785    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2g8q7" podUID="3590f4c7-efa9-4c98-ae82-99afc906db15"
Jul 27 19:27:35.511063 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.511046    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-2fqrl"
Jul 27 19:27:35.513206 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.513189    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 27 19:27:35.513298 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.513241    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-6bsx5"
Jul 27 19:27:35.513298 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.513260    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-xng96\""
Jul 27 19:27:35.515643 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.515579    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 27 19:27:35.515643 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.515617    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 27 19:27:35.515815 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.515671    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bnrsg"
Jul 27 19:27:35.515888 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.515841    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 27 19:27:35.515943 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.515889    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-48rxj\""
Jul 27 19:27:35.517929 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.517896    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-nmhfp\""
Jul 27 19:27:35.517929 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.517908    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 27 19:27:35.518095 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.518079    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 27 19:27:35.518156 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.518130    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qljqz"
Jul 27 19:27:35.518520 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.518499    2558 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 27 19:27:35.520252 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.520234    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 27 19:27:35.520330 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.520319    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-f4fwn\""
Jul 27 19:27:35.520455 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.520430    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:27:35.520538 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.520521    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 27 19:27:35.520633 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.520521    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 27 19:27:35.522714 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.522699    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-5zsvw"
Jul 27 19:27:35.523317 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.523213    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 27 19:27:35.523317 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.523213    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-pklxz\""
Jul 27 19:27:35.523781 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.523624    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 27 19:27:35.523781 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.523650    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 27 19:27:35.523781 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.523686    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 27 19:27:35.524527 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.524507    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 27 19:27:35.524619 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.524514    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 27 19:27:35.525029 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.525003    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 27 19:27:35.525103 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.525044    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2vtnn"
Jul 27 19:27:35.525269 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.525253    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 27 19:27:35.525321 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.525289    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-6zvbh\""
Jul 27 19:27:35.525453 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:35.525435    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2vtnn" podUID="88050dfb-f434-42bd-8c5f-9dc6f98673e0"
Jul 27 19:27:35.525562 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:35.525532    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"setup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=setup pod=kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal_openshift-machine-config-operator(3c4ec7bc2375d832f57ca8b11d2ae551)\"" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal" podUID="3c4ec7bc2375d832f57ca8b11d2ae551"
Jul 27 19:27:35.573595 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.573570    2558 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 27 19:27:35.594474 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.594445    2558 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-67dfb"
Jul 27 19:27:35.602974 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.602951    2558 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-67dfb"
Jul 27 19:27:35.628841 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.628814    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4869a388-94e3-4a30-a33e-29579dcad555-host\") pod \"tuned-2cw79\" (UID: \"4869a388-94e3-4a30-a33e-29579dcad555\") " pod="openshift-cluster-node-tuning-operator/tuned-2cw79"
Jul 27 19:27:35.628841 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.628839    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9a0f4ae-3761-4764-baa1-8d8c005113a0-system-cni-dir\") pod \"multus-additional-cni-plugins-2xk7d\" (UID: \"e9a0f4ae-3761-4764-baa1-8d8c005113a0\") " pod="openshift-multus/multus-additional-cni-plugins-2xk7d"
Jul 27 19:27:35.628986 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.628855    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/103bab23-bca3-472a-974b-7731f83c5bfb-run-systemd\") pod \"ovnkube-node-5dfc2\" (UID: \"103bab23-bca3-472a-974b-7731f83c5bfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:27:35.628986 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.628870    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/103bab23-bca3-472a-974b-7731f83c5bfb-node-log\") pod \"ovnkube-node-5dfc2\" (UID: \"103bab23-bca3-472a-974b-7731f83c5bfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:27:35.628986 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.628907    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/103bab23-bca3-472a-974b-7731f83c5bfb-env-overrides\") pod \"ovnkube-node-5dfc2\" (UID: \"103bab23-bca3-472a-974b-7731f83c5bfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:27:35.628986 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.628930    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/103bab23-bca3-472a-974b-7731f83c5bfb-ovnkube-script-lib\") pod \"ovnkube-node-5dfc2\" (UID: \"103bab23-bca3-472a-974b-7731f83c5bfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:27:35.628986 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.628947    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/438069a4-ca90-4995-ba78-32af922abb36-host-var-lib-kubelet\") pod \"multus-2fqrl\" (UID: \"438069a4-ca90-4995-ba78-32af922abb36\") " pod="openshift-multus/multus-2fqrl"
Jul 27 19:27:35.628986 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.628961    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/438069a4-ca90-4995-ba78-32af922abb36-etc-kubernetes\") pod \"multus-2fqrl\" (UID: \"438069a4-ca90-4995-ba78-32af922abb36\") " pod="openshift-multus/multus-2fqrl"
Jul 27 19:27:35.628986 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.628975    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e0267f96-3a9d-45d1-b943-92899defe086-host\") pod \"node-ca-qljqz\" (UID: \"e0267f96-3a9d-45d1-b943-92899defe086\") " pod="openshift-image-registry/node-ca-qljqz"
Jul 27 19:27:35.629267 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.628993    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2xd5\" (UniqueName: \"kubernetes.io/projected/229036e0-7551-4987-98d3-0ccef9c511e5-kube-api-access-r2xd5\") pod \"network-metrics-daemon-zbjtr\" (UID: \"229036e0-7551-4987-98d3-0ccef9c511e5\") " pod="openshift-multus/network-metrics-daemon-zbjtr"
Jul 27 19:27:35.629267 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.629014    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/438069a4-ca90-4995-ba78-32af922abb36-os-release\") pod \"multus-2fqrl\" (UID: \"438069a4-ca90-4995-ba78-32af922abb36\") " pod="openshift-multus/multus-2fqrl"
Jul 27 19:27:35.629267 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.629029    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpvcw\" (UniqueName: \"kubernetes.io/projected/e9a0f4ae-3761-4764-baa1-8d8c005113a0-kube-api-access-bpvcw\") pod \"multus-additional-cni-plugins-2xk7d\" (UID: \"e9a0f4ae-3761-4764-baa1-8d8c005113a0\") " pod="openshift-multus/multus-additional-cni-plugins-2xk7d"
Jul 27 19:27:35.629267 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.629048    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/24afe7ac-4ce8-43b3-a919-1dc0a0e76fb5-etc-selinux\") pod \"aws-ebs-csi-driver-node-m4dkh\" (UID: \"24afe7ac-4ce8-43b3-a919-1dc0a0e76fb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4dkh"
Jul 27 19:27:35.629267 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.629095    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4869a388-94e3-4a30-a33e-29579dcad555-etc-sysctl-d\") pod \"tuned-2cw79\" (UID: \"4869a388-94e3-4a30-a33e-29579dcad555\") " pod="openshift-cluster-node-tuning-operator/tuned-2cw79"
Jul 27 19:27:35.629267 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.629120    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e9a0f4ae-3761-4764-baa1-8d8c005113a0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2xk7d\" (UID: \"e9a0f4ae-3761-4764-baa1-8d8c005113a0\") " pod="openshift-multus/multus-additional-cni-plugins-2xk7d"
Jul 27 19:27:35.629267 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.629138    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/103bab23-bca3-472a-974b-7731f83c5bfb-host-cni-netd\") pod \"ovnkube-node-5dfc2\" (UID: \"103bab23-bca3-472a-974b-7731f83c5bfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:27:35.629267 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.629178    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/438069a4-ca90-4995-ba78-32af922abb36-cni-binary-copy\") pod \"multus-2fqrl\" (UID: \"438069a4-ca90-4995-ba78-32af922abb36\") " pod="openshift-multus/multus-2fqrl"
Jul 27 19:27:35.629267 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.629217    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/438069a4-ca90-4995-ba78-32af922abb36-host-run-k8s-cni-cncf-io\") pod \"multus-2fqrl\" (UID: \"438069a4-ca90-4995-ba78-32af922abb36\") " pod="openshift-multus/multus-2fqrl"
Jul 27 19:27:35.629267 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.629260    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/87618c00-0367-482a-813e-9aab8f341fa5-host-slash\") pod \"iptables-alerter-6bsx5\" (UID: \"87618c00-0367-482a-813e-9aab8f341fa5\") " pod="openshift-network-operator/iptables-alerter-6bsx5"
Jul 27 19:27:35.629664 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.629286    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fddl\" (UniqueName: \"kubernetes.io/projected/4869a388-94e3-4a30-a33e-29579dcad555-kube-api-access-6fddl\") pod \"tuned-2cw79\" (UID: \"4869a388-94e3-4a30-a33e-29579dcad555\") " pod="openshift-cluster-node-tuning-operator/tuned-2cw79"
Jul 27 19:27:35.629664 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.629318    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/438069a4-ca90-4995-ba78-32af922abb36-cnibin\") pod \"multus-2fqrl\" (UID: \"438069a4-ca90-4995-ba78-32af922abb36\") " pod="openshift-multus/multus-2fqrl"
Jul 27 19:27:35.629664 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.629342    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/24afe7ac-4ce8-43b3-a919-1dc0a0e76fb5-registration-dir\") pod \"aws-ebs-csi-driver-node-m4dkh\" (UID: \"24afe7ac-4ce8-43b3-a919-1dc0a0e76fb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4dkh"
Jul 27 19:27:35.629664 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.629373    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3590f4c7-efa9-4c98-ae82-99afc906db15-kubelet-config\") pod \"global-pull-secret-syncer-2g8q7\" (UID: \"3590f4c7-efa9-4c98-ae82-99afc906db15\") " pod="kube-system/global-pull-secret-syncer-2g8q7"
Jul 27 19:27:35.629664 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.629401    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nx94q\" (UniqueName: \"kubernetes.io/projected/103bab23-bca3-472a-974b-7731f83c5bfb-kube-api-access-nx94q\") pod \"ovnkube-node-5dfc2\" (UID: \"103bab23-bca3-472a-974b-7731f83c5bfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:27:35.629664 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.629533    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/438069a4-ca90-4995-ba78-32af922abb36-multus-socket-dir-parent\") pod \"multus-2fqrl\" (UID: \"438069a4-ca90-4995-ba78-32af922abb36\") " pod="openshift-multus/multus-2fqrl"
Jul 27 19:27:35.629664 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.629559    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqjj9\" (UniqueName: \"kubernetes.io/projected/e0267f96-3a9d-45d1-b943-92899defe086-kube-api-access-jqjj9\") pod \"node-ca-qljqz\" (UID: \"e0267f96-3a9d-45d1-b943-92899defe086\") " pod="openshift-image-registry/node-ca-qljqz"
Jul 27 19:27:35.629664 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.629574    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e9a0f4ae-3761-4764-baa1-8d8c005113a0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2xk7d\" (UID: \"e9a0f4ae-3761-4764-baa1-8d8c005113a0\") " pod="openshift-multus/multus-additional-cni-plugins-2xk7d"
Jul 27 19:27:35.629664 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.629590    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3590f4c7-efa9-4c98-ae82-99afc906db15-dbus\") pod \"global-pull-secret-syncer-2g8q7\" (UID: \"3590f4c7-efa9-4c98-ae82-99afc906db15\") " pod="kube-system/global-pull-secret-syncer-2g8q7"
Jul 27 19:27:35.629664 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.629603    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/103bab23-bca3-472a-974b-7731f83c5bfb-var-lib-openvswitch\") pod \"ovnkube-node-5dfc2\" (UID: \"103bab23-bca3-472a-974b-7731f83c5bfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:27:35.629664 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.629621    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/103bab23-bca3-472a-974b-7731f83c5bfb-run-ovn\") pod \"ovnkube-node-5dfc2\" (UID: \"103bab23-bca3-472a-974b-7731f83c5bfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:27:35.629664 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.629644    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4869a388-94e3-4a30-a33e-29579dcad555-etc-sysconfig\") pod \"tuned-2cw79\" (UID: \"4869a388-94e3-4a30-a33e-29579dcad555\") " pod="openshift-cluster-node-tuning-operator/tuned-2cw79"
Jul 27 19:27:35.629664 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.629664    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4869a388-94e3-4a30-a33e-29579dcad555-sys\") pod \"tuned-2cw79\" (UID: \"4869a388-94e3-4a30-a33e-29579dcad555\") " pod="openshift-cluster-node-tuning-operator/tuned-2cw79"
Jul 27 19:27:35.630146 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.629682    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/438069a4-ca90-4995-ba78-32af922abb36-multus-daemon-config\") pod \"multus-2fqrl\" (UID: \"438069a4-ca90-4995-ba78-32af922abb36\") " pod="openshift-multus/multus-2fqrl"
Jul 27 19:27:35.630146 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.629707    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e9a0f4ae-3761-4764-baa1-8d8c005113a0-cni-binary-copy\") pod \"multus-additional-cni-plugins-2xk7d\" (UID: \"e9a0f4ae-3761-4764-baa1-8d8c005113a0\") " pod="openshift-multus/multus-additional-cni-plugins-2xk7d"
Jul 27 19:27:35.630146 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.629724    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e9a0f4ae-3761-4764-baa1-8d8c005113a0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2xk7d\" (UID: \"e9a0f4ae-3761-4764-baa1-8d8c005113a0\") " pod="openshift-multus/multus-additional-cni-plugins-2xk7d"
Jul 27 19:27:35.630146 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.629741    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/438069a4-ca90-4995-ba78-32af922abb36-system-cni-dir\") pod \"multus-2fqrl\" (UID: \"438069a4-ca90-4995-ba78-32af922abb36\") " pod="openshift-multus/multus-2fqrl"
Jul 27 19:27:35.630146 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.629759    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/103bab23-bca3-472a-974b-7731f83c5bfb-systemd-units\") pod \"ovnkube-node-5dfc2\" (UID: \"103bab23-bca3-472a-974b-7731f83c5bfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:27:35.630146 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.629790    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/103bab23-bca3-472a-974b-7731f83c5bfb-host-run-ovn-kubernetes\") pod \"ovnkube-node-5dfc2\" (UID: \"103bab23-bca3-472a-974b-7731f83c5bfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:27:35.630146 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.629826    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/103bab23-bca3-472a-974b-7731f83c5bfb-host-cni-bin\") pod \"ovnkube-node-5dfc2\" (UID: \"103bab23-bca3-472a-974b-7731f83c5bfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:27:35.630146 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.629844    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/103bab23-bca3-472a-974b-7731f83c5bfb-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5dfc2\" (UID: \"103bab23-bca3-472a-974b-7731f83c5bfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:27:35.630146 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.629858    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e9a0f4ae-3761-4764-baa1-8d8c005113a0-cnibin\") pod \"multus-additional-cni-plugins-2xk7d\" (UID: \"e9a0f4ae-3761-4764-baa1-8d8c005113a0\") " pod="openshift-multus/multus-additional-cni-plugins-2xk7d"
Jul 27 19:27:35.630146 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.629872    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/103bab23-bca3-472a-974b-7731f83c5bfb-log-socket\") pod \"ovnkube-node-5dfc2\" (UID: \"103bab23-bca3-472a-974b-7731f83c5bfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:27:35.630146 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.629893    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4869a388-94e3-4a30-a33e-29579dcad555-etc-sysctl-conf\") pod \"tuned-2cw79\" (UID: \"4869a388-94e3-4a30-a33e-29579dcad555\") " pod="openshift-cluster-node-tuning-operator/tuned-2cw79"
Jul 27 19:27:35.630146 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.629907    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4869a388-94e3-4a30-a33e-29579dcad555-lib-modules\") pod \"tuned-2cw79\" (UID: \"4869a388-94e3-4a30-a33e-29579dcad555\") " pod="openshift-cluster-node-tuning-operator/tuned-2cw79"
Jul 27 19:27:35.630146 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.629921    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/438069a4-ca90-4995-ba78-32af922abb36-multus-cni-dir\") pod \"multus-2fqrl\" (UID: \"438069a4-ca90-4995-ba78-32af922abb36\") " pod="openshift-multus/multus-2fqrl"
Jul 27 19:27:35.630146 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.629939    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/24afe7ac-4ce8-43b3-a919-1dc0a0e76fb5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-m4dkh\" (UID: \"24afe7ac-4ce8-43b3-a919-1dc0a0e76fb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4dkh"
Jul 27 19:27:35.630146 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.629955    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/229036e0-7551-4987-98d3-0ccef9c511e5-metrics-certs\") pod \"network-metrics-daemon-zbjtr\" (UID: \"229036e0-7551-4987-98d3-0ccef9c511e5\") " pod="openshift-multus/network-metrics-daemon-zbjtr"
Jul 27 19:27:35.630146 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.629976    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/103bab23-bca3-472a-974b-7731f83c5bfb-etc-openvswitch\") pod \"ovnkube-node-5dfc2\" (UID: \"103bab23-bca3-472a-974b-7731f83c5bfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:27:35.630911 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.629990    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4869a388-94e3-4a30-a33e-29579dcad555-etc-systemd\") pod \"tuned-2cw79\" (UID: \"4869a388-94e3-4a30-a33e-29579dcad555\") " pod="openshift-cluster-node-tuning-operator/tuned-2cw79"
Jul 27 19:27:35.630911 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.630003    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4869a388-94e3-4a30-a33e-29579dcad555-etc-tuned\") pod \"tuned-2cw79\" (UID: \"4869a388-94e3-4a30-a33e-29579dcad555\") " pod="openshift-cluster-node-tuning-operator/tuned-2cw79"
Jul 27 19:27:35.630911 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.630025    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4869a388-94e3-4a30-a33e-29579dcad555-tmp\") pod \"tuned-2cw79\" (UID: \"4869a388-94e3-4a30-a33e-29579dcad555\") " pod="openshift-cluster-node-tuning-operator/tuned-2cw79"
Jul 27 19:27:35.630911 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.630052    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/438069a4-ca90-4995-ba78-32af922abb36-host-run-multus-certs\") pod \"multus-2fqrl\" (UID: \"438069a4-ca90-4995-ba78-32af922abb36\") " pod="openshift-multus/multus-2fqrl"
Jul 27 19:27:35.630911 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.630089    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqjr4\" (UniqueName: \"kubernetes.io/projected/d48f4d00-ad6d-4b8e-9a57-de8ed6729c98-kube-api-access-gqjr4\") pod \"node-resolver-bnrsg\" (UID: \"d48f4d00-ad6d-4b8e-9a57-de8ed6729c98\") " pod="openshift-dns/node-resolver-bnrsg"
Jul 27 19:27:35.630911 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.630115    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7719045e-d600-437b-aa76-7f03b29fecd9-agent-certs\") pod \"konnectivity-agent-5zsvw\" (UID: \"7719045e-d600-437b-aa76-7f03b29fecd9\") " pod="kube-system/konnectivity-agent-5zsvw"
Jul 27 19:27:35.630911 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.630139    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3590f4c7-efa9-4c98-ae82-99afc906db15-original-pull-secret\") pod \"global-pull-secret-syncer-2g8q7\" (UID: \"3590f4c7-efa9-4c98-ae82-99afc906db15\") " pod="kube-system/global-pull-secret-syncer-2g8q7"
Jul 27 19:27:35.630911 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.630160    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/103bab23-bca3-472a-974b-7731f83c5bfb-host-kubelet\") pod \"ovnkube-node-5dfc2\" (UID: \"103bab23-bca3-472a-974b-7731f83c5bfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:27:35.630911 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.630181    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9llnv\" (UniqueName: \"kubernetes.io/projected/87618c00-0367-482a-813e-9aab8f341fa5-kube-api-access-9llnv\") pod \"iptables-alerter-6bsx5\" (UID: \"87618c00-0367-482a-813e-9aab8f341fa5\") " pod="openshift-network-operator/iptables-alerter-6bsx5"
Jul 27 19:27:35.630911 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.630206    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d48f4d00-ad6d-4b8e-9a57-de8ed6729c98-hosts-file\") pod \"node-resolver-bnrsg\" (UID: \"d48f4d00-ad6d-4b8e-9a57-de8ed6729c98\") " pod="openshift-dns/node-resolver-bnrsg"
Jul 27 19:27:35.630911 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.630227    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e0267f96-3a9d-45d1-b943-92899defe086-serviceca\") pod \"node-ca-qljqz\" (UID: \"e0267f96-3a9d-45d1-b943-92899defe086\") " pod="openshift-image-registry/node-ca-qljqz"
Jul 27 19:27:35.630911 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.630251    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7719045e-d600-437b-aa76-7f03b29fecd9-konnectivity-ca\") pod \"konnectivity-agent-5zsvw\" (UID: \"7719045e-d600-437b-aa76-7f03b29fecd9\") " pod="kube-system/konnectivity-agent-5zsvw"
Jul 27 19:27:35.630911 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.630272    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4869a388-94e3-4a30-a33e-29579dcad555-run\") pod \"tuned-2cw79\" (UID: \"4869a388-94e3-4a30-a33e-29579dcad555\") " pod="openshift-cluster-node-tuning-operator/tuned-2cw79"
Jul 27 19:27:35.630911 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.630296    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vpzg\" (UniqueName: \"kubernetes.io/projected/438069a4-ca90-4995-ba78-32af922abb36-kube-api-access-7vpzg\") pod \"multus-2fqrl\" (UID: \"438069a4-ca90-4995-ba78-32af922abb36\") " pod="openshift-multus/multus-2fqrl"
Jul 27 19:27:35.630911 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.630324    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4869a388-94e3-4a30-a33e-29579dcad555-etc-kubernetes\") pod \"tuned-2cw79\" (UID: \"4869a388-94e3-4a30-a33e-29579dcad555\") " pod="openshift-cluster-node-tuning-operator/tuned-2cw79"
Jul 27 19:27:35.630911 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.630347    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/438069a4-ca90-4995-ba78-32af922abb36-host-run-netns\") pod \"multus-2fqrl\" (UID: \"438069a4-ca90-4995-ba78-32af922abb36\") " pod="openshift-multus/multus-2fqrl"
Jul 27 19:27:35.630911 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.630368    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/438069a4-ca90-4995-ba78-32af922abb36-multus-conf-dir\") pod \"multus-2fqrl\" (UID: \"438069a4-ca90-4995-ba78-32af922abb36\") " pod="openshift-multus/multus-2fqrl"
Jul 27 19:27:35.631402 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.630396    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmgsm\" (UniqueName: \"kubernetes.io/projected/88050dfb-f434-42bd-8c5f-9dc6f98673e0-kube-api-access-wmgsm\") pod \"network-check-target-2vtnn\" (UID: \"88050dfb-f434-42bd-8c5f-9dc6f98673e0\") " pod="openshift-network-diagnostics/network-check-target-2vtnn"
Jul 27 19:27:35.631402 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.630456    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hr79d\" (UniqueName: \"kubernetes.io/projected/24afe7ac-4ce8-43b3-a919-1dc0a0e76fb5-kube-api-access-hr79d\") pod \"aws-ebs-csi-driver-node-m4dkh\" (UID: \"24afe7ac-4ce8-43b3-a919-1dc0a0e76fb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4dkh"
Jul 27 19:27:35.631402 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.630488    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/103bab23-bca3-472a-974b-7731f83c5bfb-host-run-netns\") pod \"ovnkube-node-5dfc2\" (UID: \"103bab23-bca3-472a-974b-7731f83c5bfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:27:35.631402 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.630526    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/103bab23-bca3-472a-974b-7731f83c5bfb-ovnkube-config\") pod \"ovnkube-node-5dfc2\" (UID: \"103bab23-bca3-472a-974b-7731f83c5bfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:27:35.631402 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.630553    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/438069a4-ca90-4995-ba78-32af922abb36-host-var-lib-cni-bin\") pod \"multus-2fqrl\" (UID: \"438069a4-ca90-4995-ba78-32af922abb36\") " pod="openshift-multus/multus-2fqrl"
Jul 27 19:27:35.631402 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.630579    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/438069a4-ca90-4995-ba78-32af922abb36-host-var-lib-cni-multus\") pod \"multus-2fqrl\" (UID: \"438069a4-ca90-4995-ba78-32af922abb36\") " pod="openshift-multus/multus-2fqrl"
Jul 27 19:27:35.631402 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.630601    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/438069a4-ca90-4995-ba78-32af922abb36-hostroot\") pod \"multus-2fqrl\" (UID: \"438069a4-ca90-4995-ba78-32af922abb36\") " pod="openshift-multus/multus-2fqrl"
Jul 27 19:27:35.631402 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.630623    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/24afe7ac-4ce8-43b3-a919-1dc0a0e76fb5-socket-dir\") pod \"aws-ebs-csi-driver-node-m4dkh\" (UID: \"24afe7ac-4ce8-43b3-a919-1dc0a0e76fb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4dkh"
Jul 27 19:27:35.631402 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.630646    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/103bab23-bca3-472a-974b-7731f83c5bfb-host-slash\") pod \"ovnkube-node-5dfc2\" (UID: \"103bab23-bca3-472a-974b-7731f83c5bfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:27:35.631402 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.630669    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4869a388-94e3-4a30-a33e-29579dcad555-etc-modprobe-d\") pod \"tuned-2cw79\" (UID: \"4869a388-94e3-4a30-a33e-29579dcad555\") " pod="openshift-cluster-node-tuning-operator/tuned-2cw79"
Jul 27 19:27:35.631402 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.630695    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/87618c00-0367-482a-813e-9aab8f341fa5-iptables-alerter-script\") pod \"iptables-alerter-6bsx5\" (UID: \"87618c00-0367-482a-813e-9aab8f341fa5\") " pod="openshift-network-operator/iptables-alerter-6bsx5"
Jul 27 19:27:35.631402 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.630719    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d48f4d00-ad6d-4b8e-9a57-de8ed6729c98-tmp-dir\") pod \"node-resolver-bnrsg\" (UID: \"d48f4d00-ad6d-4b8e-9a57-de8ed6729c98\") " pod="openshift-dns/node-resolver-bnrsg"
Jul 27 19:27:35.631402 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.630742    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e9a0f4ae-3761-4764-baa1-8d8c005113a0-os-release\") pod \"multus-additional-cni-plugins-2xk7d\" (UID: \"e9a0f4ae-3761-4764-baa1-8d8c005113a0\") " pod="openshift-multus/multus-additional-cni-plugins-2xk7d"
Jul 27 19:27:35.631402 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.630766    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/24afe7ac-4ce8-43b3-a919-1dc0a0e76fb5-device-dir\") pod \"aws-ebs-csi-driver-node-m4dkh\" (UID: \"24afe7ac-4ce8-43b3-a919-1dc0a0e76fb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4dkh"
Jul 27 19:27:35.631402 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.630795    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/24afe7ac-4ce8-43b3-a919-1dc0a0e76fb5-sys-fs\") pod \"aws-ebs-csi-driver-node-m4dkh\" (UID: \"24afe7ac-4ce8-43b3-a919-1dc0a0e76fb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4dkh"
Jul 27 19:27:35.631402 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.630823    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/103bab23-bca3-472a-974b-7731f83c5bfb-run-openvswitch\") pod \"ovnkube-node-5dfc2\" (UID: \"103bab23-bca3-472a-974b-7731f83c5bfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:27:35.631855 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.630853    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/103bab23-bca3-472a-974b-7731f83c5bfb-ovn-node-metrics-cert\") pod \"ovnkube-node-5dfc2\" (UID: \"103bab23-bca3-472a-974b-7731f83c5bfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:27:35.631855 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.630875    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4869a388-94e3-4a30-a33e-29579dcad555-var-lib-kubelet\") pod \"tuned-2cw79\" (UID: \"4869a388-94e3-4a30-a33e-29579dcad555\") " pod="openshift-cluster-node-tuning-operator/tuned-2cw79"
Jul 27 19:27:35.681100 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.681057    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-235.ec2.internal" podStartSLOduration=0.681045358 podStartE2EDuration="681.045358ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 19:27:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:27:35.680579349 +0000 UTC m=+68.552771323" watchObservedRunningTime="2026-07-27 19:27:35.681045358 +0000 UTC m=+68.553237326"
Jul 27 19:27:35.732014 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.731992    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/103bab23-bca3-472a-974b-7731f83c5bfb-node-log\") pod \"ovnkube-node-5dfc2\" (UID: \"103bab23-bca3-472a-974b-7731f83c5bfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:27:35.732124 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.732021    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/103bab23-bca3-472a-974b-7731f83c5bfb-env-overrides\") pod \"ovnkube-node-5dfc2\" (UID: \"103bab23-bca3-472a-974b-7731f83c5bfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:27:35.732124 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.732043    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/103bab23-bca3-472a-974b-7731f83c5bfb-ovnkube-script-lib\") pod \"ovnkube-node-5dfc2\" (UID: \"103bab23-bca3-472a-974b-7731f83c5bfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:27:35.732124 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.732058    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/438069a4-ca90-4995-ba78-32af922abb36-host-var-lib-kubelet\") pod \"multus-2fqrl\" (UID: \"438069a4-ca90-4995-ba78-32af922abb36\") " pod="openshift-multus/multus-2fqrl"
Jul 27 19:27:35.732124 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.732115    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/438069a4-ca90-4995-ba78-32af922abb36-etc-kubernetes\") pod \"multus-2fqrl\" (UID: \"438069a4-ca90-4995-ba78-32af922abb36\") " pod="openshift-multus/multus-2fqrl"
Jul 27 19:27:35.732283 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.732115    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/103bab23-bca3-472a-974b-7731f83c5bfb-node-log\") pod \"ovnkube-node-5dfc2\" (UID: \"103bab23-bca3-472a-974b-7731f83c5bfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:27:35.732283 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.732142    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e0267f96-3a9d-45d1-b943-92899defe086-host\") pod \"node-ca-qljqz\" (UID: \"e0267f96-3a9d-45d1-b943-92899defe086\") " pod="openshift-image-registry/node-ca-qljqz"
Jul 27 19:27:35.732283 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.732169    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2xd5\" (UniqueName: \"kubernetes.io/projected/229036e0-7551-4987-98d3-0ccef9c511e5-kube-api-access-r2xd5\") pod \"network-metrics-daemon-zbjtr\" (UID: \"229036e0-7551-4987-98d3-0ccef9c511e5\") " pod="openshift-multus/network-metrics-daemon-zbjtr"
Jul 27 19:27:35.732283 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.732176    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/438069a4-ca90-4995-ba78-32af922abb36-host-var-lib-kubelet\") pod \"multus-2fqrl\" (UID: \"438069a4-ca90-4995-ba78-32af922abb36\") " pod="openshift-multus/multus-2fqrl"
Jul 27 19:27:35.732283 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.732189    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/438069a4-ca90-4995-ba78-32af922abb36-os-release\") pod \"multus-2fqrl\" (UID: \"438069a4-ca90-4995-ba78-32af922abb36\") " pod="openshift-multus/multus-2fqrl"
Jul 27 19:27:35.732283 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.732207    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/438069a4-ca90-4995-ba78-32af922abb36-etc-kubernetes\") pod \"multus-2fqrl\" (UID: \"438069a4-ca90-4995-ba78-32af922abb36\") " pod="openshift-multus/multus-2fqrl"
Jul 27 19:27:35.732283 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.732219    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e0267f96-3a9d-45d1-b943-92899defe086-host\") pod \"node-ca-qljqz\" (UID: \"e0267f96-3a9d-45d1-b943-92899defe086\") " pod="openshift-image-registry/node-ca-qljqz"
Jul 27 19:27:35.732283 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.732211    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpvcw\" (UniqueName: \"kubernetes.io/projected/e9a0f4ae-3761-4764-baa1-8d8c005113a0-kube-api-access-bpvcw\") pod \"multus-additional-cni-plugins-2xk7d\" (UID: \"e9a0f4ae-3761-4764-baa1-8d8c005113a0\") " pod="openshift-multus/multus-additional-cni-plugins-2xk7d"
Jul 27 19:27:35.732283 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.732272    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/24afe7ac-4ce8-43b3-a919-1dc0a0e76fb5-etc-selinux\") pod \"aws-ebs-csi-driver-node-m4dkh\" (UID: \"24afe7ac-4ce8-43b3-a919-1dc0a0e76fb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4dkh"
Jul 27 19:27:35.732283 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.732277    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/438069a4-ca90-4995-ba78-32af922abb36-os-release\") pod \"multus-2fqrl\" (UID: \"438069a4-ca90-4995-ba78-32af922abb36\") " pod="openshift-multus/multus-2fqrl"
Jul 27 19:27:35.732812 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.732302    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4869a388-94e3-4a30-a33e-29579dcad555-etc-sysctl-d\") pod \"tuned-2cw79\" (UID: \"4869a388-94e3-4a30-a33e-29579dcad555\") " pod="openshift-cluster-node-tuning-operator/tuned-2cw79"
Jul 27 19:27:35.732812 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.732325    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e9a0f4ae-3761-4764-baa1-8d8c005113a0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2xk7d\" (UID: \"e9a0f4ae-3761-4764-baa1-8d8c005113a0\") " pod="openshift-multus/multus-additional-cni-plugins-2xk7d"
Jul 27 19:27:35.732812 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.732407    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/24afe7ac-4ce8-43b3-a919-1dc0a0e76fb5-etc-selinux\") pod \"aws-ebs-csi-driver-node-m4dkh\" (UID: \"24afe7ac-4ce8-43b3-a919-1dc0a0e76fb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4dkh"
Jul 27 19:27:35.732812 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.732460    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/103bab23-bca3-472a-974b-7731f83c5bfb-host-cni-netd\") pod \"ovnkube-node-5dfc2\" (UID: \"103bab23-bca3-472a-974b-7731f83c5bfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:27:35.732812 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.732499    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/438069a4-ca90-4995-ba78-32af922abb36-cni-binary-copy\") pod \"multus-2fqrl\" (UID: \"438069a4-ca90-4995-ba78-32af922abb36\") " pod="openshift-multus/multus-2fqrl"
Jul 27 19:27:35.732812 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.732524    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e9a0f4ae-3761-4764-baa1-8d8c005113a0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2xk7d\" (UID: \"e9a0f4ae-3761-4764-baa1-8d8c005113a0\") " pod="openshift-multus/multus-additional-cni-plugins-2xk7d"
Jul 27 19:27:35.732812 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.732536    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4869a388-94e3-4a30-a33e-29579dcad555-etc-sysctl-d\") pod \"tuned-2cw79\" (UID: \"4869a388-94e3-4a30-a33e-29579dcad555\") " pod="openshift-cluster-node-tuning-operator/tuned-2cw79"
Jul 27 19:27:35.732812 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.732564    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/438069a4-ca90-4995-ba78-32af922abb36-host-run-k8s-cni-cncf-io\") pod \"multus-2fqrl\" (UID: \"438069a4-ca90-4995-ba78-32af922abb36\") " pod="openshift-multus/multus-2fqrl"
Jul 27 19:27:35.732812 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.732545    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/103bab23-bca3-472a-974b-7731f83c5bfb-host-cni-netd\") pod \"ovnkube-node-5dfc2\" (UID: \"103bab23-bca3-472a-974b-7731f83c5bfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:27:35.732812 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.732596    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/87618c00-0367-482a-813e-9aab8f341fa5-host-slash\") pod \"iptables-alerter-6bsx5\" (UID: \"87618c00-0367-482a-813e-9aab8f341fa5\") " pod="openshift-network-operator/iptables-alerter-6bsx5"
Jul 27 19:27:35.732812 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.732611    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/438069a4-ca90-4995-ba78-32af922abb36-host-run-k8s-cni-cncf-io\") pod \"multus-2fqrl\" (UID: \"438069a4-ca90-4995-ba78-32af922abb36\") " pod="openshift-multus/multus-2fqrl"
Jul 27 19:27:35.732812 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.732626    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/87618c00-0367-482a-813e-9aab8f341fa5-host-slash\") pod \"iptables-alerter-6bsx5\" (UID: \"87618c00-0367-482a-813e-9aab8f341fa5\") " pod="openshift-network-operator/iptables-alerter-6bsx5"
Jul 27 19:27:35.732812 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.732627    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/103bab23-bca3-472a-974b-7731f83c5bfb-env-overrides\") pod \"ovnkube-node-5dfc2\" (UID: \"103bab23-bca3-472a-974b-7731f83c5bfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:27:35.732812 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.732647    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fddl\" (UniqueName: \"kubernetes.io/projected/4869a388-94e3-4a30-a33e-29579dcad555-kube-api-access-6fddl\") pod \"tuned-2cw79\" (UID: \"4869a388-94e3-4a30-a33e-29579dcad555\") " pod="openshift-cluster-node-tuning-operator/tuned-2cw79"
Jul 27 19:27:35.732812 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.732669    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/438069a4-ca90-4995-ba78-32af922abb36-cnibin\") pod \"multus-2fqrl\" (UID: \"438069a4-ca90-4995-ba78-32af922abb36\") " pod="openshift-multus/multus-2fqrl"
Jul 27 19:27:35.732812 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.732683    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/24afe7ac-4ce8-43b3-a919-1dc0a0e76fb5-registration-dir\") pod \"aws-ebs-csi-driver-node-m4dkh\" (UID: \"24afe7ac-4ce8-43b3-a919-1dc0a0e76fb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4dkh"
Jul 27 19:27:35.732812 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.732703    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3590f4c7-efa9-4c98-ae82-99afc906db15-kubelet-config\") pod \"global-pull-secret-syncer-2g8q7\" (UID: \"3590f4c7-efa9-4c98-ae82-99afc906db15\") " pod="kube-system/global-pull-secret-syncer-2g8q7"
Jul 27 19:27:35.733595 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.732723    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nx94q\" (UniqueName: \"kubernetes.io/projected/103bab23-bca3-472a-974b-7731f83c5bfb-kube-api-access-nx94q\") pod \"ovnkube-node-5dfc2\" (UID: \"103bab23-bca3-472a-974b-7731f83c5bfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:27:35.733595 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.732738    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/438069a4-ca90-4995-ba78-32af922abb36-multus-socket-dir-parent\") pod \"multus-2fqrl\" (UID: \"438069a4-ca90-4995-ba78-32af922abb36\") " pod="openshift-multus/multus-2fqrl"
Jul 27 19:27:35.733595 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.732743    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/438069a4-ca90-4995-ba78-32af922abb36-cnibin\") pod \"multus-2fqrl\" (UID: \"438069a4-ca90-4995-ba78-32af922abb36\") " pod="openshift-multus/multus-2fqrl"
Jul 27 19:27:35.733595 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.732753    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqjj9\" (UniqueName: \"kubernetes.io/projected/e0267f96-3a9d-45d1-b943-92899defe086-kube-api-access-jqjj9\") pod \"node-ca-qljqz\" (UID: \"e0267f96-3a9d-45d1-b943-92899defe086\") " pod="openshift-image-registry/node-ca-qljqz"
Jul 27 19:27:35.733595 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.732775    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e9a0f4ae-3761-4764-baa1-8d8c005113a0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2xk7d\" (UID: \"e9a0f4ae-3761-4764-baa1-8d8c005113a0\") " pod="openshift-multus/multus-additional-cni-plugins-2xk7d"
Jul 27 19:27:35.733595 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.732800    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3590f4c7-efa9-4c98-ae82-99afc906db15-dbus\") pod \"global-pull-secret-syncer-2g8q7\" (UID: \"3590f4c7-efa9-4c98-ae82-99afc906db15\") " pod="kube-system/global-pull-secret-syncer-2g8q7"
Jul 27 19:27:35.733595 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.732812    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/438069a4-ca90-4995-ba78-32af922abb36-multus-socket-dir-parent\") pod \"multus-2fqrl\" (UID: \"438069a4-ca90-4995-ba78-32af922abb36\") " pod="openshift-multus/multus-2fqrl"
Jul 27 19:27:35.733595 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.732822    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/103bab23-bca3-472a-974b-7731f83c5bfb-var-lib-openvswitch\") pod \"ovnkube-node-5dfc2\" (UID: \"103bab23-bca3-472a-974b-7731f83c5bfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:27:35.733595 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.732847    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/103bab23-bca3-472a-974b-7731f83c5bfb-run-ovn\") pod \"ovnkube-node-5dfc2\" (UID: \"103bab23-bca3-472a-974b-7731f83c5bfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:27:35.733595 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.732865    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3590f4c7-efa9-4c98-ae82-99afc906db15-kubelet-config\") pod \"global-pull-secret-syncer-2g8q7\" (UID: \"3590f4c7-efa9-4c98-ae82-99afc906db15\") " pod="kube-system/global-pull-secret-syncer-2g8q7"
Jul 27 19:27:35.733595 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.732871    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4869a388-94e3-4a30-a33e-29579dcad555-etc-sysconfig\") pod \"tuned-2cw79\" (UID: \"4869a388-94e3-4a30-a33e-29579dcad555\") " pod="openshift-cluster-node-tuning-operator/tuned-2cw79"
Jul 27 19:27:35.733595 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.732893    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4869a388-94e3-4a30-a33e-29579dcad555-sys\") pod \"tuned-2cw79\" (UID: \"4869a388-94e3-4a30-a33e-29579dcad555\") " pod="openshift-cluster-node-tuning-operator/tuned-2cw79"
Jul 27 19:27:35.733595 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.732903    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/103bab23-bca3-472a-974b-7731f83c5bfb-var-lib-openvswitch\") pod \"ovnkube-node-5dfc2\" (UID: \"103bab23-bca3-472a-974b-7731f83c5bfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:27:35.733595 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.732916    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/438069a4-ca90-4995-ba78-32af922abb36-multus-daemon-config\") pod \"multus-2fqrl\" (UID: \"438069a4-ca90-4995-ba78-32af922abb36\") " pod="openshift-multus/multus-2fqrl"
Jul 27 19:27:35.733595 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.732823    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/24afe7ac-4ce8-43b3-a919-1dc0a0e76fb5-registration-dir\") pod \"aws-ebs-csi-driver-node-m4dkh\" (UID: \"24afe7ac-4ce8-43b3-a919-1dc0a0e76fb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4dkh"
Jul 27 19:27:35.733595 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.732961    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/103bab23-bca3-472a-974b-7731f83c5bfb-run-ovn\") pod \"ovnkube-node-5dfc2\" (UID: \"103bab23-bca3-472a-974b-7731f83c5bfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:27:35.733595 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.733012    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4869a388-94e3-4a30-a33e-29579dcad555-sys\") pod \"tuned-2cw79\" (UID: \"4869a388-94e3-4a30-a33e-29579dcad555\") " pod="openshift-cluster-node-tuning-operator/tuned-2cw79"
Jul 27 19:27:35.733595 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.733017    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3590f4c7-efa9-4c98-ae82-99afc906db15-dbus\") pod \"global-pull-secret-syncer-2g8q7\" (UID: \"3590f4c7-efa9-4c98-ae82-99afc906db15\") " pod="kube-system/global-pull-secret-syncer-2g8q7"
Jul 27 19:27:35.735224 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.733009    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/438069a4-ca90-4995-ba78-32af922abb36-cni-binary-copy\") pod \"multus-2fqrl\" (UID: \"438069a4-ca90-4995-ba78-32af922abb36\") " pod="openshift-multus/multus-2fqrl"
Jul 27 19:27:35.735224 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.733018    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4869a388-94e3-4a30-a33e-29579dcad555-etc-sysconfig\") pod \"tuned-2cw79\" (UID: \"4869a388-94e3-4a30-a33e-29579dcad555\") " pod="openshift-cluster-node-tuning-operator/tuned-2cw79"
Jul 27 19:27:35.735224 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.733063    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e9a0f4ae-3761-4764-baa1-8d8c005113a0-cni-binary-copy\") pod \"multus-additional-cni-plugins-2xk7d\" (UID: \"e9a0f4ae-3761-4764-baa1-8d8c005113a0\") " pod="openshift-multus/multus-additional-cni-plugins-2xk7d"
Jul 27 19:27:35.735224 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.733088    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e9a0f4ae-3761-4764-baa1-8d8c005113a0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2xk7d\" (UID: \"e9a0f4ae-3761-4764-baa1-8d8c005113a0\") " pod="openshift-multus/multus-additional-cni-plugins-2xk7d"
Jul 27 19:27:35.735224 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.733125    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/438069a4-ca90-4995-ba78-32af922abb36-system-cni-dir\") pod \"multus-2fqrl\" (UID: \"438069a4-ca90-4995-ba78-32af922abb36\") " pod="openshift-multus/multus-2fqrl"
Jul 27 19:27:35.735224 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.733140    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/103bab23-bca3-472a-974b-7731f83c5bfb-systemd-units\") pod \"ovnkube-node-5dfc2\" (UID: \"103bab23-bca3-472a-974b-7731f83c5bfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:27:35.735224 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.733154    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/103bab23-bca3-472a-974b-7731f83c5bfb-host-run-ovn-kubernetes\") pod \"ovnkube-node-5dfc2\" (UID: \"103bab23-bca3-472a-974b-7731f83c5bfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:27:35.735224 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.733170    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/103bab23-bca3-472a-974b-7731f83c5bfb-host-cni-bin\") pod \"ovnkube-node-5dfc2\" (UID: \"103bab23-bca3-472a-974b-7731f83c5bfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:27:35.735224 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.733195    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/103bab23-bca3-472a-974b-7731f83c5bfb-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5dfc2\" (UID: \"103bab23-bca3-472a-974b-7731f83c5bfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:27:35.735224 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.733219    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/103bab23-bca3-472a-974b-7731f83c5bfb-systemd-units\") pod \"ovnkube-node-5dfc2\" (UID: \"103bab23-bca3-472a-974b-7731f83c5bfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:27:35.735224 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.733223    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e9a0f4ae-3761-4764-baa1-8d8c005113a0-cnibin\") pod \"multus-additional-cni-plugins-2xk7d\" (UID: \"e9a0f4ae-3761-4764-baa1-8d8c005113a0\") " pod="openshift-multus/multus-additional-cni-plugins-2xk7d"
Jul 27 19:27:35.735224 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.733226    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/438069a4-ca90-4995-ba78-32af922abb36-system-cni-dir\") pod \"multus-2fqrl\" (UID: \"438069a4-ca90-4995-ba78-32af922abb36\") " pod="openshift-multus/multus-2fqrl"
Jul 27 19:27:35.735224 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.733232    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/103bab23-bca3-472a-974b-7731f83c5bfb-host-run-ovn-kubernetes\") pod \"ovnkube-node-5dfc2\" (UID: \"103bab23-bca3-472a-974b-7731f83c5bfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:27:35.735224 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.733260    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/103bab23-bca3-472a-974b-7731f83c5bfb-log-socket\") pod \"ovnkube-node-5dfc2\" (UID: \"103bab23-bca3-472a-974b-7731f83c5bfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:27:35.735224 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.733266    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/103bab23-bca3-472a-974b-7731f83c5bfb-host-cni-bin\") pod \"ovnkube-node-5dfc2\" (UID: \"103bab23-bca3-472a-974b-7731f83c5bfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:27:35.735224 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.733281    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e9a0f4ae-3761-4764-baa1-8d8c005113a0-cnibin\") pod \"multus-additional-cni-plugins-2xk7d\" (UID: \"e9a0f4ae-3761-4764-baa1-8d8c005113a0\") " pod="openshift-multus/multus-additional-cni-plugins-2xk7d"
Jul 27 19:27:35.735224 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.733293    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4869a388-94e3-4a30-a33e-29579dcad555-etc-sysctl-conf\") pod \"tuned-2cw79\" (UID: \"4869a388-94e3-4a30-a33e-29579dcad555\") " pod="openshift-cluster-node-tuning-operator/tuned-2cw79"
Jul 27 19:27:35.736183 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.733298    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/103bab23-bca3-472a-974b-7731f83c5bfb-log-socket\") pod \"ovnkube-node-5dfc2\" (UID: \"103bab23-bca3-472a-974b-7731f83c5bfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:27:35.736183 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.733293    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e9a0f4ae-3761-4764-baa1-8d8c005113a0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2xk7d\" (UID: \"e9a0f4ae-3761-4764-baa1-8d8c005113a0\") " pod="openshift-multus/multus-additional-cni-plugins-2xk7d"
Jul 27 19:27:35.736183 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.733326    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/103bab23-bca3-472a-974b-7731f83c5bfb-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5dfc2\" (UID: \"103bab23-bca3-472a-974b-7731f83c5bfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:27:35.736183 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.733317    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4869a388-94e3-4a30-a33e-29579dcad555-lib-modules\") pod \"tuned-2cw79\" (UID: \"4869a388-94e3-4a30-a33e-29579dcad555\") " pod="openshift-cluster-node-tuning-operator/tuned-2cw79"
Jul 27 19:27:35.736183 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.733363    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/438069a4-ca90-4995-ba78-32af922abb36-multus-cni-dir\") pod \"multus-2fqrl\" (UID: \"438069a4-ca90-4995-ba78-32af922abb36\") " pod="openshift-multus/multus-2fqrl"
Jul 27 19:27:35.736183 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.733378    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/24afe7ac-4ce8-43b3-a919-1dc0a0e76fb5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-m4dkh\" (UID: \"24afe7ac-4ce8-43b3-a919-1dc0a0e76fb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4dkh"
Jul 27 19:27:35.736183 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.733403    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/229036e0-7551-4987-98d3-0ccef9c511e5-metrics-certs\") pod \"network-metrics-daemon-zbjtr\" (UID: \"229036e0-7551-4987-98d3-0ccef9c511e5\") " pod="openshift-multus/network-metrics-daemon-zbjtr"
Jul 27 19:27:35.736183 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.733443    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/103bab23-bca3-472a-974b-7731f83c5bfb-etc-openvswitch\") pod \"ovnkube-node-5dfc2\" (UID: \"103bab23-bca3-472a-974b-7731f83c5bfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:27:35.736183 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.733458    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/438069a4-ca90-4995-ba78-32af922abb36-multus-cni-dir\") pod \"multus-2fqrl\" (UID: \"438069a4-ca90-4995-ba78-32af922abb36\") " pod="openshift-multus/multus-2fqrl"
Jul 27 19:27:35.736183 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.733497    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4869a388-94e3-4a30-a33e-29579dcad555-etc-systemd\") pod \"tuned-2cw79\" (UID: \"4869a388-94e3-4a30-a33e-29579dcad555\") " pod="openshift-cluster-node-tuning-operator/tuned-2cw79"
Jul 27 19:27:35.736183 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.733511    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/24afe7ac-4ce8-43b3-a919-1dc0a0e76fb5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-m4dkh\" (UID: \"24afe7ac-4ce8-43b3-a919-1dc0a0e76fb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4dkh"
Jul 27 19:27:35.736183 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.733444    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4869a388-94e3-4a30-a33e-29579dcad555-etc-sysctl-conf\") pod \"tuned-2cw79\" (UID: \"4869a388-94e3-4a30-a33e-29579dcad555\") " pod="openshift-cluster-node-tuning-operator/tuned-2cw79"
Jul 27 19:27:35.736183 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:35.733526    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 19:27:35.736183 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.733530    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4869a388-94e3-4a30-a33e-29579dcad555-lib-modules\") pod \"tuned-2cw79\" (UID: \"4869a388-94e3-4a30-a33e-29579dcad555\") " pod="openshift-cluster-node-tuning-operator/tuned-2cw79"
Jul 27 19:27:35.736183 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.733562    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/103bab23-bca3-472a-974b-7731f83c5bfb-etc-openvswitch\") pod \"ovnkube-node-5dfc2\" (UID: \"103bab23-bca3-472a-974b-7731f83c5bfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:27:35.736183 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.733532    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/438069a4-ca90-4995-ba78-32af922abb36-multus-daemon-config\") pod \"multus-2fqrl\" (UID: \"438069a4-ca90-4995-ba78-32af922abb36\") " pod="openshift-multus/multus-2fqrl"
Jul 27 19:27:35.736183 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.733521    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4869a388-94e3-4a30-a33e-29579dcad555-etc-tuned\") pod \"tuned-2cw79\" (UID: \"4869a388-94e3-4a30-a33e-29579dcad555\") " pod="openshift-cluster-node-tuning-operator/tuned-2cw79"
Jul 27 19:27:35.737005 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:35.733590    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/229036e0-7551-4987-98d3-0ccef9c511e5-metrics-certs podName:229036e0-7551-4987-98d3-0ccef9c511e5 nodeName:}" failed. No retries permitted until 2026-07-27 19:27:36.233572574 +0000 UTC m=+69.105764527 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/229036e0-7551-4987-98d3-0ccef9c511e5-metrics-certs") pod "network-metrics-daemon-zbjtr" (UID: "229036e0-7551-4987-98d3-0ccef9c511e5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 19:27:35.737005 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.733586    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4869a388-94e3-4a30-a33e-29579dcad555-etc-systemd\") pod \"tuned-2cw79\" (UID: \"4869a388-94e3-4a30-a33e-29579dcad555\") " pod="openshift-cluster-node-tuning-operator/tuned-2cw79"
Jul 27 19:27:35.737005 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.733664    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4869a388-94e3-4a30-a33e-29579dcad555-tmp\") pod \"tuned-2cw79\" (UID: \"4869a388-94e3-4a30-a33e-29579dcad555\") " pod="openshift-cluster-node-tuning-operator/tuned-2cw79"
Jul 27 19:27:35.737005 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.733692    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/438069a4-ca90-4995-ba78-32af922abb36-host-run-multus-certs\") pod \"multus-2fqrl\" (UID: \"438069a4-ca90-4995-ba78-32af922abb36\") " pod="openshift-multus/multus-2fqrl"
Jul 27 19:27:35.737005 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.733703    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e9a0f4ae-3761-4764-baa1-8d8c005113a0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2xk7d\" (UID: \"e9a0f4ae-3761-4764-baa1-8d8c005113a0\") " pod="openshift-multus/multus-additional-cni-plugins-2xk7d"
Jul 27 19:27:35.737005 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.733717    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqjr4\" (UniqueName: \"kubernetes.io/projected/d48f4d00-ad6d-4b8e-9a57-de8ed6729c98-kube-api-access-gqjr4\") pod \"node-resolver-bnrsg\" (UID: \"d48f4d00-ad6d-4b8e-9a57-de8ed6729c98\") " pod="openshift-dns/node-resolver-bnrsg"
Jul 27 19:27:35.737005 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.733740    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7719045e-d600-437b-aa76-7f03b29fecd9-agent-certs\") pod \"konnectivity-agent-5zsvw\" (UID: \"7719045e-d600-437b-aa76-7f03b29fecd9\") " pod="kube-system/konnectivity-agent-5zsvw"
Jul 27 19:27:35.737005 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.733777    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/438069a4-ca90-4995-ba78-32af922abb36-host-run-multus-certs\") pod \"multus-2fqrl\" (UID: \"438069a4-ca90-4995-ba78-32af922abb36\") " pod="openshift-multus/multus-2fqrl"
Jul 27 19:27:35.737005 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.733776    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3590f4c7-efa9-4c98-ae82-99afc906db15-original-pull-secret\") pod \"global-pull-secret-syncer-2g8q7\" (UID: \"3590f4c7-efa9-4c98-ae82-99afc906db15\") " pod="kube-system/global-pull-secret-syncer-2g8q7"
Jul 27 19:27:35.737005 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.733810    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/103bab23-bca3-472a-974b-7731f83c5bfb-ovnkube-script-lib\") pod \"ovnkube-node-5dfc2\" (UID: \"103bab23-bca3-472a-974b-7731f83c5bfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:27:35.737005 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.733828    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/103bab23-bca3-472a-974b-7731f83c5bfb-host-kubelet\") pod \"ovnkube-node-5dfc2\" (UID: \"103bab23-bca3-472a-974b-7731f83c5bfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:27:35.737005 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:35.733837    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 19:27:35.737005 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.733850    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9llnv\" (UniqueName: \"kubernetes.io/projected/87618c00-0367-482a-813e-9aab8f341fa5-kube-api-access-9llnv\") pod \"iptables-alerter-6bsx5\" (UID: \"87618c00-0367-482a-813e-9aab8f341fa5\") " pod="openshift-network-operator/iptables-alerter-6bsx5"
Jul 27 19:27:35.737005 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.733867    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d48f4d00-ad6d-4b8e-9a57-de8ed6729c98-hosts-file\") pod \"node-resolver-bnrsg\" (UID: \"d48f4d00-ad6d-4b8e-9a57-de8ed6729c98\") " pod="openshift-dns/node-resolver-bnrsg"
Jul 27 19:27:35.737005 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:35.733878    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3590f4c7-efa9-4c98-ae82-99afc906db15-original-pull-secret podName:3590f4c7-efa9-4c98-ae82-99afc906db15 nodeName:}" failed. No retries permitted until 2026-07-27 19:27:36.233865039 +0000 UTC m=+69.106057006 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3590f4c7-efa9-4c98-ae82-99afc906db15-original-pull-secret") pod "global-pull-secret-syncer-2g8q7" (UID: "3590f4c7-efa9-4c98-ae82-99afc906db15") : object "kube-system"/"original-pull-secret" not registered
Jul 27 19:27:35.737005 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.733875    2558 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 27 19:27:35.737005 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.733902    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d48f4d00-ad6d-4b8e-9a57-de8ed6729c98-hosts-file\") pod \"node-resolver-bnrsg\" (UID: \"d48f4d00-ad6d-4b8e-9a57-de8ed6729c98\") " pod="openshift-dns/node-resolver-bnrsg"
Jul 27 19:27:35.737798 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.733904    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e0267f96-3a9d-45d1-b943-92899defe086-serviceca\") pod \"node-ca-qljqz\" (UID: \"e0267f96-3a9d-45d1-b943-92899defe086\") " pod="openshift-image-registry/node-ca-qljqz"
Jul 27 19:27:35.737798 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.733930    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7719045e-d600-437b-aa76-7f03b29fecd9-konnectivity-ca\") pod \"konnectivity-agent-5zsvw\" (UID: \"7719045e-d600-437b-aa76-7f03b29fecd9\") " pod="kube-system/konnectivity-agent-5zsvw"
Jul 27 19:27:35.737798 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.733942    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/103bab23-bca3-472a-974b-7731f83c5bfb-host-kubelet\") pod \"ovnkube-node-5dfc2\" (UID: \"103bab23-bca3-472a-974b-7731f83c5bfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:27:35.737798 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.733954    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4869a388-94e3-4a30-a33e-29579dcad555-run\") pod \"tuned-2cw79\" (UID: \"4869a388-94e3-4a30-a33e-29579dcad555\") " pod="openshift-cluster-node-tuning-operator/tuned-2cw79"
Jul 27 19:27:35.737798 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.733988    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vpzg\" (UniqueName: \"kubernetes.io/projected/438069a4-ca90-4995-ba78-32af922abb36-kube-api-access-7vpzg\") pod \"multus-2fqrl\" (UID: \"438069a4-ca90-4995-ba78-32af922abb36\") " pod="openshift-multus/multus-2fqrl"
Jul 27 19:27:35.737798 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.734022    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4869a388-94e3-4a30-a33e-29579dcad555-etc-kubernetes\") pod \"tuned-2cw79\" (UID: \"4869a388-94e3-4a30-a33e-29579dcad555\") " pod="openshift-cluster-node-tuning-operator/tuned-2cw79"
Jul 27 19:27:35.737798 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.734073    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/438069a4-ca90-4995-ba78-32af922abb36-host-run-netns\") pod \"multus-2fqrl\" (UID: \"438069a4-ca90-4995-ba78-32af922abb36\") " pod="openshift-multus/multus-2fqrl"
Jul 27 19:27:35.737798 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.733993    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4869a388-94e3-4a30-a33e-29579dcad555-run\") pod \"tuned-2cw79\" (UID: \"4869a388-94e3-4a30-a33e-29579dcad555\") " pod="openshift-cluster-node-tuning-operator/tuned-2cw79"
Jul 27 19:27:35.737798 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.734097    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/438069a4-ca90-4995-ba78-32af922abb36-multus-conf-dir\") pod \"multus-2fqrl\" (UID: \"438069a4-ca90-4995-ba78-32af922abb36\") " pod="openshift-multus/multus-2fqrl"
Jul 27 19:27:35.737798 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.734110    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e9a0f4ae-3761-4764-baa1-8d8c005113a0-cni-binary-copy\") pod \"multus-additional-cni-plugins-2xk7d\" (UID: \"e9a0f4ae-3761-4764-baa1-8d8c005113a0\") " pod="openshift-multus/multus-additional-cni-plugins-2xk7d"
Jul 27 19:27:35.737798 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.734127    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmgsm\" (UniqueName: \"kubernetes.io/projected/88050dfb-f434-42bd-8c5f-9dc6f98673e0-kube-api-access-wmgsm\") pod \"network-check-target-2vtnn\" (UID: \"88050dfb-f434-42bd-8c5f-9dc6f98673e0\") " pod="openshift-network-diagnostics/network-check-target-2vtnn"
Jul 27 19:27:35.737798 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.734153    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hr79d\" (UniqueName: \"kubernetes.io/projected/24afe7ac-4ce8-43b3-a919-1dc0a0e76fb5-kube-api-access-hr79d\") pod \"aws-ebs-csi-driver-node-m4dkh\" (UID: \"24afe7ac-4ce8-43b3-a919-1dc0a0e76fb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4dkh"
Jul 27 19:27:35.737798 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.734174    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/438069a4-ca90-4995-ba78-32af922abb36-host-run-netns\") pod \"multus-2fqrl\" (UID: \"438069a4-ca90-4995-ba78-32af922abb36\") " pod="openshift-multus/multus-2fqrl"
Jul 27 19:27:35.737798 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.734179    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/103bab23-bca3-472a-974b-7731f83c5bfb-host-run-netns\") pod \"ovnkube-node-5dfc2\" (UID: \"103bab23-bca3-472a-974b-7731f83c5bfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:27:35.737798 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.734202    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/103bab23-bca3-472a-974b-7731f83c5bfb-ovnkube-config\") pod \"ovnkube-node-5dfc2\" (UID: \"103bab23-bca3-472a-974b-7731f83c5bfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:27:35.737798 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.734221    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4869a388-94e3-4a30-a33e-29579dcad555-etc-kubernetes\") pod \"tuned-2cw79\" (UID: \"4869a388-94e3-4a30-a33e-29579dcad555\") " pod="openshift-cluster-node-tuning-operator/tuned-2cw79"
Jul 27 19:27:35.737798 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.734244    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/438069a4-ca90-4995-ba78-32af922abb36-host-var-lib-cni-bin\") pod \"multus-2fqrl\" (UID: \"438069a4-ca90-4995-ba78-32af922abb36\") " pod="openshift-multus/multus-2fqrl"
Jul 27 19:27:35.737798 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.734269    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/438069a4-ca90-4995-ba78-32af922abb36-host-var-lib-cni-multus\") pod \"multus-2fqrl\" (UID: \"438069a4-ca90-4995-ba78-32af922abb36\") " pod="openshift-multus/multus-2fqrl"
Jul 27 19:27:35.738322 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.734293    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/438069a4-ca90-4995-ba78-32af922abb36-hostroot\") pod \"multus-2fqrl\" (UID: \"438069a4-ca90-4995-ba78-32af922abb36\") " pod="openshift-multus/multus-2fqrl"
Jul 27 19:27:35.738322 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.734312    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/438069a4-ca90-4995-ba78-32af922abb36-multus-conf-dir\") pod \"multus-2fqrl\" (UID: \"438069a4-ca90-4995-ba78-32af922abb36\") " pod="openshift-multus/multus-2fqrl"
Jul 27 19:27:35.738322 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.734356    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/103bab23-bca3-472a-974b-7731f83c5bfb-host-run-netns\") pod \"ovnkube-node-5dfc2\" (UID: \"103bab23-bca3-472a-974b-7731f83c5bfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:27:35.738322 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.734389    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e0267f96-3a9d-45d1-b943-92899defe086-serviceca\") pod \"node-ca-qljqz\" (UID: \"e0267f96-3a9d-45d1-b943-92899defe086\") " pod="openshift-image-registry/node-ca-qljqz"
Jul 27 19:27:35.738322 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.734448    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/24afe7ac-4ce8-43b3-a919-1dc0a0e76fb5-socket-dir\") pod \"aws-ebs-csi-driver-node-m4dkh\" (UID: \"24afe7ac-4ce8-43b3-a919-1dc0a0e76fb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4dkh"
Jul 27 19:27:35.738322 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.734316    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/24afe7ac-4ce8-43b3-a919-1dc0a0e76fb5-socket-dir\") pod \"aws-ebs-csi-driver-node-m4dkh\" (UID: \"24afe7ac-4ce8-43b3-a919-1dc0a0e76fb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4dkh"
Jul 27 19:27:35.738322 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.734465    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/438069a4-ca90-4995-ba78-32af922abb36-host-var-lib-cni-multus\") pod \"multus-2fqrl\" (UID: \"438069a4-ca90-4995-ba78-32af922abb36\") " pod="openshift-multus/multus-2fqrl"
Jul 27 19:27:35.738322 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.734492    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/103bab23-bca3-472a-974b-7731f83c5bfb-host-slash\") pod \"ovnkube-node-5dfc2\" (UID: \"103bab23-bca3-472a-974b-7731f83c5bfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:27:35.738322 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.734502    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/438069a4-ca90-4995-ba78-32af922abb36-host-var-lib-cni-bin\") pod \"multus-2fqrl\" (UID: \"438069a4-ca90-4995-ba78-32af922abb36\") " pod="openshift-multus/multus-2fqrl"
Jul 27 19:27:35.738322 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.734510    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7719045e-d600-437b-aa76-7f03b29fecd9-konnectivity-ca\") pod \"konnectivity-agent-5zsvw\" (UID: \"7719045e-d600-437b-aa76-7f03b29fecd9\") " pod="kube-system/konnectivity-agent-5zsvw"
Jul 27 19:27:35.738322 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.734516    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4869a388-94e3-4a30-a33e-29579dcad555-etc-modprobe-d\") pod \"tuned-2cw79\" (UID: \"4869a388-94e3-4a30-a33e-29579dcad555\") " pod="openshift-cluster-node-tuning-operator/tuned-2cw79"
Jul 27 19:27:35.738322 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.734546    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/438069a4-ca90-4995-ba78-32af922abb36-hostroot\") pod \"multus-2fqrl\" (UID: \"438069a4-ca90-4995-ba78-32af922abb36\") " pod="openshift-multus/multus-2fqrl"
Jul 27 19:27:35.738322 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.734560    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/87618c00-0367-482a-813e-9aab8f341fa5-iptables-alerter-script\") pod \"iptables-alerter-6bsx5\" (UID: \"87618c00-0367-482a-813e-9aab8f341fa5\") " pod="openshift-network-operator/iptables-alerter-6bsx5"
Jul 27 19:27:35.738322 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.734577    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/103bab23-bca3-472a-974b-7731f83c5bfb-host-slash\") pod \"ovnkube-node-5dfc2\" (UID: \"103bab23-bca3-472a-974b-7731f83c5bfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:27:35.738322 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.734581    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4869a388-94e3-4a30-a33e-29579dcad555-etc-modprobe-d\") pod \"tuned-2cw79\" (UID: \"4869a388-94e3-4a30-a33e-29579dcad555\") " pod="openshift-cluster-node-tuning-operator/tuned-2cw79"
Jul 27 19:27:35.738322 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.734585    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d48f4d00-ad6d-4b8e-9a57-de8ed6729c98-tmp-dir\") pod \"node-resolver-bnrsg\" (UID: \"d48f4d00-ad6d-4b8e-9a57-de8ed6729c98\") " pod="openshift-dns/node-resolver-bnrsg"
Jul 27 19:27:35.738322 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.734613    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e9a0f4ae-3761-4764-baa1-8d8c005113a0-os-release\") pod \"multus-additional-cni-plugins-2xk7d\" (UID: \"e9a0f4ae-3761-4764-baa1-8d8c005113a0\") " pod="openshift-multus/multus-additional-cni-plugins-2xk7d"
Jul 27 19:27:35.738322 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.734639    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/24afe7ac-4ce8-43b3-a919-1dc0a0e76fb5-device-dir\") pod \"aws-ebs-csi-driver-node-m4dkh\" (UID: \"24afe7ac-4ce8-43b3-a919-1dc0a0e76fb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4dkh"
Jul 27 19:27:35.738995 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.734659    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/24afe7ac-4ce8-43b3-a919-1dc0a0e76fb5-sys-fs\") pod \"aws-ebs-csi-driver-node-m4dkh\" (UID: \"24afe7ac-4ce8-43b3-a919-1dc0a0e76fb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4dkh"
Jul 27 19:27:35.738995 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.734683    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/103bab23-bca3-472a-974b-7731f83c5bfb-run-openvswitch\") pod \"ovnkube-node-5dfc2\" (UID: \"103bab23-bca3-472a-974b-7731f83c5bfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:27:35.738995 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.734702    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/103bab23-bca3-472a-974b-7731f83c5bfb-ovn-node-metrics-cert\") pod \"ovnkube-node-5dfc2\" (UID: \"103bab23-bca3-472a-974b-7731f83c5bfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:27:35.738995 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.734714    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/24afe7ac-4ce8-43b3-a919-1dc0a0e76fb5-device-dir\") pod \"aws-ebs-csi-driver-node-m4dkh\" (UID: \"24afe7ac-4ce8-43b3-a919-1dc0a0e76fb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4dkh"
Jul 27 19:27:35.738995 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.734721    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4869a388-94e3-4a30-a33e-29579dcad555-var-lib-kubelet\") pod \"tuned-2cw79\" (UID: \"4869a388-94e3-4a30-a33e-29579dcad555\") " pod="openshift-cluster-node-tuning-operator/tuned-2cw79"
Jul 27 19:27:35.738995 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.734721    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e9a0f4ae-3761-4764-baa1-8d8c005113a0-os-release\") pod \"multus-additional-cni-plugins-2xk7d\" (UID: \"e9a0f4ae-3761-4764-baa1-8d8c005113a0\") " pod="openshift-multus/multus-additional-cni-plugins-2xk7d"
Jul 27 19:27:35.738995 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.734741    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4869a388-94e3-4a30-a33e-29579dcad555-host\") pod \"tuned-2cw79\" (UID: \"4869a388-94e3-4a30-a33e-29579dcad555\") " pod="openshift-cluster-node-tuning-operator/tuned-2cw79"
Jul 27 19:27:35.738995 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.734779    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/103bab23-bca3-472a-974b-7731f83c5bfb-run-openvswitch\") pod \"ovnkube-node-5dfc2\" (UID: \"103bab23-bca3-472a-974b-7731f83c5bfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:27:35.738995 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.734790    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4869a388-94e3-4a30-a33e-29579dcad555-host\") pod \"tuned-2cw79\" (UID: \"4869a388-94e3-4a30-a33e-29579dcad555\") " pod="openshift-cluster-node-tuning-operator/tuned-2cw79"
Jul 27 19:27:35.738995 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.734808    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9a0f4ae-3761-4764-baa1-8d8c005113a0-system-cni-dir\") pod \"multus-additional-cni-plugins-2xk7d\" (UID: \"e9a0f4ae-3761-4764-baa1-8d8c005113a0\") " pod="openshift-multus/multus-additional-cni-plugins-2xk7d"
Jul 27 19:27:35.738995 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.734834    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/103bab23-bca3-472a-974b-7731f83c5bfb-run-systemd\") pod \"ovnkube-node-5dfc2\" (UID: \"103bab23-bca3-472a-974b-7731f83c5bfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:27:35.738995 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.734838    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/103bab23-bca3-472a-974b-7731f83c5bfb-ovnkube-config\") pod \"ovnkube-node-5dfc2\" (UID: \"103bab23-bca3-472a-974b-7731f83c5bfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:27:35.738995 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.734898    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4869a388-94e3-4a30-a33e-29579dcad555-var-lib-kubelet\") pod \"tuned-2cw79\" (UID: \"4869a388-94e3-4a30-a33e-29579dcad555\") " pod="openshift-cluster-node-tuning-operator/tuned-2cw79"
Jul 27 19:27:35.738995 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.734948    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/24afe7ac-4ce8-43b3-a919-1dc0a0e76fb5-sys-fs\") pod \"aws-ebs-csi-driver-node-m4dkh\" (UID: \"24afe7ac-4ce8-43b3-a919-1dc0a0e76fb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4dkh"
Jul 27 19:27:35.738995 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.734963    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d48f4d00-ad6d-4b8e-9a57-de8ed6729c98-tmp-dir\") pod \"node-resolver-bnrsg\" (UID: \"d48f4d00-ad6d-4b8e-9a57-de8ed6729c98\") " pod="openshift-dns/node-resolver-bnrsg"
Jul 27 19:27:35.738995 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.734969    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9a0f4ae-3761-4764-baa1-8d8c005113a0-system-cni-dir\") pod \"multus-additional-cni-plugins-2xk7d\" (UID: \"e9a0f4ae-3761-4764-baa1-8d8c005113a0\") " pod="openshift-multus/multus-additional-cni-plugins-2xk7d"
Jul 27 19:27:35.738995 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.734987    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/103bab23-bca3-472a-974b-7731f83c5bfb-run-systemd\") pod \"ovnkube-node-5dfc2\" (UID: \"103bab23-bca3-472a-974b-7731f83c5bfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:27:35.740028 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.735671    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/87618c00-0367-482a-813e-9aab8f341fa5-iptables-alerter-script\") pod \"iptables-alerter-6bsx5\" (UID: \"87618c00-0367-482a-813e-9aab8f341fa5\") " pod="openshift-network-operator/iptables-alerter-6bsx5"
Jul 27 19:27:35.740028 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.737782    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4869a388-94e3-4a30-a33e-29579dcad555-etc-tuned\") pod \"tuned-2cw79\" (UID: \"4869a388-94e3-4a30-a33e-29579dcad555\") " pod="openshift-cluster-node-tuning-operator/tuned-2cw79"
Jul 27 19:27:35.740028 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.737802    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4869a388-94e3-4a30-a33e-29579dcad555-tmp\") pod \"tuned-2cw79\" (UID: \"4869a388-94e3-4a30-a33e-29579dcad555\") " pod="openshift-cluster-node-tuning-operator/tuned-2cw79"
Jul 27 19:27:35.740028 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.737972    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7719045e-d600-437b-aa76-7f03b29fecd9-agent-certs\") pod \"konnectivity-agent-5zsvw\" (UID: \"7719045e-d600-437b-aa76-7f03b29fecd9\") " pod="kube-system/konnectivity-agent-5zsvw"
Jul 27 19:27:35.740028 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.738600    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/103bab23-bca3-472a-974b-7731f83c5bfb-ovn-node-metrics-cert\") pod \"ovnkube-node-5dfc2\" (UID: \"103bab23-bca3-472a-974b-7731f83c5bfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:27:35.740610 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:35.740399    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 19:27:35.740610 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:35.740446    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 19:27:35.740610 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:35.740462    2558 projected.go:196] Error preparing data for projected volume kube-api-access-wmgsm for pod openshift-network-diagnostics/network-check-target-2vtnn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 19:27:35.740610 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:35.740515    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/88050dfb-f434-42bd-8c5f-9dc6f98673e0-kube-api-access-wmgsm podName:88050dfb-f434-42bd-8c5f-9dc6f98673e0 nodeName:}" failed. No retries permitted until 2026-07-27 19:27:36.240499249 +0000 UTC m=+69.112691217 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-wmgsm" (UniqueName: "kubernetes.io/projected/88050dfb-f434-42bd-8c5f-9dc6f98673e0-kube-api-access-wmgsm") pod "network-check-target-2vtnn" (UID: "88050dfb-f434-42bd-8c5f-9dc6f98673e0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 19:27:35.745246 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.741500    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fddl\" (UniqueName: \"kubernetes.io/projected/4869a388-94e3-4a30-a33e-29579dcad555-kube-api-access-6fddl\") pod \"tuned-2cw79\" (UID: \"4869a388-94e3-4a30-a33e-29579dcad555\") " pod="openshift-cluster-node-tuning-operator/tuned-2cw79"
Jul 27 19:27:35.745246 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.741524    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqjj9\" (UniqueName: \"kubernetes.io/projected/e0267f96-3a9d-45d1-b943-92899defe086-kube-api-access-jqjj9\") pod \"node-ca-qljqz\" (UID: \"e0267f96-3a9d-45d1-b943-92899defe086\") " pod="openshift-image-registry/node-ca-qljqz"
Jul 27 19:27:35.745246 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.741993    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpvcw\" (UniqueName: \"kubernetes.io/projected/e9a0f4ae-3761-4764-baa1-8d8c005113a0-kube-api-access-bpvcw\") pod \"multus-additional-cni-plugins-2xk7d\" (UID: \"e9a0f4ae-3761-4764-baa1-8d8c005113a0\") " pod="openshift-multus/multus-additional-cni-plugins-2xk7d"
Jul 27 19:27:35.745246 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.742444    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2xd5\" (UniqueName: \"kubernetes.io/projected/229036e0-7551-4987-98d3-0ccef9c511e5-kube-api-access-r2xd5\") pod \"network-metrics-daemon-zbjtr\" (UID: \"229036e0-7551-4987-98d3-0ccef9c511e5\") " pod="openshift-multus/network-metrics-daemon-zbjtr"
Jul 27 19:27:35.745246 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.743012    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nx94q\" (UniqueName: \"kubernetes.io/projected/103bab23-bca3-472a-974b-7731f83c5bfb-kube-api-access-nx94q\") pod \"ovnkube-node-5dfc2\" (UID: \"103bab23-bca3-472a-974b-7731f83c5bfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:27:35.745246 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.743082    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqjr4\" (UniqueName: \"kubernetes.io/projected/d48f4d00-ad6d-4b8e-9a57-de8ed6729c98-kube-api-access-gqjr4\") pod \"node-resolver-bnrsg\" (UID: \"d48f4d00-ad6d-4b8e-9a57-de8ed6729c98\") " pod="openshift-dns/node-resolver-bnrsg"
Jul 27 19:27:35.745246 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.745079    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9llnv\" (UniqueName: \"kubernetes.io/projected/87618c00-0367-482a-813e-9aab8f341fa5-kube-api-access-9llnv\") pod \"iptables-alerter-6bsx5\" (UID: \"87618c00-0367-482a-813e-9aab8f341fa5\") " pod="openshift-network-operator/iptables-alerter-6bsx5"
Jul 27 19:27:35.746120 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.746103    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hr79d\" (UniqueName: \"kubernetes.io/projected/24afe7ac-4ce8-43b3-a919-1dc0a0e76fb5-kube-api-access-hr79d\") pod \"aws-ebs-csi-driver-node-m4dkh\" (UID: \"24afe7ac-4ce8-43b3-a919-1dc0a0e76fb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4dkh"
Jul 27 19:27:35.747041 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.747026    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vpzg\" (UniqueName: \"kubernetes.io/projected/438069a4-ca90-4995-ba78-32af922abb36-kube-api-access-7vpzg\") pod \"multus-2fqrl\" (UID: \"438069a4-ca90-4995-ba78-32af922abb36\") " pod="openshift-multus/multus-2fqrl"
Jul 27 19:27:35.809568 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.809538    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2xk7d"
Jul 27 19:27:35.814887 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.814867    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4dkh"
Jul 27 19:27:35.820226 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.820201    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-2cw79"
Jul 27 19:27:35.822583 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:27:35.822557    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9a0f4ae_3761_4764_baa1_8d8c005113a0.slice/crio-d62d4ff6a992feff9e14914519df9171111416526adc43db7911e39ee5f8c39b WatchSource:0}: Error finding container d62d4ff6a992feff9e14914519df9171111416526adc43db7911e39ee5f8c39b: Status 404 returned error can't find the container with id d62d4ff6a992feff9e14914519df9171111416526adc43db7911e39ee5f8c39b
Jul 27 19:27:35.824560 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:27:35.824539    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24afe7ac_4ce8_43b3_a919_1dc0a0e76fb5.slice/crio-90ca8ed799e6ea6c076ff32b698cdbdaab5dd6997c2963a31c24cc04e3c08534 WatchSource:0}: Error finding container 90ca8ed799e6ea6c076ff32b698cdbdaab5dd6997c2963a31c24cc04e3c08534: Status 404 returned error can't find the container with id 90ca8ed799e6ea6c076ff32b698cdbdaab5dd6997c2963a31c24cc04e3c08534
Jul 27 19:27:35.826074 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.826055    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-2fqrl"
Jul 27 19:27:35.829110 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:27:35.828959    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4869a388_94e3_4a30_a33e_29579dcad555.slice/crio-141361678aeae6b702f87c1eab95968e707452141bb71b731fbcc83b444f7b66 WatchSource:0}: Error finding container 141361678aeae6b702f87c1eab95968e707452141bb71b731fbcc83b444f7b66: Status 404 returned error can't find the container with id 141361678aeae6b702f87c1eab95968e707452141bb71b731fbcc83b444f7b66
Jul 27 19:27:35.832727 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.832710    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-6bsx5"
Jul 27 19:27:35.834214 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:27:35.834195    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod438069a4_ca90_4995_ba78_32af922abb36.slice/crio-7b9c01f2a99e85b7cd4378179ddbdef36e62615a100d594d3afac46cf5b86977 WatchSource:0}: Error finding container 7b9c01f2a99e85b7cd4378179ddbdef36e62615a100d594d3afac46cf5b86977: Status 404 returned error can't find the container with id 7b9c01f2a99e85b7cd4378179ddbdef36e62615a100d594d3afac46cf5b86977
Jul 27 19:27:35.838549 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.838529    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bnrsg"
Jul 27 19:27:35.842654 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:27:35.842635    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87618c00_0367_482a_813e_9aab8f341fa5.slice/crio-abc7a352f0f2a6c61ff21c9db3d17a01bf585407c48baeeac0c778951a4308ee WatchSource:0}: Error finding container abc7a352f0f2a6c61ff21c9db3d17a01bf585407c48baeeac0c778951a4308ee: Status 404 returned error can't find the container with id abc7a352f0f2a6c61ff21c9db3d17a01bf585407c48baeeac0c778951a4308ee
Jul 27 19:27:35.843563 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.843547    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qljqz"
Jul 27 19:27:35.847957 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:27:35.847938    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd48f4d00_ad6d_4b8e_9a57_de8ed6729c98.slice/crio-12db68599b9b487b2dbb6dd6b1762a3e301203ba92bf19578093ac7a5f7c01de WatchSource:0}: Error finding container 12db68599b9b487b2dbb6dd6b1762a3e301203ba92bf19578093ac7a5f7c01de: Status 404 returned error can't find the container with id 12db68599b9b487b2dbb6dd6b1762a3e301203ba92bf19578093ac7a5f7c01de
Jul 27 19:27:35.848729 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.848711    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:27:35.852755 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:27:35.852732    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0267f96_3a9d_45d1_b943_92899defe086.slice/crio-14ffde006705a89d065a03d1aa610f42390d9f5ed2ba601d2117ebc64e359980 WatchSource:0}: Error finding container 14ffde006705a89d065a03d1aa610f42390d9f5ed2ba601d2117ebc64e359980: Status 404 returned error can't find the container with id 14ffde006705a89d065a03d1aa610f42390d9f5ed2ba601d2117ebc64e359980
Jul 27 19:27:35.852834 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:35.852778    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-5zsvw"
Jul 27 19:27:35.859574 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:27:35.859551    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod103bab23_bca3_472a_974b_7731f83c5bfb.slice/crio-a07c1c11ea6c310413e3ea3ff3981da88ebd48389addef6d54d1d72a4e434f4f WatchSource:0}: Error finding container a07c1c11ea6c310413e3ea3ff3981da88ebd48389addef6d54d1d72a4e434f4f: Status 404 returned error can't find the container with id a07c1c11ea6c310413e3ea3ff3981da88ebd48389addef6d54d1d72a4e434f4f
Jul 27 19:27:35.863194 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:27:35.863174    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7719045e_d600_437b_aa76_7f03b29fecd9.slice/crio-6a9e9d07f1ab8c44dceaf10466147edac364fd59fac0aeaf83cef9cfc77e5ab9 WatchSource:0}: Error finding container 6a9e9d07f1ab8c44dceaf10466147edac364fd59fac0aeaf83cef9cfc77e5ab9: Status 404 returned error can't find the container with id 6a9e9d07f1ab8c44dceaf10466147edac364fd59fac0aeaf83cef9cfc77e5ab9
Jul 27 19:27:36.237137 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:36.237107    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/229036e0-7551-4987-98d3-0ccef9c511e5-metrics-certs\") pod \"network-metrics-daemon-zbjtr\" (UID: \"229036e0-7551-4987-98d3-0ccef9c511e5\") " pod="openshift-multus/network-metrics-daemon-zbjtr"
Jul 27 19:27:36.237137 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:36.237153    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3590f4c7-efa9-4c98-ae82-99afc906db15-original-pull-secret\") pod \"global-pull-secret-syncer-2g8q7\" (UID: \"3590f4c7-efa9-4c98-ae82-99afc906db15\") " pod="kube-system/global-pull-secret-syncer-2g8q7"
Jul 27 19:27:36.237393 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:36.237264    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 19:27:36.237393 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:36.237265    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 19:27:36.237393 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:36.237323    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3590f4c7-efa9-4c98-ae82-99afc906db15-original-pull-secret podName:3590f4c7-efa9-4c98-ae82-99afc906db15 nodeName:}" failed. No retries permitted until 2026-07-27 19:27:37.237305325 +0000 UTC m=+70.109497295 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3590f4c7-efa9-4c98-ae82-99afc906db15-original-pull-secret") pod "global-pull-secret-syncer-2g8q7" (UID: "3590f4c7-efa9-4c98-ae82-99afc906db15") : object "kube-system"/"original-pull-secret" not registered
Jul 27 19:27:36.237393 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:36.237346    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/229036e0-7551-4987-98d3-0ccef9c511e5-metrics-certs podName:229036e0-7551-4987-98d3-0ccef9c511e5 nodeName:}" failed. No retries permitted until 2026-07-27 19:27:37.237332296 +0000 UTC m=+70.109524253 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/229036e0-7551-4987-98d3-0ccef9c511e5-metrics-certs") pod "network-metrics-daemon-zbjtr" (UID: "229036e0-7551-4987-98d3-0ccef9c511e5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 19:27:36.337667 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:36.337630    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmgsm\" (UniqueName: \"kubernetes.io/projected/88050dfb-f434-42bd-8c5f-9dc6f98673e0-kube-api-access-wmgsm\") pod \"network-check-target-2vtnn\" (UID: \"88050dfb-f434-42bd-8c5f-9dc6f98673e0\") " pod="openshift-network-diagnostics/network-check-target-2vtnn"
Jul 27 19:27:36.337845 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:36.337820    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 19:27:36.337845 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:36.337843    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 19:27:36.337959 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:36.337855    2558 projected.go:196] Error preparing data for projected volume kube-api-access-wmgsm for pod openshift-network-diagnostics/network-check-target-2vtnn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 19:27:36.337959 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:36.337913    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/88050dfb-f434-42bd-8c5f-9dc6f98673e0-kube-api-access-wmgsm podName:88050dfb-f434-42bd-8c5f-9dc6f98673e0 nodeName:}" failed. No retries permitted until 2026-07-27 19:27:37.337894546 +0000 UTC m=+70.210086501 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-wmgsm" (UniqueName: "kubernetes.io/projected/88050dfb-f434-42bd-8c5f-9dc6f98673e0-kube-api-access-wmgsm") pod "network-check-target-2vtnn" (UID: "88050dfb-f434-42bd-8c5f-9dc6f98673e0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 19:27:36.606846 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:36.606706    2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-26 19:22:35 +0000 UTC" deadline="2028-02-10 02:39:11.352310846 +0000 UTC"
Jul 27 19:27:36.606846 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:36.606743    2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13495h11m34.745573076s"
Jul 27 19:27:36.669923 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:36.669373    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zbjtr"
Jul 27 19:27:36.669923 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:36.669538    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zbjtr" podUID="229036e0-7551-4987-98d3-0ccef9c511e5"
Jul 27 19:27:36.801035 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:36.800998    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qljqz" event={"ID":"e0267f96-3a9d-45d1-b943-92899defe086","Type":"ContainerStarted","Data":"14ffde006705a89d065a03d1aa610f42390d9f5ed2ba601d2117ebc64e359980"}
Jul 27 19:27:36.818483 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:36.818445    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-2cw79" event={"ID":"4869a388-94e3-4a30-a33e-29579dcad555","Type":"ContainerStarted","Data":"141361678aeae6b702f87c1eab95968e707452141bb71b731fbcc83b444f7b66"}
Jul 27 19:27:36.829137 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:36.828102    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2xk7d" event={"ID":"e9a0f4ae-3761-4764-baa1-8d8c005113a0","Type":"ContainerStarted","Data":"d62d4ff6a992feff9e14914519df9171111416526adc43db7911e39ee5f8c39b"}
Jul 27 19:27:36.832766 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:36.832734    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-5zsvw" event={"ID":"7719045e-d600-437b-aa76-7f03b29fecd9","Type":"ContainerStarted","Data":"6a9e9d07f1ab8c44dceaf10466147edac364fd59fac0aeaf83cef9cfc77e5ab9"}
Jul 27 19:27:36.835216 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:36.834980    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2" event={"ID":"103bab23-bca3-472a-974b-7731f83c5bfb","Type":"ContainerStarted","Data":"a07c1c11ea6c310413e3ea3ff3981da88ebd48389addef6d54d1d72a4e434f4f"}
Jul 27 19:27:36.839636 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:36.839610    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bnrsg" event={"ID":"d48f4d00-ad6d-4b8e-9a57-de8ed6729c98","Type":"ContainerStarted","Data":"12db68599b9b487b2dbb6dd6b1762a3e301203ba92bf19578093ac7a5f7c01de"}
Jul 27 19:27:36.843499 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:36.843472    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-6bsx5" event={"ID":"87618c00-0367-482a-813e-9aab8f341fa5","Type":"ContainerStarted","Data":"abc7a352f0f2a6c61ff21c9db3d17a01bf585407c48baeeac0c778951a4308ee"}
Jul 27 19:27:36.846704 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:36.846676    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-2fqrl" event={"ID":"438069a4-ca90-4995-ba78-32af922abb36","Type":"ContainerStarted","Data":"7b9c01f2a99e85b7cd4378179ddbdef36e62615a100d594d3afac46cf5b86977"}
Jul 27 19:27:36.853278 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:36.853251    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4dkh" event={"ID":"24afe7ac-4ce8-43b3-a919-1dc0a0e76fb5","Type":"ContainerStarted","Data":"90ca8ed799e6ea6c076ff32b698cdbdaab5dd6997c2963a31c24cc04e3c08534"}
Jul 27 19:27:37.248052 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:37.247264    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/229036e0-7551-4987-98d3-0ccef9c511e5-metrics-certs\") pod \"network-metrics-daemon-zbjtr\" (UID: \"229036e0-7551-4987-98d3-0ccef9c511e5\") " pod="openshift-multus/network-metrics-daemon-zbjtr"
Jul 27 19:27:37.248052 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:37.247326    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3590f4c7-efa9-4c98-ae82-99afc906db15-original-pull-secret\") pod \"global-pull-secret-syncer-2g8q7\" (UID: \"3590f4c7-efa9-4c98-ae82-99afc906db15\") " pod="kube-system/global-pull-secret-syncer-2g8q7"
Jul 27 19:27:37.248052 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:37.247499    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 19:27:37.248052 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:37.247559    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3590f4c7-efa9-4c98-ae82-99afc906db15-original-pull-secret podName:3590f4c7-efa9-4c98-ae82-99afc906db15 nodeName:}" failed. No retries permitted until 2026-07-27 19:27:39.247541253 +0000 UTC m=+72.119733212 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3590f4c7-efa9-4c98-ae82-99afc906db15-original-pull-secret") pod "global-pull-secret-syncer-2g8q7" (UID: "3590f4c7-efa9-4c98-ae82-99afc906db15") : object "kube-system"/"original-pull-secret" not registered
Jul 27 19:27:37.248052 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:37.247933    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 19:27:37.248052 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:37.247977    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/229036e0-7551-4987-98d3-0ccef9c511e5-metrics-certs podName:229036e0-7551-4987-98d3-0ccef9c511e5 nodeName:}" failed. No retries permitted until 2026-07-27 19:27:39.247962832 +0000 UTC m=+72.120154789 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/229036e0-7551-4987-98d3-0ccef9c511e5-metrics-certs") pod "network-metrics-daemon-zbjtr" (UID: "229036e0-7551-4987-98d3-0ccef9c511e5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 19:27:37.348005 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:37.347969    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmgsm\" (UniqueName: \"kubernetes.io/projected/88050dfb-f434-42bd-8c5f-9dc6f98673e0-kube-api-access-wmgsm\") pod \"network-check-target-2vtnn\" (UID: \"88050dfb-f434-42bd-8c5f-9dc6f98673e0\") " pod="openshift-network-diagnostics/network-check-target-2vtnn"
Jul 27 19:27:37.348187 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:37.348149    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 19:27:37.348187 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:37.348170    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 19:27:37.348187 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:37.348182    2558 projected.go:196] Error preparing data for projected volume kube-api-access-wmgsm for pod openshift-network-diagnostics/network-check-target-2vtnn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 19:27:37.348342 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:37.348238    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/88050dfb-f434-42bd-8c5f-9dc6f98673e0-kube-api-access-wmgsm podName:88050dfb-f434-42bd-8c5f-9dc6f98673e0 nodeName:}" failed. No retries permitted until 2026-07-27 19:27:39.34822072 +0000 UTC m=+72.220412680 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-wmgsm" (UniqueName: "kubernetes.io/projected/88050dfb-f434-42bd-8c5f-9dc6f98673e0-kube-api-access-wmgsm") pod "network-check-target-2vtnn" (UID: "88050dfb-f434-42bd-8c5f-9dc6f98673e0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 19:27:37.608189 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:37.607279    2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-26 19:22:35 +0000 UTC" deadline="2028-02-26 12:30:59.16207635 +0000 UTC"
Jul 27 19:27:37.608189 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:37.607315    2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13889h3m21.5547657s"
Jul 27 19:27:37.669928 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:37.669473    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2vtnn"
Jul 27 19:27:37.669928 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:37.669581    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2vtnn" podUID="88050dfb-f434-42bd-8c5f-9dc6f98673e0"
Jul 27 19:27:37.670364 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:37.670231    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2g8q7"
Jul 27 19:27:37.670364 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:37.670330    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2g8q7" podUID="3590f4c7-efa9-4c98-ae82-99afc906db15"
Jul 27 19:27:38.668219 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:38.668184    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zbjtr"
Jul 27 19:27:38.668707 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:38.668351    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zbjtr" podUID="229036e0-7551-4987-98d3-0ccef9c511e5"
Jul 27 19:27:39.265545 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:39.264934    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/229036e0-7551-4987-98d3-0ccef9c511e5-metrics-certs\") pod \"network-metrics-daemon-zbjtr\" (UID: \"229036e0-7551-4987-98d3-0ccef9c511e5\") " pod="openshift-multus/network-metrics-daemon-zbjtr"
Jul 27 19:27:39.265545 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:39.264992    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3590f4c7-efa9-4c98-ae82-99afc906db15-original-pull-secret\") pod \"global-pull-secret-syncer-2g8q7\" (UID: \"3590f4c7-efa9-4c98-ae82-99afc906db15\") " pod="kube-system/global-pull-secret-syncer-2g8q7"
Jul 27 19:27:39.265545 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:39.265134    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 19:27:39.265545 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:39.265134    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 19:27:39.265545 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:39.265201    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/229036e0-7551-4987-98d3-0ccef9c511e5-metrics-certs podName:229036e0-7551-4987-98d3-0ccef9c511e5 nodeName:}" failed. No retries permitted until 2026-07-27 19:27:43.265183241 +0000 UTC m=+76.137375194 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/229036e0-7551-4987-98d3-0ccef9c511e5-metrics-certs") pod "network-metrics-daemon-zbjtr" (UID: "229036e0-7551-4987-98d3-0ccef9c511e5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 19:27:39.265545 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:39.265218    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3590f4c7-efa9-4c98-ae82-99afc906db15-original-pull-secret podName:3590f4c7-efa9-4c98-ae82-99afc906db15 nodeName:}" failed. No retries permitted until 2026-07-27 19:27:43.265209439 +0000 UTC m=+76.137401391 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3590f4c7-efa9-4c98-ae82-99afc906db15-original-pull-secret") pod "global-pull-secret-syncer-2g8q7" (UID: "3590f4c7-efa9-4c98-ae82-99afc906db15") : object "kube-system"/"original-pull-secret" not registered
Jul 27 19:27:39.366339 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:39.365671    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmgsm\" (UniqueName: \"kubernetes.io/projected/88050dfb-f434-42bd-8c5f-9dc6f98673e0-kube-api-access-wmgsm\") pod \"network-check-target-2vtnn\" (UID: \"88050dfb-f434-42bd-8c5f-9dc6f98673e0\") " pod="openshift-network-diagnostics/network-check-target-2vtnn"
Jul 27 19:27:39.366339 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:39.365888    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 19:27:39.366339 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:39.365903    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 19:27:39.366339 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:39.365912    2558 projected.go:196] Error preparing data for projected volume kube-api-access-wmgsm for pod openshift-network-diagnostics/network-check-target-2vtnn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 19:27:39.366339 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:39.365998    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/88050dfb-f434-42bd-8c5f-9dc6f98673e0-kube-api-access-wmgsm podName:88050dfb-f434-42bd-8c5f-9dc6f98673e0 nodeName:}" failed. No retries permitted until 2026-07-27 19:27:43.365977722 +0000 UTC m=+76.238169727 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-wmgsm" (UniqueName: "kubernetes.io/projected/88050dfb-f434-42bd-8c5f-9dc6f98673e0-kube-api-access-wmgsm") pod "network-check-target-2vtnn" (UID: "88050dfb-f434-42bd-8c5f-9dc6f98673e0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 19:27:39.670013 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:39.669535    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2vtnn"
Jul 27 19:27:39.670013 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:39.669660    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2vtnn" podUID="88050dfb-f434-42bd-8c5f-9dc6f98673e0"
Jul 27 19:27:39.670799 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:39.670638    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2g8q7"
Jul 27 19:27:39.670799 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:39.670747    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2g8q7" podUID="3590f4c7-efa9-4c98-ae82-99afc906db15"
Jul 27 19:27:40.668669 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:40.668634    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zbjtr"
Jul 27 19:27:40.668861 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:40.668768    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zbjtr" podUID="229036e0-7551-4987-98d3-0ccef9c511e5"
Jul 27 19:27:41.668555 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:41.668519    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2g8q7"
Jul 27 19:27:41.669040 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:41.668654    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2g8q7" podUID="3590f4c7-efa9-4c98-ae82-99afc906db15"
Jul 27 19:27:41.669040 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:41.669024    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2vtnn"
Jul 27 19:27:41.669139 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:41.669116    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2vtnn" podUID="88050dfb-f434-42bd-8c5f-9dc6f98673e0"
Jul 27 19:27:42.669313 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:42.669138    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zbjtr"
Jul 27 19:27:42.669313 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:42.669276    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zbjtr" podUID="229036e0-7551-4987-98d3-0ccef9c511e5"
Jul 27 19:27:43.298953 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:43.298403    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/229036e0-7551-4987-98d3-0ccef9c511e5-metrics-certs\") pod \"network-metrics-daemon-zbjtr\" (UID: \"229036e0-7551-4987-98d3-0ccef9c511e5\") " pod="openshift-multus/network-metrics-daemon-zbjtr"
Jul 27 19:27:43.298953 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:43.298479    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3590f4c7-efa9-4c98-ae82-99afc906db15-original-pull-secret\") pod \"global-pull-secret-syncer-2g8q7\" (UID: \"3590f4c7-efa9-4c98-ae82-99afc906db15\") " pod="kube-system/global-pull-secret-syncer-2g8q7"
Jul 27 19:27:43.298953 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:43.298551    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 19:27:43.298953 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:43.298628    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/229036e0-7551-4987-98d3-0ccef9c511e5-metrics-certs podName:229036e0-7551-4987-98d3-0ccef9c511e5 nodeName:}" failed. No retries permitted until 2026-07-27 19:27:51.298605466 +0000 UTC m=+84.170797432 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/229036e0-7551-4987-98d3-0ccef9c511e5-metrics-certs") pod "network-metrics-daemon-zbjtr" (UID: "229036e0-7551-4987-98d3-0ccef9c511e5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 19:27:43.298953 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:43.298625    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 19:27:43.298953 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:43.298691    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3590f4c7-efa9-4c98-ae82-99afc906db15-original-pull-secret podName:3590f4c7-efa9-4c98-ae82-99afc906db15 nodeName:}" failed. No retries permitted until 2026-07-27 19:27:51.29867445 +0000 UTC m=+84.170866403 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3590f4c7-efa9-4c98-ae82-99afc906db15-original-pull-secret") pod "global-pull-secret-syncer-2g8q7" (UID: "3590f4c7-efa9-4c98-ae82-99afc906db15") : object "kube-system"/"original-pull-secret" not registered
Jul 27 19:27:43.399048 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:43.399013    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmgsm\" (UniqueName: \"kubernetes.io/projected/88050dfb-f434-42bd-8c5f-9dc6f98673e0-kube-api-access-wmgsm\") pod \"network-check-target-2vtnn\" (UID: \"88050dfb-f434-42bd-8c5f-9dc6f98673e0\") " pod="openshift-network-diagnostics/network-check-target-2vtnn"
Jul 27 19:27:43.399243 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:43.399163    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 19:27:43.399243 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:43.399187    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 19:27:43.399243 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:43.399201    2558 projected.go:196] Error preparing data for projected volume kube-api-access-wmgsm for pod openshift-network-diagnostics/network-check-target-2vtnn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 19:27:43.399405 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:43.399263    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/88050dfb-f434-42bd-8c5f-9dc6f98673e0-kube-api-access-wmgsm podName:88050dfb-f434-42bd-8c5f-9dc6f98673e0 nodeName:}" failed. No retries permitted until 2026-07-27 19:27:51.399244218 +0000 UTC m=+84.271436170 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-wmgsm" (UniqueName: "kubernetes.io/projected/88050dfb-f434-42bd-8c5f-9dc6f98673e0-kube-api-access-wmgsm") pod "network-check-target-2vtnn" (UID: "88050dfb-f434-42bd-8c5f-9dc6f98673e0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 19:27:43.668852 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:43.668820    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2vtnn"
Jul 27 19:27:43.669038 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:43.668953    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2vtnn" podUID="88050dfb-f434-42bd-8c5f-9dc6f98673e0"
Jul 27 19:27:43.669038 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:43.668998    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2g8q7"
Jul 27 19:27:43.669149 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:43.669122    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2g8q7" podUID="3590f4c7-efa9-4c98-ae82-99afc906db15"
Jul 27 19:27:43.728697 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:43.728662    2558 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 19:27:44.668696 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:44.668644    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zbjtr"
Jul 27 19:27:44.668865 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:44.668794    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zbjtr" podUID="229036e0-7551-4987-98d3-0ccef9c511e5"
Jul 27 19:27:45.671132 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:45.671094    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2g8q7"
Jul 27 19:27:45.671604 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:45.671219    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2g8q7" podUID="3590f4c7-efa9-4c98-ae82-99afc906db15"
Jul 27 19:27:45.671692 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:45.671625    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2vtnn"
Jul 27 19:27:45.671747 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:45.671724    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2vtnn" podUID="88050dfb-f434-42bd-8c5f-9dc6f98673e0"
Jul 27 19:27:46.668699 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:46.668656    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zbjtr"
Jul 27 19:27:46.668883 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:46.668797    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zbjtr" podUID="229036e0-7551-4987-98d3-0ccef9c511e5"
Jul 27 19:27:47.668984 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:47.668846    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2vtnn"
Jul 27 19:27:47.668984 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:47.668948    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2vtnn" podUID="88050dfb-f434-42bd-8c5f-9dc6f98673e0"
Jul 27 19:27:47.668984 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:47.668988    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2g8q7"
Jul 27 19:27:47.669549 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:47.669036    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2g8q7" podUID="3590f4c7-efa9-4c98-ae82-99afc906db15"
Jul 27 19:27:48.668736 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:48.668703    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zbjtr"
Jul 27 19:27:48.668939 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:48.668841    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zbjtr" podUID="229036e0-7551-4987-98d3-0ccef9c511e5"
Jul 27 19:27:49.669115 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:49.669084    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2g8q7"
Jul 27 19:27:49.669664 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:49.669204    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2g8q7" podUID="3590f4c7-efa9-4c98-ae82-99afc906db15"
Jul 27 19:27:49.669664 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:49.669234    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2vtnn"
Jul 27 19:27:49.669664 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:49.669302    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2vtnn" podUID="88050dfb-f434-42bd-8c5f-9dc6f98673e0"
Jul 27 19:27:50.668935 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:50.668897    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zbjtr"
Jul 27 19:27:50.669096 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:50.669033    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zbjtr" podUID="229036e0-7551-4987-98d3-0ccef9c511e5"
Jul 27 19:27:51.357949 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:51.357905    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3590f4c7-efa9-4c98-ae82-99afc906db15-original-pull-secret\") pod \"global-pull-secret-syncer-2g8q7\" (UID: \"3590f4c7-efa9-4c98-ae82-99afc906db15\") " pod="kube-system/global-pull-secret-syncer-2g8q7"
Jul 27 19:27:51.358478 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:51.358017    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/229036e0-7551-4987-98d3-0ccef9c511e5-metrics-certs\") pod \"network-metrics-daemon-zbjtr\" (UID: \"229036e0-7551-4987-98d3-0ccef9c511e5\") " pod="openshift-multus/network-metrics-daemon-zbjtr"
Jul 27 19:27:51.358478 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:51.358079    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 19:27:51.358478 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:51.358111    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 19:27:51.358478 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:51.358154    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3590f4c7-efa9-4c98-ae82-99afc906db15-original-pull-secret podName:3590f4c7-efa9-4c98-ae82-99afc906db15 nodeName:}" failed. No retries permitted until 2026-07-27 19:28:07.358137572 +0000 UTC m=+100.230329524 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3590f4c7-efa9-4c98-ae82-99afc906db15-original-pull-secret") pod "global-pull-secret-syncer-2g8q7" (UID: "3590f4c7-efa9-4c98-ae82-99afc906db15") : object "kube-system"/"original-pull-secret" not registered
Jul 27 19:27:51.358478 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:51.358175    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/229036e0-7551-4987-98d3-0ccef9c511e5-metrics-certs podName:229036e0-7551-4987-98d3-0ccef9c511e5 nodeName:}" failed. No retries permitted until 2026-07-27 19:28:07.358165066 +0000 UTC m=+100.230357026 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/229036e0-7551-4987-98d3-0ccef9c511e5-metrics-certs") pod "network-metrics-daemon-zbjtr" (UID: "229036e0-7551-4987-98d3-0ccef9c511e5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 19:27:51.459009 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:51.458973    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmgsm\" (UniqueName: \"kubernetes.io/projected/88050dfb-f434-42bd-8c5f-9dc6f98673e0-kube-api-access-wmgsm\") pod \"network-check-target-2vtnn\" (UID: \"88050dfb-f434-42bd-8c5f-9dc6f98673e0\") " pod="openshift-network-diagnostics/network-check-target-2vtnn"
Jul 27 19:27:51.459191 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:51.459171    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 19:27:51.459255 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:51.459200    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 19:27:51.459255 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:51.459215    2558 projected.go:196] Error preparing data for projected volume kube-api-access-wmgsm for pod openshift-network-diagnostics/network-check-target-2vtnn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 19:27:51.459349 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:51.459279    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/88050dfb-f434-42bd-8c5f-9dc6f98673e0-kube-api-access-wmgsm podName:88050dfb-f434-42bd-8c5f-9dc6f98673e0 nodeName:}" failed. No retries permitted until 2026-07-27 19:28:07.459260742 +0000 UTC m=+100.331452705 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-wmgsm" (UniqueName: "kubernetes.io/projected/88050dfb-f434-42bd-8c5f-9dc6f98673e0-kube-api-access-wmgsm") pod "network-check-target-2vtnn" (UID: "88050dfb-f434-42bd-8c5f-9dc6f98673e0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 19:27:51.668535 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:51.668502    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2vtnn"
Jul 27 19:27:51.668698 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:51.668540    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2g8q7"
Jul 27 19:27:51.668698 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:51.668609    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2vtnn" podUID="88050dfb-f434-42bd-8c5f-9dc6f98673e0"
Jul 27 19:27:51.668814 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:51.668717    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2g8q7" podUID="3590f4c7-efa9-4c98-ae82-99afc906db15"
Jul 27 19:27:52.668505 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:52.668468    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zbjtr"
Jul 27 19:27:52.668987 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:52.668613    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zbjtr" podUID="229036e0-7551-4987-98d3-0ccef9c511e5"
Jul 27 19:27:53.177768 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:53.177735    2558 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 19:27:53.668494 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:53.668461    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2vtnn"
Jul 27 19:27:53.668494 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:53.668488    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2g8q7"
Jul 27 19:27:53.668955 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:53.668570    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2vtnn" podUID="88050dfb-f434-42bd-8c5f-9dc6f98673e0"
Jul 27 19:27:53.668955 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:53.668696    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2g8q7" podUID="3590f4c7-efa9-4c98-ae82-99afc906db15"
Jul 27 19:27:54.668303 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:54.668137    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zbjtr"
Jul 27 19:27:54.668457 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:54.668397    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zbjtr" podUID="229036e0-7551-4987-98d3-0ccef9c511e5"
Jul 27 19:27:54.896065 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:54.896045    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5dfc2_103bab23-bca3-472a-974b-7731f83c5bfb/ovn-acl-logging/0.log"
Jul 27 19:27:54.896642 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:54.896383    2558 generic.go:356] "Generic (PLEG): container finished" podID="103bab23-bca3-472a-974b-7731f83c5bfb" containerID="d5fb760a5c19a5b80ee94dd0fcc33432bf60fa3fb5720b6bd729f5303ae0f0df" exitCode=1
Jul 27 19:27:54.896642 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:54.896440    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2" event={"ID":"103bab23-bca3-472a-974b-7731f83c5bfb","Type":"ContainerStarted","Data":"810fcb53d658d413bcc0b7f772042dd01b2ef1bffde8972602173d973be0dc27"}
Jul 27 19:27:54.896642 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:54.896527    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2" event={"ID":"103bab23-bca3-472a-974b-7731f83c5bfb","Type":"ContainerDied","Data":"d5fb760a5c19a5b80ee94dd0fcc33432bf60fa3fb5720b6bd729f5303ae0f0df"}
Jul 27 19:27:54.896642 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:54.896582    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2" event={"ID":"103bab23-bca3-472a-974b-7731f83c5bfb","Type":"ContainerStarted","Data":"9b74bc3ac8569661a2109ab699e99268be3adda62b8ab8ff9a243bc277d933ed"}
Jul 27 19:27:54.900939 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:54.900488    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bnrsg" event={"ID":"d48f4d00-ad6d-4b8e-9a57-de8ed6729c98","Type":"ContainerStarted","Data":"772e6c03d30daaf288e394c9cd41054f79ba004f9a52f2e0d9b1a59a29cd423a"}
Jul 27 19:27:54.902308 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:54.902288    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-2fqrl" event={"ID":"438069a4-ca90-4995-ba78-32af922abb36","Type":"ContainerStarted","Data":"9ea3f2668d3ea86b50ca2b1a3c17c53953d836d0e195c6a3431e02f7b4d0719c"}
Jul 27 19:27:54.903763 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:54.903743    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4dkh" event={"ID":"24afe7ac-4ce8-43b3-a919-1dc0a0e76fb5","Type":"ContainerStarted","Data":"b90d864430d0054f338df0c4a1245d22105f507a7f5ad0a9449a729dd2b2ddf4"}
Jul 27 19:27:54.905566 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:54.905549    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal_3c4ec7bc2375d832f57ca8b11d2ae551/setup/2.log"
Jul 27 19:27:54.905653 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:54.905578    2558 generic.go:356] "Generic (PLEG): container finished" podID="3c4ec7bc2375d832f57ca8b11d2ae551" containerID="bed0b0002ec59c0e3975dbd89f43afe2699dcfc566b4263a5e281a124699d572" exitCode=0
Jul 27 19:27:54.905653 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:54.905631    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal" event={"ID":"3c4ec7bc2375d832f57ca8b11d2ae551","Type":"ContainerDied","Data":"bed0b0002ec59c0e3975dbd89f43afe2699dcfc566b4263a5e281a124699d572"}
Jul 27 19:27:54.905747 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:54.905666    2558 scope.go:117] "RemoveContainer" containerID="71e65ca902a5f8d6923443b1c9a99ec5e16e561aa1f7acfbab194c31fa90884e"
Jul 27 19:27:54.905918 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:54.905901    2558 scope.go:117] "RemoveContainer" containerID="71e65ca902a5f8d6923443b1c9a99ec5e16e561aa1f7acfbab194c31fa90884e"
Jul 27 19:27:54.907240 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:54.907214    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qljqz" event={"ID":"e0267f96-3a9d-45d1-b943-92899defe086","Type":"ContainerStarted","Data":"6ee20b6b812cdae5349c30e9c9487f978fb0025bb6a810a6d2d3e41b23e35390"}
Jul 27 19:27:54.908622 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:54.908603    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-2cw79" event={"ID":"4869a388-94e3-4a30-a33e-29579dcad555","Type":"ContainerStarted","Data":"52bbc85d4b69069d117bbbd985ef991bbf38b1adecf0af7fe241558584645e77"}
Jul 27 19:27:54.909921 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:54.909898    2558 generic.go:356] "Generic (PLEG): container finished" podID="e9a0f4ae-3761-4764-baa1-8d8c005113a0" containerID="b5cb6b60b5f34d68797be42c0e27c9588572b9fdd7111b80964a56d52d57c0ed" exitCode=0
Jul 27 19:27:54.910002 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:54.909974    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2xk7d" event={"ID":"e9a0f4ae-3761-4764-baa1-8d8c005113a0","Type":"ContainerDied","Data":"b5cb6b60b5f34d68797be42c0e27c9588572b9fdd7111b80964a56d52d57c0ed"}
Jul 27 19:27:54.911225 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:54.911204    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-5zsvw" event={"ID":"7719045e-d600-437b-aa76-7f03b29fecd9","Type":"ContainerStarted","Data":"e03ad16f19d2cc87759b2d8774e963ceff72767c9d648c44c921e274b201b122"}
Jul 27 19:27:54.920735 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:54.920700    2558 log.go:32] "RemoveContainer from runtime service failed" err="rpc error: code = Unknown desc = failed to delete container k8s_setup_kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal_openshift-machine-config-operator_3c4ec7bc2375d832f57ca8b11d2ae551_2 in pod sandbox 6223ac60670451ea8d93063d83f9fe6fa7a3fac77a3fb51a2b25d10225fcd2f7 from index: no such id: '71e65ca902a5f8d6923443b1c9a99ec5e16e561aa1f7acfbab194c31fa90884e'" containerID="71e65ca902a5f8d6923443b1c9a99ec5e16e561aa1f7acfbab194c31fa90884e"
Jul 27 19:27:54.920794 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:54.920754    2558 kuberuntime_container.go:962] "Unhandled Error" err="failed to remove pod init container \"setup\": rpc error: code = Unknown desc = failed to delete container k8s_setup_kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal_openshift-machine-config-operator_3c4ec7bc2375d832f57ca8b11d2ae551_2 in pod sandbox 6223ac60670451ea8d93063d83f9fe6fa7a3fac77a3fb51a2b25d10225fcd2f7 from index: no such id: '71e65ca902a5f8d6923443b1c9a99ec5e16e561aa1f7acfbab194c31fa90884e'; Skipping pod \"kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal_openshift-machine-config-operator(3c4ec7bc2375d832f57ca8b11d2ae551)\"" logger="UnhandledError"
Jul 27 19:27:54.936724 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:54.936687    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-bnrsg" podStartSLOduration=21.464118308 podStartE2EDuration="39.936673685s" totalImagesPullingTime="18.472555377s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 19:27:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:27:54.915983548 +0000 UTC m=+87.788175524" watchObservedRunningTime="2026-07-27 19:27:54.936673685 +0000 UTC m=+87.808865714"
Jul 27 19:27:54.950923 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:54.950876    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-2cw79" podStartSLOduration=21.458867745 podStartE2EDuration="39.950862782s" totalImagesPullingTime="18.491995037s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 19:27:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:27:54.950123464 +0000 UTC m=+87.822315437" watchObservedRunningTime="2026-07-27 19:27:54.950862782 +0000 UTC m=+87.823054755"
Jul 27 19:27:54.964109 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:54.964068    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-5zsvw" podStartSLOduration=26.072005974 podStartE2EDuration="39.964054865s" totalImagesPullingTime="13.892048891s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:27:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:27:54.963586261 +0000 UTC m=+87.835778236" watchObservedRunningTime="2026-07-27 19:27:54.964054865 +0000 UTC m=+87.836246839"
Jul 27 19:27:54.994452 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:54.994181    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-2fqrl" podStartSLOduration=21.492802058 podStartE2EDuration="39.994160959s" totalImagesPullingTime="18.501358901s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 19:27:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:27:54.993524669 +0000 UTC m=+87.865716643" watchObservedRunningTime="2026-07-27 19:27:54.994160959 +0000 UTC m=+87.866352935"
Jul 27 19:27:55.010464 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:55.008933    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-qljqz" podStartSLOduration=21.572364963 podStartE2EDuration="40.00891749s" totalImagesPullingTime="18.436552527s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 19:27:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:27:55.008598833 +0000 UTC m=+87.880790808" watchObservedRunningTime="2026-07-27 19:27:55.00891749 +0000 UTC m=+87.881109468"
Jul 27 19:27:55.668401 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:55.668375    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2vtnn"
Jul 27 19:27:55.668559 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:55.668498    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2vtnn" podUID="88050dfb-f434-42bd-8c5f-9dc6f98673e0"
Jul 27 19:27:55.668559 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:55.668527    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2g8q7"
Jul 27 19:27:55.668625 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:55.668586    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2g8q7" podUID="3590f4c7-efa9-4c98-ae82-99afc906db15"
Jul 27 19:27:55.824545 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:55.824517    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:55.853091 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:55.853057    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-5zsvw"
Jul 27 19:27:55.853692 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:55.853676    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-5zsvw"
Jul 27 19:27:55.883514 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:55.883491    2558 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 27 19:27:55.915399 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:55.915366    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4dkh" event={"ID":"24afe7ac-4ce8-43b3-a919-1dc0a0e76fb5","Type":"ContainerStarted","Data":"c7923eef7a430c2a9fd13163d2e9a5b6eb20265c87a432e86d6d2234833386a0"}
Jul 27 19:27:55.917533 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:55.917512    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal" event={"ID":"3c4ec7bc2375d832f57ca8b11d2ae551","Type":"ContainerStarted","Data":"f77ab63605574b417ca5cd9d2ee116a0d71cd3eb51bc77882a8b9fb92be9eca8"}
Jul 27 19:27:55.920628 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:55.920578    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5dfc2_103bab23-bca3-472a-974b-7731f83c5bfb/ovn-acl-logging/0.log"
Jul 27 19:27:55.920967 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:55.920943    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2" event={"ID":"103bab23-bca3-472a-974b-7731f83c5bfb","Type":"ContainerStarted","Data":"91552d78e7b1b4f44f0ed9edffefa3825fdfd091088636921bd9a5bfebd41d57"}
Jul 27 19:27:55.921048 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:55.921021    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2" event={"ID":"103bab23-bca3-472a-974b-7731f83c5bfb","Type":"ContainerStarted","Data":"aa07ec4e428de2891049fa9380a89e61b7c064c9202ecfad41c3503246110421"}
Jul 27 19:27:55.921102 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:55.921072    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2" event={"ID":"103bab23-bca3-472a-974b-7731f83c5bfb","Type":"ContainerStarted","Data":"4a9d0db88c89475082c8775d6bde8317514e0d4bcec8bb443644045d7121c47d"}
Jul 27 19:27:55.922428 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:55.922386    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-6bsx5" event={"ID":"87618c00-0367-482a-813e-9aab8f341fa5","Type":"ContainerStarted","Data":"3dfd761459f483ea57d4824b33b061bd785c7297370929d8d735a00a2604c12e"}
Jul 27 19:27:55.923457 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:55.923438    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-5zsvw"
Jul 27 19:27:55.923851 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:55.923833    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-5zsvw"
Jul 27 19:27:55.933027 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:55.932982    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-235.ec2.internal" podStartSLOduration=20.932967716 podStartE2EDuration="20.932967716s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 19:27:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:27:55.932666946 +0000 UTC m=+88.804858919" watchObservedRunningTime="2026-07-27 19:27:55.932967716 +0000 UTC m=+88.805159691"
Jul 27 19:27:55.937099 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:55.937072    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:55.970803 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:55.970758    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-6bsx5" podStartSLOduration=22.523432913 podStartE2EDuration="40.970740587s" totalImagesPullingTime="18.447307674s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 19:27:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:27:55.97022847 +0000 UTC m=+88.842420446" watchObservedRunningTime="2026-07-27 19:27:55.970740587 +0000 UTC m=+88.842932564"
Jul 27 19:27:56.038058 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:56.038027    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:56.143605 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:56.143571    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:56.209232 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:56.209169    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:56.463576 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:56.463473    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:56.515569 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:56.515526    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:56.583427 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:56.583306    2558 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-27T19:27:55.883510093Z","UUID":"c4aa7406-7b36-44f4-9fa8-86e15ae70c4a","Handler":null,"Name":"","Endpoint":""}
Jul 27 19:27:56.584891 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:56.584866    2558 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 27 19:27:56.585013 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:56.584909    2558 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 27 19:27:56.668609 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:56.668582    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zbjtr"
Jul 27 19:27:56.668767 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:56.668701    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zbjtr" podUID="229036e0-7551-4987-98d3-0ccef9c511e5"
Jul 27 19:27:57.671537 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:57.671505    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2g8q7"
Jul 27 19:27:57.672068 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:57.671594    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2g8q7" podUID="3590f4c7-efa9-4c98-ae82-99afc906db15"
Jul 27 19:27:57.672068 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:57.671644    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2vtnn"
Jul 27 19:27:57.672068 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:57.671718    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2vtnn" podUID="88050dfb-f434-42bd-8c5f-9dc6f98673e0"
Jul 27 19:27:57.931113 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:57.931038    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5dfc2_103bab23-bca3-472a-974b-7731f83c5bfb/ovn-acl-logging/0.log"
Jul 27 19:27:57.931539 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:57.931508    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2" event={"ID":"103bab23-bca3-472a-974b-7731f83c5bfb","Type":"ContainerStarted","Data":"6c8a0967a9fb4f6a84af9d7d48549cbf030a50ef1547ce6791ed887d6959c058"}
Jul 27 19:27:57.933828 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:57.933797    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4dkh" event={"ID":"24afe7ac-4ce8-43b3-a919-1dc0a0e76fb5","Type":"ContainerStarted","Data":"c9b5c1e98bbf1af8702897ed058f85e644ce7a56428190a7cc7faa99feb00366"}
Jul 27 19:27:57.953679 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:57.953634    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4dkh" podStartSLOduration=22.326438851 podStartE2EDuration="42.953614135s" totalImagesPullingTime="20.627175284s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 19:27:15 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:27:57.953004329 +0000 UTC m=+90.825196305" watchObservedRunningTime="2026-07-27 19:27:57.953614135 +0000 UTC m=+90.825806113"
Jul 27 19:27:58.314775 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:58.314689    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:58.379184 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:58.379145    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:58.668356 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:58.668323    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zbjtr"
Jul 27 19:27:58.668546 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:58.668470    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zbjtr" podUID="229036e0-7551-4987-98d3-0ccef9c511e5"
Jul 27 19:27:59.625691 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:59.625537    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:59.668582 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:59.668542    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2vtnn"
Jul 27 19:27:59.668745 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:59.668677    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2vtnn" podUID="88050dfb-f434-42bd-8c5f-9dc6f98673e0"
Jul 27 19:27:59.668745 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:59.668724    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2g8q7"
Jul 27 19:27:59.668858 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:27:59.668817    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2g8q7" podUID="3590f4c7-efa9-4c98-ae82-99afc906db15"
Jul 27 19:27:59.941532 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:59.941506    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5dfc2_103bab23-bca3-472a-974b-7731f83c5bfb/ovn-acl-logging/0.log"
Jul 27 19:27:59.941925 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:59.941900    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2" event={"ID":"103bab23-bca3-472a-974b-7731f83c5bfb","Type":"ContainerStarted","Data":"61a682ed13ddc0f0c913b7166e77d8dc8cec2442323cc17afa3a79b11b38d80d"}
Jul 27 19:27:59.942170 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:59.942149    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:27:59.942341 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:59.942322    2558 scope.go:117] "RemoveContainer" containerID="d5fb760a5c19a5b80ee94dd0fcc33432bf60fa3fb5720b6bd729f5303ae0f0df"
Jul 27 19:27:59.943935 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:59.943916    2558 generic.go:356] "Generic (PLEG): container finished" podID="e9a0f4ae-3761-4764-baa1-8d8c005113a0" containerID="e81b602e75bce813f3594e27b3230016e92a62beb0b80cf87fed44048804104d" exitCode=0
Jul 27 19:27:59.944018 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:59.943949    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2xk7d" event={"ID":"e9a0f4ae-3761-4764-baa1-8d8c005113a0","Type":"ContainerDied","Data":"e81b602e75bce813f3594e27b3230016e92a62beb0b80cf87fed44048804104d"}
Jul 27 19:27:59.963776 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:27:59.963755    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:28:00.098921 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:00.098765    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:00.668848 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:00.668817    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zbjtr"
Jul 27 19:28:00.669287 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:28:00.668959    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zbjtr" podUID="229036e0-7551-4987-98d3-0ccef9c511e5"
Jul 27 19:28:00.951281 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:00.951221    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5dfc2_103bab23-bca3-472a-974b-7731f83c5bfb/ovn-acl-logging/0.log"
Jul 27 19:28:00.951646 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:00.951619    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2" event={"ID":"103bab23-bca3-472a-974b-7731f83c5bfb","Type":"ContainerStarted","Data":"03628d5799c4e41060a2754e503d88044f8c530551782fea20ee76db2e0b7bbb"}
Jul 27 19:28:00.951928 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:00.951891    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:28:00.952025 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:00.951986    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:28:00.967570 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:00.967543    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:28:00.986673 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:00.986625    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2" podStartSLOduration=27.463500561 podStartE2EDuration="45.986611169s" totalImagesPullingTime="18.523110608s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 19:27:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:28:00.985542558 +0000 UTC m=+93.857734532" watchObservedRunningTime="2026-07-27 19:28:00.986611169 +0000 UTC m=+93.858803182"
Jul 27 19:28:01.182514 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:01.182488    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:01.278796 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:01.278774    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:01.484042 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:01.483970    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:01.580671 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:01.580640    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:01.617569 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:01.617249    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2vtnn"]
Jul 27 19:28:01.617569 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:01.617381    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2vtnn"
Jul 27 19:28:01.617569 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:28:01.617515    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2vtnn" podUID="88050dfb-f434-42bd-8c5f-9dc6f98673e0"
Jul 27 19:28:01.617948 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:01.617929    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2g8q7"]
Jul 27 19:28:01.618079 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:01.618025    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2g8q7"
Jul 27 19:28:01.618135 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:28:01.618117    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2g8q7" podUID="3590f4c7-efa9-4c98-ae82-99afc906db15"
Jul 27 19:28:01.622020 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:01.621931    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-zbjtr"]
Jul 27 19:28:01.622020 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:01.622023    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zbjtr"
Jul 27 19:28:01.622151 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:28:01.622098    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zbjtr" podUID="229036e0-7551-4987-98d3-0ccef9c511e5"
Jul 27 19:28:01.955401 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:01.955370    2558 generic.go:356] "Generic (PLEG): container finished" podID="e9a0f4ae-3761-4764-baa1-8d8c005113a0" containerID="45231dd44c8412ebcb355e99672211bf3b8ea54c96319704a0c4ad5f7a63fc4f" exitCode=0
Jul 27 19:28:01.956032 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:01.955448    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2xk7d" event={"ID":"e9a0f4ae-3761-4764-baa1-8d8c005113a0","Type":"ContainerDied","Data":"45231dd44c8412ebcb355e99672211bf3b8ea54c96319704a0c4ad5f7a63fc4f"}
Jul 27 19:28:02.668894 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:02.668858    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2vtnn"
Jul 27 19:28:02.669055 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:28:02.668980    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2vtnn" podUID="88050dfb-f434-42bd-8c5f-9dc6f98673e0"
Jul 27 19:28:02.959568 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:02.959542    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2xk7d" event={"ID":"e9a0f4ae-3761-4764-baa1-8d8c005113a0","Type":"ContainerStarted","Data":"51ee1f0e649c4a6a1c8370261c78bcf5b0ceca2a01438a1c1aef4591bcc917f5"}
Jul 27 19:28:03.671826 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:03.671761    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2g8q7"
Jul 27 19:28:03.672019 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:28:03.671858    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2g8q7" podUID="3590f4c7-efa9-4c98-ae82-99afc906db15"
Jul 27 19:28:03.672019 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:03.671909    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zbjtr"
Jul 27 19:28:03.672019 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:28:03.671994    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zbjtr" podUID="229036e0-7551-4987-98d3-0ccef9c511e5"
Jul 27 19:28:03.964489 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:03.964398    2558 generic.go:356] "Generic (PLEG): container finished" podID="e9a0f4ae-3761-4764-baa1-8d8c005113a0" containerID="51ee1f0e649c4a6a1c8370261c78bcf5b0ceca2a01438a1c1aef4591bcc917f5" exitCode=0
Jul 27 19:28:03.964489 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:03.964456    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2xk7d" event={"ID":"e9a0f4ae-3761-4764-baa1-8d8c005113a0","Type":"ContainerDied","Data":"51ee1f0e649c4a6a1c8370261c78bcf5b0ceca2a01438a1c1aef4591bcc917f5"}
Jul 27 19:28:04.668926 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:04.668894    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2vtnn"
Jul 27 19:28:04.669100 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:28:04.669006    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2vtnn" podUID="88050dfb-f434-42bd-8c5f-9dc6f98673e0"
Jul 27 19:28:05.672354 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:05.672324    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2g8q7"
Jul 27 19:28:05.672790 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:28:05.672476    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2g8q7" podUID="3590f4c7-efa9-4c98-ae82-99afc906db15"
Jul 27 19:28:05.672790 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:05.672532    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zbjtr"
Jul 27 19:28:05.672790 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:28:05.672643    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zbjtr" podUID="229036e0-7551-4987-98d3-0ccef9c511e5"
Jul 27 19:28:06.669564 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:06.669059    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2vtnn"
Jul 27 19:28:06.669564 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:28:06.669174    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2vtnn" podUID="88050dfb-f434-42bd-8c5f-9dc6f98673e0"
Jul 27 19:28:07.377874 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:07.377843    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/229036e0-7551-4987-98d3-0ccef9c511e5-metrics-certs\") pod \"network-metrics-daemon-zbjtr\" (UID: \"229036e0-7551-4987-98d3-0ccef9c511e5\") " pod="openshift-multus/network-metrics-daemon-zbjtr"
Jul 27 19:28:07.378306 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:07.377882    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3590f4c7-efa9-4c98-ae82-99afc906db15-original-pull-secret\") pod \"global-pull-secret-syncer-2g8q7\" (UID: \"3590f4c7-efa9-4c98-ae82-99afc906db15\") " pod="kube-system/global-pull-secret-syncer-2g8q7"
Jul 27 19:28:07.378306 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:28:07.378011    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 19:28:07.378306 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:28:07.378012    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 19:28:07.378306 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:28:07.378071    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3590f4c7-efa9-4c98-ae82-99afc906db15-original-pull-secret podName:3590f4c7-efa9-4c98-ae82-99afc906db15 nodeName:}" failed. No retries permitted until 2026-07-27 19:28:39.378053592 +0000 UTC m=+132.250245545 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3590f4c7-efa9-4c98-ae82-99afc906db15-original-pull-secret") pod "global-pull-secret-syncer-2g8q7" (UID: "3590f4c7-efa9-4c98-ae82-99afc906db15") : object "kube-system"/"original-pull-secret" not registered
Jul 27 19:28:07.378306 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:28:07.378093    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/229036e0-7551-4987-98d3-0ccef9c511e5-metrics-certs podName:229036e0-7551-4987-98d3-0ccef9c511e5 nodeName:}" failed. No retries permitted until 2026-07-27 19:28:39.378082019 +0000 UTC m=+132.250273975 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/229036e0-7551-4987-98d3-0ccef9c511e5-metrics-certs") pod "network-metrics-daemon-zbjtr" (UID: "229036e0-7551-4987-98d3-0ccef9c511e5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 19:28:07.421511 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:07.421473    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-235.ec2.internal" event="NodeReady"
Jul 27 19:28:07.421667 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:07.421605    2558 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 27 19:28:07.478706 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:07.478669    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmgsm\" (UniqueName: \"kubernetes.io/projected/88050dfb-f434-42bd-8c5f-9dc6f98673e0-kube-api-access-wmgsm\") pod \"network-check-target-2vtnn\" (UID: \"88050dfb-f434-42bd-8c5f-9dc6f98673e0\") " pod="openshift-network-diagnostics/network-check-target-2vtnn"
Jul 27 19:28:07.478886 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:28:07.478845    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 19:28:07.478886 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:28:07.478865    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 19:28:07.478886 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:28:07.478877    2558 projected.go:196] Error preparing data for projected volume kube-api-access-wmgsm for pod openshift-network-diagnostics/network-check-target-2vtnn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 19:28:07.479013 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:28:07.478930    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/88050dfb-f434-42bd-8c5f-9dc6f98673e0-kube-api-access-wmgsm podName:88050dfb-f434-42bd-8c5f-9dc6f98673e0 nodeName:}" failed. No retries permitted until 2026-07-27 19:28:39.478911756 +0000 UTC m=+132.351103711 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-wmgsm" (UniqueName: "kubernetes.io/projected/88050dfb-f434-42bd-8c5f-9dc6f98673e0-kube-api-access-wmgsm") pod "network-check-target-2vtnn" (UID: "88050dfb-f434-42bd-8c5f-9dc6f98673e0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 19:28:07.671849 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:07.671776    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2g8q7"
Jul 27 19:28:07.672015 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:07.671961    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zbjtr"
Jul 27 19:28:07.674485 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:07.674462    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 27 19:28:07.674612 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:07.674462    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 27 19:28:07.674612 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:07.674514    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-lj4rj\""
Jul 27 19:28:08.668904 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:08.668845    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2vtnn"
Jul 27 19:28:08.671473 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:08.671454    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-wght4\""
Jul 27 19:28:08.671587 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:08.671470    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 27 19:28:08.671587 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:08.671470    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 27 19:28:08.980012 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:08.979931    2558 generic.go:356] "Generic (PLEG): container finished" podID="e9a0f4ae-3761-4764-baa1-8d8c005113a0" containerID="0eb94a80f478c20f3e1165c3546eecfd251d6c7aeb366c31ae3c96914d342d9d" exitCode=0
Jul 27 19:28:08.980012 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:08.979967    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2xk7d" event={"ID":"e9a0f4ae-3761-4764-baa1-8d8c005113a0","Type":"ContainerDied","Data":"0eb94a80f478c20f3e1165c3546eecfd251d6c7aeb366c31ae3c96914d342d9d"}
Jul 27 19:28:09.984802 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:09.984762    2558 generic.go:356] "Generic (PLEG): container finished" podID="e9a0f4ae-3761-4764-baa1-8d8c005113a0" containerID="f3c8eed2abffd0e79a2a0348ee3304a73562164c6ae8c725e8d3bc6840a19601" exitCode=0
Jul 27 19:28:09.985207 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:09.984804    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2xk7d" event={"ID":"e9a0f4ae-3761-4764-baa1-8d8c005113a0","Type":"ContainerDied","Data":"f3c8eed2abffd0e79a2a0348ee3304a73562164c6ae8c725e8d3bc6840a19601"}
Jul 27 19:28:10.827835 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:10.827802    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:10.900061 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:10.900015    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:10.989648 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:10.989619    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2xk7d" event={"ID":"e9a0f4ae-3761-4764-baa1-8d8c005113a0","Type":"ContainerStarted","Data":"43b5e8e0447e2f062a64775407e0fe82eb209b0333e33da625956bb24290331d"}
Jul 27 19:28:11.013113 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:11.013065    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-2xk7d" podStartSLOduration=26.413473356 podStartE2EDuration="56.013049752s" totalImagesPullingTime="29.599576396s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 19:27:15 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:28:11.011553157 +0000 UTC m=+103.883745128" watchObservedRunningTime="2026-07-27 19:28:11.013049752 +0000 UTC m=+103.885241739"
Jul 27 19:28:11.135045 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:11.135019    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:11.235681 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:11.235653    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:11.336250 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:11.336223    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:11.748134 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:11.748104    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:11.814044 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:11.814014    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:12.823713 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:12.823678    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:12.925233 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:12.925200    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:19.337646 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:19.337615    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:19.409134 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:19.409103    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:26.886898 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:26.886859    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:26.956725 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:26.956697    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:28.644590 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:28.644552    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:28.811818 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:28.811762    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:32.970351 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:32.970323    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5dfc2"
Jul 27 19:28:33.122023 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:33.121993    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:33.182442 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:33.182397    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:39.394913 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:39.394866    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/229036e0-7551-4987-98d3-0ccef9c511e5-metrics-certs\") pod \"network-metrics-daemon-zbjtr\" (UID: \"229036e0-7551-4987-98d3-0ccef9c511e5\") " pod="openshift-multus/network-metrics-daemon-zbjtr"
Jul 27 19:28:39.394913 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:39.394915    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3590f4c7-efa9-4c98-ae82-99afc906db15-original-pull-secret\") pod \"global-pull-secret-syncer-2g8q7\" (UID: \"3590f4c7-efa9-4c98-ae82-99afc906db15\") " pod="kube-system/global-pull-secret-syncer-2g8q7"
Jul 27 19:28:39.397647 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:39.397629    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 27 19:28:39.397726 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:39.397629    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 27 19:28:39.405481 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:28:39.405460    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 27 19:28:39.405603 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:28:39.405542    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/229036e0-7551-4987-98d3-0ccef9c511e5-metrics-certs podName:229036e0-7551-4987-98d3-0ccef9c511e5 nodeName:}" failed. No retries permitted until 2026-07-27 19:29:43.405520489 +0000 UTC m=+196.277712441 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/229036e0-7551-4987-98d3-0ccef9c511e5-metrics-certs") pod "network-metrics-daemon-zbjtr" (UID: "229036e0-7551-4987-98d3-0ccef9c511e5") : secret "metrics-daemon-secret" not found
Jul 27 19:28:39.409361 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:39.409344    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3590f4c7-efa9-4c98-ae82-99afc906db15-original-pull-secret\") pod \"global-pull-secret-syncer-2g8q7\" (UID: \"3590f4c7-efa9-4c98-ae82-99afc906db15\") " pod="kube-system/global-pull-secret-syncer-2g8q7"
Jul 27 19:28:39.484163 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:39.484140    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2g8q7"
Jul 27 19:28:39.495703 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:39.495679    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmgsm\" (UniqueName: \"kubernetes.io/projected/88050dfb-f434-42bd-8c5f-9dc6f98673e0-kube-api-access-wmgsm\") pod \"network-check-target-2vtnn\" (UID: \"88050dfb-f434-42bd-8c5f-9dc6f98673e0\") " pod="openshift-network-diagnostics/network-check-target-2vtnn"
Jul 27 19:28:39.498093 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:39.498076    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 27 19:28:39.508866 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:39.508845    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 27 19:28:39.520122 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:39.520098    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmgsm\" (UniqueName: \"kubernetes.io/projected/88050dfb-f434-42bd-8c5f-9dc6f98673e0-kube-api-access-wmgsm\") pod \"network-check-target-2vtnn\" (UID: \"88050dfb-f434-42bd-8c5f-9dc6f98673e0\") " pod="openshift-network-diagnostics/network-check-target-2vtnn"
Jul 27 19:28:39.584278 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:39.584254    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-wght4\""
Jul 27 19:28:39.592459 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:39.592437    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2vtnn"
Jul 27 19:28:39.616970 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:39.616936    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2g8q7"]
Jul 27 19:28:39.620504 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:28:39.620471    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3590f4c7_efa9_4c98_ae82_99afc906db15.slice/crio-8ec969365a72a965c73a4ea04afb79b5070a40bf507c4229577e3cec72417ee9 WatchSource:0}: Error finding container 8ec969365a72a965c73a4ea04afb79b5070a40bf507c4229577e3cec72417ee9: Status 404 returned error can't find the container with id 8ec969365a72a965c73a4ea04afb79b5070a40bf507c4229577e3cec72417ee9
Jul 27 19:28:39.711191 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:39.711161    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2vtnn"]
Jul 27 19:28:39.714126 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:28:39.714103    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88050dfb_f434_42bd_8c5f_9dc6f98673e0.slice/crio-a56a9344f1168f590f25ca55dbabc15443eba089d843e6500369df340b6623d5 WatchSource:0}: Error finding container a56a9344f1168f590f25ca55dbabc15443eba089d843e6500369df340b6623d5: Status 404 returned error can't find the container with id a56a9344f1168f590f25ca55dbabc15443eba089d843e6500369df340b6623d5
Jul 27 19:28:40.061178 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:40.061082    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2vtnn" event={"ID":"88050dfb-f434-42bd-8c5f-9dc6f98673e0","Type":"ContainerStarted","Data":"a56a9344f1168f590f25ca55dbabc15443eba089d843e6500369df340b6623d5"}
Jul 27 19:28:40.062264 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:40.062232    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2g8q7" event={"ID":"3590f4c7-efa9-4c98-ae82-99afc906db15","Type":"ContainerStarted","Data":"8ec969365a72a965c73a4ea04afb79b5070a40bf507c4229577e3cec72417ee9"}
Jul 27 19:28:45.075858 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:45.075810    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2vtnn" event={"ID":"88050dfb-f434-42bd-8c5f-9dc6f98673e0","Type":"ContainerStarted","Data":"ce5356f6c7d376c18a8bfc461ec95bac4d187f7abdf86d6417d6e2c961c6f99a"}
Jul 27 19:28:45.076273 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:45.075907    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-2vtnn"
Jul 27 19:28:45.077194 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:45.077165    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2g8q7" event={"ID":"3590f4c7-efa9-4c98-ae82-99afc906db15","Type":"ContainerStarted","Data":"62967144b02b076cd27192cba713d858c2ad7c0197abf4a621a5d26abaca1e4f"}
Jul 27 19:28:45.091758 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:45.091713    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-2vtnn" podStartSLOduration=85.585435063 podStartE2EDuration="1m30.091699013s" totalImagesPullingTime="4.50626395s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:27:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:28:45.09112103 +0000 UTC m=+137.963313007" watchObservedRunningTime="2026-07-27 19:28:45.091699013 +0000 UTC m=+137.963891009"
Jul 27 19:28:45.108165 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:45.108122    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-2g8q7" podStartSLOduration=85.504913542 podStartE2EDuration="1m30.108107719s" totalImagesPullingTime="4.603194177s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 19:27:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:28:45.107220677 +0000 UTC m=+137.979412651" watchObservedRunningTime="2026-07-27 19:28:45.108107719 +0000 UTC m=+137.980299690"
Jul 27 19:28:48.891465 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:48.891430    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:48.992368 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:48.992335    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:49.062027 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:28:49.062008    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:16.082070 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:16.082031    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-2vtnn"
Jul 27 19:29:24.031597 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.031475    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:24.088709 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.088672    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7f4bbb7d8b-kjzcn"]
Jul 27 19:29:24.090726 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.090703    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f4bbb7d8b-kjzcn"
Jul 27 19:29:24.094565 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.093176    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-txhl6\""
Jul 27 19:29:24.094565 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.093369    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 27 19:29:24.094565 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.093554    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 27 19:29:24.094565 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.093765    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 27 19:29:24.108104 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.108078    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-4rxwd"]
Jul 27 19:29:24.123404 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.123367    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 27 19:29:24.123628 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.123608    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-9r7sw"]
Jul 27 19:29:24.123837 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.123697    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4rxwd"
Jul 27 19:29:24.125313 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.125287    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:24.125966 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.125945    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 27 19:29:24.126223 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.126200    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-nht7b\""
Jul 27 19:29:24.126552 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.126533    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 27 19:29:24.126777 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.126759    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7f4bbb7d8b-kjzcn"]
Jul 27 19:29:24.126866 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.126795    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-9r7sw"]
Jul 27 19:29:24.126866 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.126818    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-4rxwd"]
Jul 27 19:29:24.126866 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.126847    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-9r7sw"
Jul 27 19:29:24.128897 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.128877    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 27 19:29:24.129045 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.129029    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 27 19:29:24.129176 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.129164    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 27 19:29:24.129237 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.129194    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-mdj99\""
Jul 27 19:29:24.190919 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.190884    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6-trusted-ca\") pod \"image-registry-7f4bbb7d8b-kjzcn\" (UID: \"a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6\") " pod="openshift-image-registry/image-registry-7f4bbb7d8b-kjzcn"
Jul 27 19:29:24.190919 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.190916    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vngjq\" (UniqueName: \"kubernetes.io/projected/a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6-kube-api-access-vngjq\") pod \"image-registry-7f4bbb7d8b-kjzcn\" (UID: \"a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6\") " pod="openshift-image-registry/image-registry-7f4bbb7d8b-kjzcn"
Jul 27 19:29:24.191184 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.190943    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6-installation-pull-secrets\") pod \"image-registry-7f4bbb7d8b-kjzcn\" (UID: \"a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6\") " pod="openshift-image-registry/image-registry-7f4bbb7d8b-kjzcn"
Jul 27 19:29:24.191184 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.190994    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6-registry-certificates\") pod \"image-registry-7f4bbb7d8b-kjzcn\" (UID: \"a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6\") " pod="openshift-image-registry/image-registry-7f4bbb7d8b-kjzcn"
Jul 27 19:29:24.191184 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.191027    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6-registry-tls\") pod \"image-registry-7f4bbb7d8b-kjzcn\" (UID: \"a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6\") " pod="openshift-image-registry/image-registry-7f4bbb7d8b-kjzcn"
Jul 27 19:29:24.191184 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.191048    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6-ca-trust-extracted\") pod \"image-registry-7f4bbb7d8b-kjzcn\" (UID: \"a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6\") " pod="openshift-image-registry/image-registry-7f4bbb7d8b-kjzcn"
Jul 27 19:29:24.191184 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.191065    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6-bound-sa-token\") pod \"image-registry-7f4bbb7d8b-kjzcn\" (UID: \"a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6\") " pod="openshift-image-registry/image-registry-7f4bbb7d8b-kjzcn"
Jul 27 19:29:24.191184 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.191096    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6-image-registry-private-configuration\") pod \"image-registry-7f4bbb7d8b-kjzcn\" (UID: \"a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6\") " pod="openshift-image-registry/image-registry-7f4bbb7d8b-kjzcn"
Jul 27 19:29:24.230462 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.230422    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:24.291624 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.291529    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6-registry-certificates\") pod \"image-registry-7f4bbb7d8b-kjzcn\" (UID: \"a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6\") " pod="openshift-image-registry/image-registry-7f4bbb7d8b-kjzcn"
Jul 27 19:29:24.291624 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.291573    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6-registry-tls\") pod \"image-registry-7f4bbb7d8b-kjzcn\" (UID: \"a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6\") " pod="openshift-image-registry/image-registry-7f4bbb7d8b-kjzcn"
Jul 27 19:29:24.291624 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.291594    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6-ca-trust-extracted\") pod \"image-registry-7f4bbb7d8b-kjzcn\" (UID: \"a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6\") " pod="openshift-image-registry/image-registry-7f4bbb7d8b-kjzcn"
Jul 27 19:29:24.291624 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.291609    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6-bound-sa-token\") pod \"image-registry-7f4bbb7d8b-kjzcn\" (UID: \"a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6\") " pod="openshift-image-registry/image-registry-7f4bbb7d8b-kjzcn"
Jul 27 19:29:24.291966 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.291633    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/219afe94-ca1b-4a80-a421-e6cb1bb321b1-metrics-tls\") pod \"dns-default-4rxwd\" (UID: \"219afe94-ca1b-4a80-a421-e6cb1bb321b1\") " pod="openshift-dns/dns-default-4rxwd"
Jul 27 19:29:24.291966 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.291667    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6-image-registry-private-configuration\") pod \"image-registry-7f4bbb7d8b-kjzcn\" (UID: \"a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6\") " pod="openshift-image-registry/image-registry-7f4bbb7d8b-kjzcn"
Jul 27 19:29:24.291966 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.291690    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwk5b\" (UniqueName: \"kubernetes.io/projected/6e0976f8-4dd4-4fb2-8458-b426a96a105c-kube-api-access-cwk5b\") pod \"ingress-canary-9r7sw\" (UID: \"6e0976f8-4dd4-4fb2-8458-b426a96a105c\") " pod="openshift-ingress-canary/ingress-canary-9r7sw"
Jul 27 19:29:24.291966 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.291707    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6-trusted-ca\") pod \"image-registry-7f4bbb7d8b-kjzcn\" (UID: \"a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6\") " pod="openshift-image-registry/image-registry-7f4bbb7d8b-kjzcn"
Jul 27 19:29:24.291966 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:29:24.291710    2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 27 19:29:24.291966 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:29:24.291730    2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f4bbb7d8b-kjzcn: secret "image-registry-tls" not found
Jul 27 19:29:24.291966 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.291733    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vngjq\" (UniqueName: \"kubernetes.io/projected/a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6-kube-api-access-vngjq\") pod \"image-registry-7f4bbb7d8b-kjzcn\" (UID: \"a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6\") " pod="openshift-image-registry/image-registry-7f4bbb7d8b-kjzcn"
Jul 27 19:29:24.291966 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.291756    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/219afe94-ca1b-4a80-a421-e6cb1bb321b1-tmp-dir\") pod \"dns-default-4rxwd\" (UID: \"219afe94-ca1b-4a80-a421-e6cb1bb321b1\") " pod="openshift-dns/dns-default-4rxwd"
Jul 27 19:29:24.291966 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.291784    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6e0976f8-4dd4-4fb2-8458-b426a96a105c-cert\") pod \"ingress-canary-9r7sw\" (UID: \"6e0976f8-4dd4-4fb2-8458-b426a96a105c\") " pod="openshift-ingress-canary/ingress-canary-9r7sw"
Jul 27 19:29:24.291966 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:29:24.291804    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6-registry-tls podName:a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6 nodeName:}" failed. No retries permitted until 2026-07-27 19:29:24.791784442 +0000 UTC m=+177.663976403 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6-registry-tls") pod "image-registry-7f4bbb7d8b-kjzcn" (UID: "a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6") : secret "image-registry-tls" not found
Jul 27 19:29:24.291966 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.291954    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/219afe94-ca1b-4a80-a421-e6cb1bb321b1-config-volume\") pod \"dns-default-4rxwd\" (UID: \"219afe94-ca1b-4a80-a421-e6cb1bb321b1\") " pod="openshift-dns/dns-default-4rxwd"
Jul 27 19:29:24.292587 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.291986    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6-installation-pull-secrets\") pod \"image-registry-7f4bbb7d8b-kjzcn\" (UID: \"a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6\") " pod="openshift-image-registry/image-registry-7f4bbb7d8b-kjzcn"
Jul 27 19:29:24.292587 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.292009    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whr4r\" (UniqueName: \"kubernetes.io/projected/219afe94-ca1b-4a80-a421-e6cb1bb321b1-kube-api-access-whr4r\") pod \"dns-default-4rxwd\" (UID: \"219afe94-ca1b-4a80-a421-e6cb1bb321b1\") " pod="openshift-dns/dns-default-4rxwd"
Jul 27 19:29:24.292587 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.292106    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6-ca-trust-extracted\") pod \"image-registry-7f4bbb7d8b-kjzcn\" (UID: \"a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6\") " pod="openshift-image-registry/image-registry-7f4bbb7d8b-kjzcn"
Jul 27 19:29:24.292587 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.292310    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6-registry-certificates\") pod \"image-registry-7f4bbb7d8b-kjzcn\" (UID: \"a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6\") " pod="openshift-image-registry/image-registry-7f4bbb7d8b-kjzcn"
Jul 27 19:29:24.292717 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.292607    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6-trusted-ca\") pod \"image-registry-7f4bbb7d8b-kjzcn\" (UID: \"a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6\") " pod="openshift-image-registry/image-registry-7f4bbb7d8b-kjzcn"
Jul 27 19:29:24.294240 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.294222    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6-image-registry-private-configuration\") pod \"image-registry-7f4bbb7d8b-kjzcn\" (UID: \"a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6\") " pod="openshift-image-registry/image-registry-7f4bbb7d8b-kjzcn"
Jul 27 19:29:24.294389 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.294371    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6-installation-pull-secrets\") pod \"image-registry-7f4bbb7d8b-kjzcn\" (UID: \"a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6\") " pod="openshift-image-registry/image-registry-7f4bbb7d8b-kjzcn"
Jul 27 19:29:24.302381 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.302356    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:24.304498 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.304471    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6-bound-sa-token\") pod \"image-registry-7f4bbb7d8b-kjzcn\" (UID: \"a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6\") " pod="openshift-image-registry/image-registry-7f4bbb7d8b-kjzcn"
Jul 27 19:29:24.304647 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.304628    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vngjq\" (UniqueName: \"kubernetes.io/projected/a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6-kube-api-access-vngjq\") pod \"image-registry-7f4bbb7d8b-kjzcn\" (UID: \"a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6\") " pod="openshift-image-registry/image-registry-7f4bbb7d8b-kjzcn"
Jul 27 19:29:24.392552 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.392516    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whr4r\" (UniqueName: \"kubernetes.io/projected/219afe94-ca1b-4a80-a421-e6cb1bb321b1-kube-api-access-whr4r\") pod \"dns-default-4rxwd\" (UID: \"219afe94-ca1b-4a80-a421-e6cb1bb321b1\") " pod="openshift-dns/dns-default-4rxwd"
Jul 27 19:29:24.392721 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.392580    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/219afe94-ca1b-4a80-a421-e6cb1bb321b1-metrics-tls\") pod \"dns-default-4rxwd\" (UID: \"219afe94-ca1b-4a80-a421-e6cb1bb321b1\") " pod="openshift-dns/dns-default-4rxwd"
Jul 27 19:29:24.392721 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.392606    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwk5b\" (UniqueName: \"kubernetes.io/projected/6e0976f8-4dd4-4fb2-8458-b426a96a105c-kube-api-access-cwk5b\") pod \"ingress-canary-9r7sw\" (UID: \"6e0976f8-4dd4-4fb2-8458-b426a96a105c\") " pod="openshift-ingress-canary/ingress-canary-9r7sw"
Jul 27 19:29:24.392721 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.392623    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/219afe94-ca1b-4a80-a421-e6cb1bb321b1-tmp-dir\") pod \"dns-default-4rxwd\" (UID: \"219afe94-ca1b-4a80-a421-e6cb1bb321b1\") " pod="openshift-dns/dns-default-4rxwd"
Jul 27 19:29:24.392721 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.392643    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6e0976f8-4dd4-4fb2-8458-b426a96a105c-cert\") pod \"ingress-canary-9r7sw\" (UID: \"6e0976f8-4dd4-4fb2-8458-b426a96a105c\") " pod="openshift-ingress-canary/ingress-canary-9r7sw"
Jul 27 19:29:24.392721 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.392660    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/219afe94-ca1b-4a80-a421-e6cb1bb321b1-config-volume\") pod \"dns-default-4rxwd\" (UID: \"219afe94-ca1b-4a80-a421-e6cb1bb321b1\") " pod="openshift-dns/dns-default-4rxwd"
Jul 27 19:29:24.393009 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:29:24.392779    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 19:29:24.393009 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:29:24.392865    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/219afe94-ca1b-4a80-a421-e6cb1bb321b1-metrics-tls podName:219afe94-ca1b-4a80-a421-e6cb1bb321b1 nodeName:}" failed. No retries permitted until 2026-07-27 19:29:24.892842184 +0000 UTC m=+177.765034136 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/219afe94-ca1b-4a80-a421-e6cb1bb321b1-metrics-tls") pod "dns-default-4rxwd" (UID: "219afe94-ca1b-4a80-a421-e6cb1bb321b1") : secret "dns-default-metrics-tls" not found
Jul 27 19:29:24.393090 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.393034    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/219afe94-ca1b-4a80-a421-e6cb1bb321b1-tmp-dir\") pod \"dns-default-4rxwd\" (UID: \"219afe94-ca1b-4a80-a421-e6cb1bb321b1\") " pod="openshift-dns/dns-default-4rxwd"
Jul 27 19:29:24.393287 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.393269    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/219afe94-ca1b-4a80-a421-e6cb1bb321b1-config-volume\") pod \"dns-default-4rxwd\" (UID: \"219afe94-ca1b-4a80-a421-e6cb1bb321b1\") " pod="openshift-dns/dns-default-4rxwd"
Jul 27 19:29:24.395261 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.395241    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6e0976f8-4dd4-4fb2-8458-b426a96a105c-cert\") pod \"ingress-canary-9r7sw\" (UID: \"6e0976f8-4dd4-4fb2-8458-b426a96a105c\") " pod="openshift-ingress-canary/ingress-canary-9r7sw"
Jul 27 19:29:24.400915 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.400891    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwk5b\" (UniqueName: \"kubernetes.io/projected/6e0976f8-4dd4-4fb2-8458-b426a96a105c-kube-api-access-cwk5b\") pod \"ingress-canary-9r7sw\" (UID: \"6e0976f8-4dd4-4fb2-8458-b426a96a105c\") " pod="openshift-ingress-canary/ingress-canary-9r7sw"
Jul 27 19:29:24.401668 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.401647    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-whr4r\" (UniqueName: \"kubernetes.io/projected/219afe94-ca1b-4a80-a421-e6cb1bb321b1-kube-api-access-whr4r\") pod \"dns-default-4rxwd\" (UID: \"219afe94-ca1b-4a80-a421-e6cb1bb321b1\") " pod="openshift-dns/dns-default-4rxwd"
Jul 27 19:29:24.439996 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.439963    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-9r7sw"
Jul 27 19:29:24.466164 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.466128    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:24.546913 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.546831    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:24.560871 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.560846    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-9r7sw"]
Jul 27 19:29:24.564486 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:29:24.564450    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e0976f8_4dd4_4fb2_8458_b426a96a105c.slice/crio-8c6bc0cb859505f807a9ef42c6958db7aa7306c592015a78f686ff9f8d892541 WatchSource:0}: Error finding container 8c6bc0cb859505f807a9ef42c6958db7aa7306c592015a78f686ff9f8d892541: Status 404 returned error can't find the container with id 8c6bc0cb859505f807a9ef42c6958db7aa7306c592015a78f686ff9f8d892541
Jul 27 19:29:24.796487 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.796444    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6-registry-tls\") pod \"image-registry-7f4bbb7d8b-kjzcn\" (UID: \"a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6\") " pod="openshift-image-registry/image-registry-7f4bbb7d8b-kjzcn"
Jul 27 19:29:24.796661 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:29:24.796570    2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 27 19:29:24.796661 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:29:24.796583    2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f4bbb7d8b-kjzcn: secret "image-registry-tls" not found
Jul 27 19:29:24.796661 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:29:24.796634    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6-registry-tls podName:a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6 nodeName:}" failed. No retries permitted until 2026-07-27 19:29:25.79661885 +0000 UTC m=+178.668810802 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6-registry-tls") pod "image-registry-7f4bbb7d8b-kjzcn" (UID: "a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6") : secret "image-registry-tls" not found
Jul 27 19:29:24.897106 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:24.897078    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/219afe94-ca1b-4a80-a421-e6cb1bb321b1-metrics-tls\") pod \"dns-default-4rxwd\" (UID: \"219afe94-ca1b-4a80-a421-e6cb1bb321b1\") " pod="openshift-dns/dns-default-4rxwd"
Jul 27 19:29:24.897266 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:29:24.897212    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 19:29:24.897303 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:29:24.897270    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/219afe94-ca1b-4a80-a421-e6cb1bb321b1-metrics-tls podName:219afe94-ca1b-4a80-a421-e6cb1bb321b1 nodeName:}" failed. No retries permitted until 2026-07-27 19:29:25.897253063 +0000 UTC m=+178.769445016 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/219afe94-ca1b-4a80-a421-e6cb1bb321b1-metrics-tls") pod "dns-default-4rxwd" (UID: "219afe94-ca1b-4a80-a421-e6cb1bb321b1") : secret "dns-default-metrics-tls" not found
Jul 27 19:29:25.175122 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:25.175035    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-9r7sw" event={"ID":"6e0976f8-4dd4-4fb2-8458-b426a96a105c","Type":"ContainerStarted","Data":"8c6bc0cb859505f807a9ef42c6958db7aa7306c592015a78f686ff9f8d892541"}
Jul 27 19:29:25.804237 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:25.804197    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6-registry-tls\") pod \"image-registry-7f4bbb7d8b-kjzcn\" (UID: \"a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6\") " pod="openshift-image-registry/image-registry-7f4bbb7d8b-kjzcn"
Jul 27 19:29:25.804543 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:29:25.804384    2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 27 19:29:25.804543 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:29:25.804425    2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f4bbb7d8b-kjzcn: secret "image-registry-tls" not found
Jul 27 19:29:25.804543 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:29:25.804508    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6-registry-tls podName:a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6 nodeName:}" failed. No retries permitted until 2026-07-27 19:29:27.804487148 +0000 UTC m=+180.676679103 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6-registry-tls") pod "image-registry-7f4bbb7d8b-kjzcn" (UID: "a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6") : secret "image-registry-tls" not found
Jul 27 19:29:25.904934 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:25.904890    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/219afe94-ca1b-4a80-a421-e6cb1bb321b1-metrics-tls\") pod \"dns-default-4rxwd\" (UID: \"219afe94-ca1b-4a80-a421-e6cb1bb321b1\") " pod="openshift-dns/dns-default-4rxwd"
Jul 27 19:29:25.905102 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:29:25.905044    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 19:29:25.905154 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:29:25.905111    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/219afe94-ca1b-4a80-a421-e6cb1bb321b1-metrics-tls podName:219afe94-ca1b-4a80-a421-e6cb1bb321b1 nodeName:}" failed. No retries permitted until 2026-07-27 19:29:27.905092013 +0000 UTC m=+180.777283968 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/219afe94-ca1b-4a80-a421-e6cb1bb321b1-metrics-tls") pod "dns-default-4rxwd" (UID: "219afe94-ca1b-4a80-a421-e6cb1bb321b1") : secret "dns-default-metrics-tls" not found
Jul 27 19:29:26.594615 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:26.594570    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:26.655637 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:26.655601    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:27.181384 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:27.181352    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-9r7sw" event={"ID":"6e0976f8-4dd4-4fb2-8458-b426a96a105c","Type":"ContainerStarted","Data":"cf2a161be50a16983f21ae35cc251694c2b88096f7d6026fe2ea39b7bb5d45b7"}
Jul 27 19:29:27.196671 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:27.196610    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-9r7sw" podStartSLOduration=0.777761983 podStartE2EDuration="3.196590155s" totalImagesPullingTime="2.418828172s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:29:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:29:27.195142178 +0000 UTC m=+180.067334152" watchObservedRunningTime="2026-07-27 19:29:27.196590155 +0000 UTC m=+180.068782129"
Jul 27 19:29:27.821129 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:27.821087    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6-registry-tls\") pod \"image-registry-7f4bbb7d8b-kjzcn\" (UID: \"a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6\") " pod="openshift-image-registry/image-registry-7f4bbb7d8b-kjzcn"
Jul 27 19:29:27.821543 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:29:27.821231    2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 27 19:29:27.821543 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:29:27.821251    2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f4bbb7d8b-kjzcn: secret "image-registry-tls" not found
Jul 27 19:29:27.821543 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:29:27.821307    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6-registry-tls podName:a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6 nodeName:}" failed. No retries permitted until 2026-07-27 19:29:31.821291205 +0000 UTC m=+184.693483157 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6-registry-tls") pod "image-registry-7f4bbb7d8b-kjzcn" (UID: "a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6") : secret "image-registry-tls" not found
Jul 27 19:29:27.922008 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:27.921972    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/219afe94-ca1b-4a80-a421-e6cb1bb321b1-metrics-tls\") pod \"dns-default-4rxwd\" (UID: \"219afe94-ca1b-4a80-a421-e6cb1bb321b1\") " pod="openshift-dns/dns-default-4rxwd"
Jul 27 19:29:27.922168 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:29:27.922093    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 19:29:27.922168 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:29:27.922146    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/219afe94-ca1b-4a80-a421-e6cb1bb321b1-metrics-tls podName:219afe94-ca1b-4a80-a421-e6cb1bb321b1 nodeName:}" failed. No retries permitted until 2026-07-27 19:29:31.922134415 +0000 UTC m=+184.794326371 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/219afe94-ca1b-4a80-a421-e6cb1bb321b1-metrics-tls") pod "dns-default-4rxwd" (UID: "219afe94-ca1b-4a80-a421-e6cb1bb321b1") : secret "dns-default-metrics-tls" not found
Jul 27 19:29:31.850225 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:31.850189    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6-registry-tls\") pod \"image-registry-7f4bbb7d8b-kjzcn\" (UID: \"a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6\") " pod="openshift-image-registry/image-registry-7f4bbb7d8b-kjzcn"
Jul 27 19:29:31.850638 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:29:31.850303    2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 27 19:29:31.850638 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:29:31.850314    2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f4bbb7d8b-kjzcn: secret "image-registry-tls" not found
Jul 27 19:29:31.850638 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:29:31.850365    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6-registry-tls podName:a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6 nodeName:}" failed. No retries permitted until 2026-07-27 19:29:39.850353316 +0000 UTC m=+192.722545268 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6-registry-tls") pod "image-registry-7f4bbb7d8b-kjzcn" (UID: "a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6") : secret "image-registry-tls" not found
Jul 27 19:29:31.951258 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:31.951208    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/219afe94-ca1b-4a80-a421-e6cb1bb321b1-metrics-tls\") pod \"dns-default-4rxwd\" (UID: \"219afe94-ca1b-4a80-a421-e6cb1bb321b1\") " pod="openshift-dns/dns-default-4rxwd"
Jul 27 19:29:31.951479 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:29:31.951325    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 19:29:31.951479 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:29:31.951378    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/219afe94-ca1b-4a80-a421-e6cb1bb321b1-metrics-tls podName:219afe94-ca1b-4a80-a421-e6cb1bb321b1 nodeName:}" failed. No retries permitted until 2026-07-27 19:29:39.951363698 +0000 UTC m=+192.823555650 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/219afe94-ca1b-4a80-a421-e6cb1bb321b1-metrics-tls") pod "dns-default-4rxwd" (UID: "219afe94-ca1b-4a80-a421-e6cb1bb321b1") : secret "dns-default-metrics-tls" not found
Jul 27 19:29:35.276910 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:35.276877    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-jscrz"]
Jul 27 19:29:35.278852 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:35.278836    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jscrz"
Jul 27 19:29:35.281086 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:35.281061    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 27 19:29:35.281852 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:35.281832    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 27 19:29:35.281852 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:35.281841    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 27 19:29:35.282024 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:35.281894    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 27 19:29:35.282024 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:35.281847    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-bpnh5\""
Jul 27 19:29:35.292060 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:35.292037    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jscrz"]
Jul 27 19:29:35.318014 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:35.317974    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:35.372753 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:35.372716    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8bb804bd-b63d-43b7-8159-80cd1abfab55-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jscrz\" (UID: \"8bb804bd-b63d-43b7-8159-80cd1abfab55\") " pod="openshift-insights/insights-runtime-extractor-jscrz"
Jul 27 19:29:35.372935 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:35.372762    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8bb804bd-b63d-43b7-8159-80cd1abfab55-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jscrz\" (UID: \"8bb804bd-b63d-43b7-8159-80cd1abfab55\") " pod="openshift-insights/insights-runtime-extractor-jscrz"
Jul 27 19:29:35.372935 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:35.372793    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8bb804bd-b63d-43b7-8159-80cd1abfab55-data-volume\") pod \"insights-runtime-extractor-jscrz\" (UID: \"8bb804bd-b63d-43b7-8159-80cd1abfab55\") " pod="openshift-insights/insights-runtime-extractor-jscrz"
Jul 27 19:29:35.372935 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:35.372847    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8bb804bd-b63d-43b7-8159-80cd1abfab55-crio-socket\") pod \"insights-runtime-extractor-jscrz\" (UID: \"8bb804bd-b63d-43b7-8159-80cd1abfab55\") " pod="openshift-insights/insights-runtime-extractor-jscrz"
Jul 27 19:29:35.372935 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:35.372896    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtv72\" (UniqueName: \"kubernetes.io/projected/8bb804bd-b63d-43b7-8159-80cd1abfab55-kube-api-access-qtv72\") pod \"insights-runtime-extractor-jscrz\" (UID: \"8bb804bd-b63d-43b7-8159-80cd1abfab55\") " pod="openshift-insights/insights-runtime-extractor-jscrz"
Jul 27 19:29:35.423952 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:35.423920    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:35.473235 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:35.473184    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8bb804bd-b63d-43b7-8159-80cd1abfab55-data-volume\") pod \"insights-runtime-extractor-jscrz\" (UID: \"8bb804bd-b63d-43b7-8159-80cd1abfab55\") " pod="openshift-insights/insights-runtime-extractor-jscrz"
Jul 27 19:29:35.473437 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:35.473244    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8bb804bd-b63d-43b7-8159-80cd1abfab55-crio-socket\") pod \"insights-runtime-extractor-jscrz\" (UID: \"8bb804bd-b63d-43b7-8159-80cd1abfab55\") " pod="openshift-insights/insights-runtime-extractor-jscrz"
Jul 27 19:29:35.473437 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:35.473280    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtv72\" (UniqueName: \"kubernetes.io/projected/8bb804bd-b63d-43b7-8159-80cd1abfab55-kube-api-access-qtv72\") pod \"insights-runtime-extractor-jscrz\" (UID: \"8bb804bd-b63d-43b7-8159-80cd1abfab55\") " pod="openshift-insights/insights-runtime-extractor-jscrz"
Jul 27 19:29:35.473437 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:35.473340    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8bb804bd-b63d-43b7-8159-80cd1abfab55-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jscrz\" (UID: \"8bb804bd-b63d-43b7-8159-80cd1abfab55\") " pod="openshift-insights/insights-runtime-extractor-jscrz"
Jul 27 19:29:35.473437 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:35.473378    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8bb804bd-b63d-43b7-8159-80cd1abfab55-crio-socket\") pod \"insights-runtime-extractor-jscrz\" (UID: \"8bb804bd-b63d-43b7-8159-80cd1abfab55\") " pod="openshift-insights/insights-runtime-extractor-jscrz"
Jul 27 19:29:35.473437 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:35.473384    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8bb804bd-b63d-43b7-8159-80cd1abfab55-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jscrz\" (UID: \"8bb804bd-b63d-43b7-8159-80cd1abfab55\") " pod="openshift-insights/insights-runtime-extractor-jscrz"
Jul 27 19:29:35.473696 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:29:35.473499    2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 27 19:29:35.473696 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:29:35.473525    2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 27 19:29:35.473696 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:29:35.473567    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8bb804bd-b63d-43b7-8159-80cd1abfab55-insights-runtime-extractor-tls podName:8bb804bd-b63d-43b7-8159-80cd1abfab55 nodeName:}" failed. No retries permitted until 2026-07-27 19:29:35.97354824 +0000 UTC m=+188.845740207 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8bb804bd-b63d-43b7-8159-80cd1abfab55-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jscrz" (UID: "8bb804bd-b63d-43b7-8159-80cd1abfab55") : secret "insights-runtime-extractor-tls" not found
Jul 27 19:29:35.473696 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:29:35.473592    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8bb804bd-b63d-43b7-8159-80cd1abfab55-kube-rbac-proxy-cm podName:8bb804bd-b63d-43b7-8159-80cd1abfab55 nodeName:}" failed. No retries permitted until 2026-07-27 19:29:35.973574119 +0000 UTC m=+188.845766088 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8bb804bd-b63d-43b7-8159-80cd1abfab55-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jscrz" (UID: "8bb804bd-b63d-43b7-8159-80cd1abfab55") : configmap "kube-rbac-proxy" not found
Jul 27 19:29:35.473696 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:35.473662    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8bb804bd-b63d-43b7-8159-80cd1abfab55-data-volume\") pod \"insights-runtime-extractor-jscrz\" (UID: \"8bb804bd-b63d-43b7-8159-80cd1abfab55\") " pod="openshift-insights/insights-runtime-extractor-jscrz"
Jul 27 19:29:35.481012 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:35.480991    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtv72\" (UniqueName: \"kubernetes.io/projected/8bb804bd-b63d-43b7-8159-80cd1abfab55-kube-api-access-qtv72\") pod \"insights-runtime-extractor-jscrz\" (UID: \"8bb804bd-b63d-43b7-8159-80cd1abfab55\") " pod="openshift-insights/insights-runtime-extractor-jscrz"
Jul 27 19:29:35.621358 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:35.621313    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:35.699028 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:35.698986    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:35.978116 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:35.978021    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8bb804bd-b63d-43b7-8159-80cd1abfab55-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jscrz\" (UID: \"8bb804bd-b63d-43b7-8159-80cd1abfab55\") " pod="openshift-insights/insights-runtime-extractor-jscrz"
Jul 27 19:29:35.978116 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:35.978079    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8bb804bd-b63d-43b7-8159-80cd1abfab55-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jscrz\" (UID: \"8bb804bd-b63d-43b7-8159-80cd1abfab55\") " pod="openshift-insights/insights-runtime-extractor-jscrz"
Jul 27 19:29:35.978311 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:29:35.978157    2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 27 19:29:35.978311 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:29:35.978183    2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 27 19:29:35.978311 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:29:35.978223    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8bb804bd-b63d-43b7-8159-80cd1abfab55-kube-rbac-proxy-cm podName:8bb804bd-b63d-43b7-8159-80cd1abfab55 nodeName:}" failed. No retries permitted until 2026-07-27 19:29:36.978203191 +0000 UTC m=+189.850395159 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8bb804bd-b63d-43b7-8159-80cd1abfab55-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jscrz" (UID: "8bb804bd-b63d-43b7-8159-80cd1abfab55") : configmap "kube-rbac-proxy" not found
Jul 27 19:29:35.978311 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:29:35.978240    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8bb804bd-b63d-43b7-8159-80cd1abfab55-insights-runtime-extractor-tls podName:8bb804bd-b63d-43b7-8159-80cd1abfab55 nodeName:}" failed. No retries permitted until 2026-07-27 19:29:36.978231635 +0000 UTC m=+189.850423587 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8bb804bd-b63d-43b7-8159-80cd1abfab55-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jscrz" (UID: "8bb804bd-b63d-43b7-8159-80cd1abfab55") : secret "insights-runtime-extractor-tls" not found
Jul 27 19:29:36.062581 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:36.062550    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:36.327584 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:36.327489    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:36.427532 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:36.427502    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:36.523386 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:36.523349    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:36.984949 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:36.984908    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8bb804bd-b63d-43b7-8159-80cd1abfab55-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jscrz\" (UID: \"8bb804bd-b63d-43b7-8159-80cd1abfab55\") " pod="openshift-insights/insights-runtime-extractor-jscrz"
Jul 27 19:29:36.984949 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:36.984959    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8bb804bd-b63d-43b7-8159-80cd1abfab55-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jscrz\" (UID: \"8bb804bd-b63d-43b7-8159-80cd1abfab55\") " pod="openshift-insights/insights-runtime-extractor-jscrz"
Jul 27 19:29:36.985170 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:29:36.985054    2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 27 19:29:36.985170 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:29:36.985064    2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 27 19:29:36.985170 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:29:36.985113    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8bb804bd-b63d-43b7-8159-80cd1abfab55-kube-rbac-proxy-cm podName:8bb804bd-b63d-43b7-8159-80cd1abfab55 nodeName:}" failed. No retries permitted until 2026-07-27 19:29:38.985098251 +0000 UTC m=+191.857290205 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8bb804bd-b63d-43b7-8159-80cd1abfab55-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jscrz" (UID: "8bb804bd-b63d-43b7-8159-80cd1abfab55") : configmap "kube-rbac-proxy" not found
Jul 27 19:29:36.985170 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:29:36.985128    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8bb804bd-b63d-43b7-8159-80cd1abfab55-insights-runtime-extractor-tls podName:8bb804bd-b63d-43b7-8159-80cd1abfab55 nodeName:}" failed. No retries permitted until 2026-07-27 19:29:38.985121678 +0000 UTC m=+191.857313629 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8bb804bd-b63d-43b7-8159-80cd1abfab55-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jscrz" (UID: "8bb804bd-b63d-43b7-8159-80cd1abfab55") : secret "insights-runtime-extractor-tls" not found
Jul 27 19:29:37.475188 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:37.475158    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-bnrsg_d48f4d00-ad6d-4b8e-9a57-de8ed6729c98/dns-node-resolver/0.log"
Jul 27 19:29:37.477404 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:37.477379    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:37.578753 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:37.578702    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:37.925024 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:37.924989    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-885p8"]
Jul 27 19:29:37.927193 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:37.927177    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-885p8"
Jul 27 19:29:37.929930 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:37.929913    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 27 19:29:37.930850 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:37.930830    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-5mnxt\""
Jul 27 19:29:37.930958 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:37.930883    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 27 19:29:37.930958 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:37.930904    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 27 19:29:37.931072 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:37.930998    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 27 19:29:37.934498 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:37.934477    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-885p8"]
Jul 27 19:29:37.955541 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:37.955523    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:37.993913 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:37.993892    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/e18db0dc-f8c4-4222-9de2-4fca90fc7883-signing-cabundle\") pod \"service-ca-658d7b7c78-885p8\" (UID: \"e18db0dc-f8c4-4222-9de2-4fca90fc7883\") " pod="openshift-service-ca/service-ca-658d7b7c78-885p8"
Jul 27 19:29:37.994013 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:37.993920    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rb5t5\" (UniqueName: \"kubernetes.io/projected/e18db0dc-f8c4-4222-9de2-4fca90fc7883-kube-api-access-rb5t5\") pod \"service-ca-658d7b7c78-885p8\" (UID: \"e18db0dc-f8c4-4222-9de2-4fca90fc7883\") " pod="openshift-service-ca/service-ca-658d7b7c78-885p8"
Jul 27 19:29:37.994013 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:37.993960    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/e18db0dc-f8c4-4222-9de2-4fca90fc7883-signing-key\") pod \"service-ca-658d7b7c78-885p8\" (UID: \"e18db0dc-f8c4-4222-9de2-4fca90fc7883\") " pod="openshift-service-ca/service-ca-658d7b7c78-885p8"
Jul 27 19:29:38.011450 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:38.011429    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:38.094553 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:38.094524    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/e18db0dc-f8c4-4222-9de2-4fca90fc7883-signing-key\") pod \"service-ca-658d7b7c78-885p8\" (UID: \"e18db0dc-f8c4-4222-9de2-4fca90fc7883\") " pod="openshift-service-ca/service-ca-658d7b7c78-885p8"
Jul 27 19:29:38.094658 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:38.094624    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/e18db0dc-f8c4-4222-9de2-4fca90fc7883-signing-cabundle\") pod \"service-ca-658d7b7c78-885p8\" (UID: \"e18db0dc-f8c4-4222-9de2-4fca90fc7883\") " pod="openshift-service-ca/service-ca-658d7b7c78-885p8"
Jul 27 19:29:38.094719 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:38.094654    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rb5t5\" (UniqueName: \"kubernetes.io/projected/e18db0dc-f8c4-4222-9de2-4fca90fc7883-kube-api-access-rb5t5\") pod \"service-ca-658d7b7c78-885p8\" (UID: \"e18db0dc-f8c4-4222-9de2-4fca90fc7883\") " pod="openshift-service-ca/service-ca-658d7b7c78-885p8"
Jul 27 19:29:38.095195 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:38.095177    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/e18db0dc-f8c4-4222-9de2-4fca90fc7883-signing-cabundle\") pod \"service-ca-658d7b7c78-885p8\" (UID: \"e18db0dc-f8c4-4222-9de2-4fca90fc7883\") " pod="openshift-service-ca/service-ca-658d7b7c78-885p8"
Jul 27 19:29:38.096955 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:38.096940    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/e18db0dc-f8c4-4222-9de2-4fca90fc7883-signing-key\") pod \"service-ca-658d7b7c78-885p8\" (UID: \"e18db0dc-f8c4-4222-9de2-4fca90fc7883\") " pod="openshift-service-ca/service-ca-658d7b7c78-885p8"
Jul 27 19:29:38.102246 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:38.102228    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rb5t5\" (UniqueName: \"kubernetes.io/projected/e18db0dc-f8c4-4222-9de2-4fca90fc7883-kube-api-access-rb5t5\") pod \"service-ca-658d7b7c78-885p8\" (UID: \"e18db0dc-f8c4-4222-9de2-4fca90fc7883\") " pod="openshift-service-ca/service-ca-658d7b7c78-885p8"
Jul 27 19:29:38.236071 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:38.236003    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-885p8"
Jul 27 19:29:38.274875 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:38.274852    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qljqz_e0267f96-3a9d-45d1-b943-92899defe086/node-ca/0.log"
Jul 27 19:29:38.350118 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:38.350089    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-885p8"]
Jul 27 19:29:38.353515 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:29:38.353489    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode18db0dc_f8c4_4222_9de2_4fca90fc7883.slice/crio-a60a6bb24fa1691c6632b4659d8ae79d0079a3f78d3a8b0dd1ab9cb2ac2675c7 WatchSource:0}: Error finding container a60a6bb24fa1691c6632b4659d8ae79d0079a3f78d3a8b0dd1ab9cb2ac2675c7: Status 404 returned error can't find the container with id a60a6bb24fa1691c6632b4659d8ae79d0079a3f78d3a8b0dd1ab9cb2ac2675c7
Jul 27 19:29:38.874734 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:38.874708    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-9r7sw_6e0976f8-4dd4-4fb2-8458-b426a96a105c/serve-healthcheck-canary/0.log"
Jul 27 19:29:38.883049 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:38.883027    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:39.001387 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:39.001342    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8bb804bd-b63d-43b7-8159-80cd1abfab55-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jscrz\" (UID: \"8bb804bd-b63d-43b7-8159-80cd1abfab55\") " pod="openshift-insights/insights-runtime-extractor-jscrz"
Jul 27 19:29:39.001544 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:39.001403    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8bb804bd-b63d-43b7-8159-80cd1abfab55-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jscrz\" (UID: \"8bb804bd-b63d-43b7-8159-80cd1abfab55\") " pod="openshift-insights/insights-runtime-extractor-jscrz"
Jul 27 19:29:39.001544 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:29:39.001507    2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 27 19:29:39.001544 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:29:39.001529    2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 27 19:29:39.001639 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:29:39.001574    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8bb804bd-b63d-43b7-8159-80cd1abfab55-insights-runtime-extractor-tls podName:8bb804bd-b63d-43b7-8159-80cd1abfab55 nodeName:}" failed. No retries permitted until 2026-07-27 19:29:43.001553022 +0000 UTC m=+195.873744974 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8bb804bd-b63d-43b7-8159-80cd1abfab55-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jscrz" (UID: "8bb804bd-b63d-43b7-8159-80cd1abfab55") : secret "insights-runtime-extractor-tls" not found
Jul 27 19:29:39.001639 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:29:39.001589    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8bb804bd-b63d-43b7-8159-80cd1abfab55-kube-rbac-proxy-cm podName:8bb804bd-b63d-43b7-8159-80cd1abfab55 nodeName:}" failed. No retries permitted until 2026-07-27 19:29:43.001582853 +0000 UTC m=+195.873774804 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8bb804bd-b63d-43b7-8159-80cd1abfab55-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jscrz" (UID: "8bb804bd-b63d-43b7-8159-80cd1abfab55") : configmap "kube-rbac-proxy" not found
Jul 27 19:29:39.211968 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:39.211884    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-885p8" event={"ID":"e18db0dc-f8c4-4222-9de2-4fca90fc7883","Type":"ContainerStarted","Data":"a60a6bb24fa1691c6632b4659d8ae79d0079a3f78d3a8b0dd1ab9cb2ac2675c7"}
Jul 27 19:29:39.909162 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:39.909130    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6-registry-tls\") pod \"image-registry-7f4bbb7d8b-kjzcn\" (UID: \"a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6\") " pod="openshift-image-registry/image-registry-7f4bbb7d8b-kjzcn"
Jul 27 19:29:39.909553 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:29:39.909238    2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 27 19:29:39.909553 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:29:39.909250    2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f4bbb7d8b-kjzcn: secret "image-registry-tls" not found
Jul 27 19:29:39.909553 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:29:39.909299    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6-registry-tls podName:a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6 nodeName:}" failed. No retries permitted until 2026-07-27 19:29:55.909286136 +0000 UTC m=+208.781478092 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6-registry-tls") pod "image-registry-7f4bbb7d8b-kjzcn" (UID: "a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6") : secret "image-registry-tls" not found
Jul 27 19:29:40.010346 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:40.010315    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/219afe94-ca1b-4a80-a421-e6cb1bb321b1-metrics-tls\") pod \"dns-default-4rxwd\" (UID: \"219afe94-ca1b-4a80-a421-e6cb1bb321b1\") " pod="openshift-dns/dns-default-4rxwd"
Jul 27 19:29:40.010521 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:29:40.010449    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 19:29:40.010521 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:29:40.010505    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/219afe94-ca1b-4a80-a421-e6cb1bb321b1-metrics-tls podName:219afe94-ca1b-4a80-a421-e6cb1bb321b1 nodeName:}" failed. No retries permitted until 2026-07-27 19:29:56.010490127 +0000 UTC m=+208.882682080 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/219afe94-ca1b-4a80-a421-e6cb1bb321b1-metrics-tls") pod "dns-default-4rxwd" (UID: "219afe94-ca1b-4a80-a421-e6cb1bb321b1") : secret "dns-default-metrics-tls" not found
Jul 27 19:29:41.218293 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:41.218254    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-885p8" event={"ID":"e18db0dc-f8c4-4222-9de2-4fca90fc7883","Type":"ContainerStarted","Data":"df37e5002a29d5badc1e7694b7276fe6496c8283d99c4564ce6bf6efdee37c47"}
Jul 27 19:29:41.234850 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:41.234800    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-658d7b7c78-885p8" podStartSLOduration=2.097759378 podStartE2EDuration="4.234784857s" totalImagesPullingTime="2.137025479s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:29:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:29:41.233807415 +0000 UTC m=+194.105999389" watchObservedRunningTime="2026-07-27 19:29:41.234784857 +0000 UTC m=+194.106976830"
Jul 27 19:29:41.755559 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:41.755521    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:41.999396 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:41.999363    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:42.146252 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:42.146202    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:42.246749 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:42.246713    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:42.347838 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:42.347807    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:42.407942 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:42.407856    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:42.470207 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:42.470175    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:43.037699 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:43.037667    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8bb804bd-b63d-43b7-8159-80cd1abfab55-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jscrz\" (UID: \"8bb804bd-b63d-43b7-8159-80cd1abfab55\") " pod="openshift-insights/insights-runtime-extractor-jscrz"
Jul 27 19:29:43.037890 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:43.037716    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8bb804bd-b63d-43b7-8159-80cd1abfab55-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jscrz\" (UID: \"8bb804bd-b63d-43b7-8159-80cd1abfab55\") " pod="openshift-insights/insights-runtime-extractor-jscrz"
Jul 27 19:29:43.037890 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:29:43.037803    2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 27 19:29:43.037890 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:29:43.037832    2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 27 19:29:43.037890 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:29:43.037870    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8bb804bd-b63d-43b7-8159-80cd1abfab55-kube-rbac-proxy-cm podName:8bb804bd-b63d-43b7-8159-80cd1abfab55 nodeName:}" failed. No retries permitted until 2026-07-27 19:29:51.037856366 +0000 UTC m=+203.910048317 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8bb804bd-b63d-43b7-8159-80cd1abfab55-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jscrz" (UID: "8bb804bd-b63d-43b7-8159-80cd1abfab55") : configmap "kube-rbac-proxy" not found
Jul 27 19:29:43.038052 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:29:43.037894    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8bb804bd-b63d-43b7-8159-80cd1abfab55-insights-runtime-extractor-tls podName:8bb804bd-b63d-43b7-8159-80cd1abfab55 nodeName:}" failed. No retries permitted until 2026-07-27 19:29:51.037877056 +0000 UTC m=+203.910069009 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8bb804bd-b63d-43b7-8159-80cd1abfab55-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jscrz" (UID: "8bb804bd-b63d-43b7-8159-80cd1abfab55") : secret "insights-runtime-extractor-tls" not found
Jul 27 19:29:43.054425 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:43.054388    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:43.259544 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:43.259508    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:43.440917 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:43.440881    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/229036e0-7551-4987-98d3-0ccef9c511e5-metrics-certs\") pod \"network-metrics-daemon-zbjtr\" (UID: \"229036e0-7551-4987-98d3-0ccef9c511e5\") " pod="openshift-multus/network-metrics-daemon-zbjtr"
Jul 27 19:29:43.441084 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:29:43.441041    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 27 19:29:43.441143 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:29:43.441106    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/229036e0-7551-4987-98d3-0ccef9c511e5-metrics-certs podName:229036e0-7551-4987-98d3-0ccef9c511e5 nodeName:}" failed. No retries permitted until 2026-07-27 19:31:45.441091012 +0000 UTC m=+318.313282964 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/229036e0-7551-4987-98d3-0ccef9c511e5-metrics-certs") pod "network-metrics-daemon-zbjtr" (UID: "229036e0-7551-4987-98d3-0ccef9c511e5") : secret "metrics-daemon-secret" not found
Jul 27 19:29:44.267029 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:44.266995    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:44.456987 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:44.456957    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:44.652859 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:44.652816    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:45.056827 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:45.056748    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:45.681953 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:45.681916    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:45.768598 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:45.768563    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:46.060472 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:46.060379    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:46.453547 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:46.453487    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:46.652540 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:46.652508    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:47.064189 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:47.064158    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:47.172082 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:47.172049    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:47.271620 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:47.271589    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:47.371991 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:47.371958    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:47.658710 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:47.658626    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:47.768254 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:47.768222    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:48.109188 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:48.109081    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:48.260129 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:48.260099    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:48.532608 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:48.532578    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:48.650181 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:48.650151    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:49.469044 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:49.469014    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:49.875028 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:49.874999    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:50.070169 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:50.070136    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:50.273693 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:50.273612    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:50.487815 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:50.487779    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:51.100873 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:51.100844    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8bb804bd-b63d-43b7-8159-80cd1abfab55-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jscrz\" (UID: \"8bb804bd-b63d-43b7-8159-80cd1abfab55\") " pod="openshift-insights/insights-runtime-extractor-jscrz"
Jul 27 19:29:51.101094 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:51.100891    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8bb804bd-b63d-43b7-8159-80cd1abfab55-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jscrz\" (UID: \"8bb804bd-b63d-43b7-8159-80cd1abfab55\") " pod="openshift-insights/insights-runtime-extractor-jscrz"
Jul 27 19:29:51.101094 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:29:51.101014    2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 27 19:29:51.101201 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:29:51.101109    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8bb804bd-b63d-43b7-8159-80cd1abfab55-insights-runtime-extractor-tls podName:8bb804bd-b63d-43b7-8159-80cd1abfab55 nodeName:}" failed. No retries permitted until 2026-07-27 19:30:07.101086991 +0000 UTC m=+219.973278956 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8bb804bd-b63d-43b7-8159-80cd1abfab55-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jscrz" (UID: "8bb804bd-b63d-43b7-8159-80cd1abfab55") : secret "insights-runtime-extractor-tls" not found
Jul 27 19:29:51.101372 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:51.101356    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8bb804bd-b63d-43b7-8159-80cd1abfab55-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jscrz\" (UID: \"8bb804bd-b63d-43b7-8159-80cd1abfab55\") " pod="openshift-insights/insights-runtime-extractor-jscrz"
Jul 27 19:29:51.231508 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:51.231476    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:51.458424 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:51.458385    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:51.658640 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:51.658610    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:52.055102 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:52.055059    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:52.266337 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:52.266307    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:52.456046 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:52.456019    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:55.805019 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:55.804989    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:55.881030 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:55.880997    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:55.935139 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:55.935113    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6-registry-tls\") pod \"image-registry-7f4bbb7d8b-kjzcn\" (UID: \"a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6\") " pod="openshift-image-registry/image-registry-7f4bbb7d8b-kjzcn"
Jul 27 19:29:55.937745 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:55.937712    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6-registry-tls\") pod \"image-registry-7f4bbb7d8b-kjzcn\" (UID: \"a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6\") " pod="openshift-image-registry/image-registry-7f4bbb7d8b-kjzcn"
Jul 27 19:29:56.036213 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:56.036178    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/219afe94-ca1b-4a80-a421-e6cb1bb321b1-metrics-tls\") pod \"dns-default-4rxwd\" (UID: \"219afe94-ca1b-4a80-a421-e6cb1bb321b1\") " pod="openshift-dns/dns-default-4rxwd"
Jul 27 19:29:56.038629 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:56.038603    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/219afe94-ca1b-4a80-a421-e6cb1bb321b1-metrics-tls\") pod \"dns-default-4rxwd\" (UID: \"219afe94-ca1b-4a80-a421-e6cb1bb321b1\") " pod="openshift-dns/dns-default-4rxwd"
Jul 27 19:29:56.227443 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:56.227399    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-txhl6\""
Jul 27 19:29:56.235891 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:56.235873    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f4bbb7d8b-kjzcn"
Jul 27 19:29:56.238102 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:56.238083    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-nht7b\""
Jul 27 19:29:56.246906 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:56.246882    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4rxwd"
Jul 27 19:29:56.369811 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:56.369732    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7f4bbb7d8b-kjzcn"]
Jul 27 19:29:56.372544 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:29:56.372519    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3d3ea53_7b03_4acf_ae92_d4a7d762dbf6.slice/crio-c5e9cdbd95b9e6dbf015117436a93f4609d7ada88637f522cd95f7b2c60eaa7a WatchSource:0}: Error finding container c5e9cdbd95b9e6dbf015117436a93f4609d7ada88637f522cd95f7b2c60eaa7a: Status 404 returned error can't find the container with id c5e9cdbd95b9e6dbf015117436a93f4609d7ada88637f522cd95f7b2c60eaa7a
Jul 27 19:29:56.387338 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:56.387314    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-4rxwd"]
Jul 27 19:29:56.391154 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:29:56.391128    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod219afe94_ca1b_4a80_a421_e6cb1bb321b1.slice/crio-b2702451c6e1d023dca7b506340afbb6fcb754d4620210f8399fe87410d428c4 WatchSource:0}: Error finding container b2702451c6e1d023dca7b506340afbb6fcb754d4620210f8399fe87410d428c4: Status 404 returned error can't find the container with id b2702451c6e1d023dca7b506340afbb6fcb754d4620210f8399fe87410d428c4
Jul 27 19:29:57.262960 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:57.262887    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7f4bbb7d8b-kjzcn" event={"ID":"a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6","Type":"ContainerStarted","Data":"5eea4f12c152892d9b493303d402ddf619f10dc681341f350c6a1baeebd09083"}
Jul 27 19:29:57.262960 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:57.262962    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7f4bbb7d8b-kjzcn"
Jul 27 19:29:57.263537 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:57.262984    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7f4bbb7d8b-kjzcn" event={"ID":"a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6","Type":"ContainerStarted","Data":"c5e9cdbd95b9e6dbf015117436a93f4609d7ada88637f522cd95f7b2c60eaa7a"}
Jul 27 19:29:57.264115 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:57.264092    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4rxwd" event={"ID":"219afe94-ca1b-4a80-a421-e6cb1bb321b1","Type":"ContainerStarted","Data":"b2702451c6e1d023dca7b506340afbb6fcb754d4620210f8399fe87410d428c4"}
Jul 27 19:29:57.284027 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:57.283978    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7f4bbb7d8b-kjzcn" podStartSLOduration=132.283963668 podStartE2EDuration="2m12.283963668s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:27:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:29:57.282309467 +0000 UTC m=+210.154501441" watchObservedRunningTime="2026-07-27 19:29:57.283963668 +0000 UTC m=+210.156155643"
Jul 27 19:29:58.100748 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:58.100722    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:58.201226 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:58.201145    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:58.266951 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:58.266925    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:58.268874 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:58.268852    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4rxwd" event={"ID":"219afe94-ca1b-4a80-a421-e6cb1bb321b1","Type":"ContainerStarted","Data":"b7b0b4f20d98857433996cb37895ccebf0aef8f84fb0298ffb98c2239302a481"}
Jul 27 19:29:58.268973 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:58.268899    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4rxwd" event={"ID":"219afe94-ca1b-4a80-a421-e6cb1bb321b1","Type":"ContainerStarted","Data":"884e273a5e8a5cc11d2072609cbcbe570cb7e7e1e943bb43f897d506d81eaeb5"}
Jul 27 19:29:58.287051 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:58.287012    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-4rxwd" podStartSLOduration=32.753238095 podStartE2EDuration="34.286999645s" totalImagesPullingTime="1.53376155s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:29:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:29:58.286359282 +0000 UTC m=+211.158551257" watchObservedRunningTime="2026-07-27 19:29:58.286999645 +0000 UTC m=+211.159191618"
Jul 27 19:29:58.417493 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:58.417452    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:58.500809 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:58.500739    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-7c97dc4555-qm44q"]
Jul 27 19:29:58.502864 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:58.502847    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-qm44q"
Jul 27 19:29:58.505660 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:58.505624    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 27 19:29:58.505771 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:58.505697    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-gjgzv\""
Jul 27 19:29:58.505826 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:58.505631    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 27 19:29:58.514361 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:58.514336    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-qm44q"]
Jul 27 19:29:58.559460 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:58.559433    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkff8\" (UniqueName: \"kubernetes.io/projected/212c42c2-b2a1-4082-8ea8-2a7f867bd45a-kube-api-access-rkff8\") pod \"downloads-7c97dc4555-qm44q\" (UID: \"212c42c2-b2a1-4082-8ea8-2a7f867bd45a\") " pod="openshift-console/downloads-7c97dc4555-qm44q"
Jul 27 19:29:58.569058 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:58.569030    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:58.659897 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:58.659869    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkff8\" (UniqueName: \"kubernetes.io/projected/212c42c2-b2a1-4082-8ea8-2a7f867bd45a-kube-api-access-rkff8\") pod \"downloads-7c97dc4555-qm44q\" (UID: \"212c42c2-b2a1-4082-8ea8-2a7f867bd45a\") " pod="openshift-console/downloads-7c97dc4555-qm44q"
Jul 27 19:29:58.669693 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:58.669671    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:58.672164 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:58.672135    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkff8\" (UniqueName: \"kubernetes.io/projected/212c42c2-b2a1-4082-8ea8-2a7f867bd45a-kube-api-access-rkff8\") pod \"downloads-7c97dc4555-qm44q\" (UID: \"212c42c2-b2a1-4082-8ea8-2a7f867bd45a\") " pod="openshift-console/downloads-7c97dc4555-qm44q"
Jul 27 19:29:58.791349 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:58.791271    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:58.815075 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:58.815042    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-qm44q"
Jul 27 19:29:58.939618 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:58.939395    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-qm44q"]
Jul 27 19:29:58.942112 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:29:58.942087    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod212c42c2_b2a1_4082_8ea8_2a7f867bd45a.slice/crio-9d3085faf4334ae0ba1465ffb9fc2654e76ab02e31c3d6c02b3da55b8ab72fa2 WatchSource:0}: Error finding container 9d3085faf4334ae0ba1465ffb9fc2654e76ab02e31c3d6c02b3da55b8ab72fa2: Status 404 returned error can't find the container with id 9d3085faf4334ae0ba1465ffb9fc2654e76ab02e31c3d6c02b3da55b8ab72fa2
Jul 27 19:29:58.970603 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:58.970577    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:59.074301 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:59.074215    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:59.202980 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:59.202952    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:59.269532 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:59.269488    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:59.272505 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:59.272474    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-qm44q" event={"ID":"212c42c2-b2a1-4082-8ea8-2a7f867bd45a","Type":"ContainerStarted","Data":"9d3085faf4334ae0ba1465ffb9fc2654e76ab02e31c3d6c02b3da55b8ab72fa2"}
Jul 27 19:29:59.272631 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:59.272617    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-4rxwd"
Jul 27 19:29:59.999630 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:29:59.999599    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:00.192247 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:00.192216    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:00.292692 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:00.292610    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:00.674138 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:00.674100    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:00.789491 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:00.789455    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:00.881736 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:00.881700    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:01.222369 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:01.222338    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:01.324759 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:01.324720    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:01.497934 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:01.497835    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:01.596618 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:01.596581    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:01.792923 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:01.792835    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:04.826055 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:04.826023    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:04.996644 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:04.996595    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:05.242649 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:05.242594    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:07.128941 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:07.128902    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8bb804bd-b63d-43b7-8159-80cd1abfab55-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jscrz\" (UID: \"8bb804bd-b63d-43b7-8159-80cd1abfab55\") " pod="openshift-insights/insights-runtime-extractor-jscrz"
Jul 27 19:30:07.131471 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:07.131444    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8bb804bd-b63d-43b7-8159-80cd1abfab55-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jscrz\" (UID: \"8bb804bd-b63d-43b7-8159-80cd1abfab55\") " pod="openshift-insights/insights-runtime-extractor-jscrz"
Jul 27 19:30:07.387520 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:07.387428    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jscrz"
Jul 27 19:30:07.522987 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:07.522818    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jscrz"]
Jul 27 19:30:07.525838 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:30:07.525798    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bb804bd_b63d_43b7_8159_80cd1abfab55.slice/crio-e37513134843d8dea48855dcfd197587c10f3738f4f4a63fcc89b985b9527bc1 WatchSource:0}: Error finding container e37513134843d8dea48855dcfd197587c10f3738f4f4a63fcc89b985b9527bc1: Status 404 returned error can't find the container with id e37513134843d8dea48855dcfd197587c10f3738f4f4a63fcc89b985b9527bc1
Jul 27 19:30:07.997026 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:07.996956    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:08.321479 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:08.321379    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jscrz" event={"ID":"8bb804bd-b63d-43b7-8159-80cd1abfab55","Type":"ContainerStarted","Data":"e37513134843d8dea48855dcfd197587c10f3738f4f4a63fcc89b985b9527bc1"}
Jul 27 19:30:08.419850 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:08.418847    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-564f6df4f-d4kcc"]
Jul 27 19:30:08.422113 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:08.422074    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-564f6df4f-d4kcc"
Jul 27 19:30:08.425801 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:08.425601    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 27 19:30:08.425801 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:08.425630    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-bxqs5\""
Jul 27 19:30:08.425801 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:08.425694    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 27 19:30:08.426041 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:08.425905    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 27 19:30:08.426041 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:08.425965    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 27 19:30:08.426157 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:08.426096    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 27 19:30:08.432405 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:08.431285    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-564f6df4f-d4kcc"]
Jul 27 19:30:08.539775 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:08.539682    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f-console-config\") pod \"console-564f6df4f-d4kcc\" (UID: \"8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f\") " pod="openshift-console/console-564f6df4f-d4kcc"
Jul 27 19:30:08.539775 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:08.539737    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f-console-serving-cert\") pod \"console-564f6df4f-d4kcc\" (UID: \"8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f\") " pod="openshift-console/console-564f6df4f-d4kcc"
Jul 27 19:30:08.540034 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:08.539802    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f-console-oauth-config\") pod \"console-564f6df4f-d4kcc\" (UID: \"8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f\") " pod="openshift-console/console-564f6df4f-d4kcc"
Jul 27 19:30:08.540034 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:08.539840    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f-oauth-serving-cert\") pod \"console-564f6df4f-d4kcc\" (UID: \"8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f\") " pod="openshift-console/console-564f6df4f-d4kcc"
Jul 27 19:30:08.540034 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:08.539948    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6k2kt\" (UniqueName: \"kubernetes.io/projected/8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f-kube-api-access-6k2kt\") pod \"console-564f6df4f-d4kcc\" (UID: \"8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f\") " pod="openshift-console/console-564f6df4f-d4kcc"
Jul 27 19:30:08.540034 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:08.540005    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f-service-ca\") pod \"console-564f6df4f-d4kcc\" (UID: \"8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f\") " pod="openshift-console/console-564f6df4f-d4kcc"
Jul 27 19:30:08.641108 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:08.641059    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6k2kt\" (UniqueName: \"kubernetes.io/projected/8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f-kube-api-access-6k2kt\") pod \"console-564f6df4f-d4kcc\" (UID: \"8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f\") " pod="openshift-console/console-564f6df4f-d4kcc"
Jul 27 19:30:08.641305 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:08.641130    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f-service-ca\") pod \"console-564f6df4f-d4kcc\" (UID: \"8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f\") " pod="openshift-console/console-564f6df4f-d4kcc"
Jul 27 19:30:08.641305 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:08.641193    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f-console-config\") pod \"console-564f6df4f-d4kcc\" (UID: \"8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f\") " pod="openshift-console/console-564f6df4f-d4kcc"
Jul 27 19:30:08.641305 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:08.641229    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f-console-serving-cert\") pod \"console-564f6df4f-d4kcc\" (UID: \"8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f\") " pod="openshift-console/console-564f6df4f-d4kcc"
Jul 27 19:30:08.641305 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:08.641252    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f-console-oauth-config\") pod \"console-564f6df4f-d4kcc\" (UID: \"8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f\") " pod="openshift-console/console-564f6df4f-d4kcc"
Jul 27 19:30:08.641305 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:08.641275    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f-oauth-serving-cert\") pod \"console-564f6df4f-d4kcc\" (UID: \"8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f\") " pod="openshift-console/console-564f6df4f-d4kcc"
Jul 27 19:30:08.642027 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:08.641996    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f-service-ca\") pod \"console-564f6df4f-d4kcc\" (UID: \"8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f\") " pod="openshift-console/console-564f6df4f-d4kcc"
Jul 27 19:30:08.642156 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:08.641997    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f-oauth-serving-cert\") pod \"console-564f6df4f-d4kcc\" (UID: \"8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f\") " pod="openshift-console/console-564f6df4f-d4kcc"
Jul 27 19:30:08.642462 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:08.642401    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f-console-config\") pod \"console-564f6df4f-d4kcc\" (UID: \"8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f\") " pod="openshift-console/console-564f6df4f-d4kcc"
Jul 27 19:30:08.644676 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:08.644654    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f-console-oauth-config\") pod \"console-564f6df4f-d4kcc\" (UID: \"8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f\") " pod="openshift-console/console-564f6df4f-d4kcc"
Jul 27 19:30:08.645355 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:08.645294    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f-console-serving-cert\") pod \"console-564f6df4f-d4kcc\" (UID: \"8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f\") " pod="openshift-console/console-564f6df4f-d4kcc"
Jul 27 19:30:08.650051 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:08.650029    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6k2kt\" (UniqueName: \"kubernetes.io/projected/8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f-kube-api-access-6k2kt\") pod \"console-564f6df4f-d4kcc\" (UID: \"8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f\") " pod="openshift-console/console-564f6df4f-d4kcc"
Jul 27 19:30:08.735942 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:08.735904    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-564f6df4f-d4kcc"
Jul 27 19:30:09.279539 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:09.279506    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-4rxwd"
Jul 27 19:30:09.603118 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:09.603031    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:09.987768 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:09.987730    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:10.174971 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.174927    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:10.198700 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.198659    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-p4c8p"]
Jul 27 19:30:10.202024 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.202003    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-p4c8p"
Jul 27 19:30:10.204677 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.204651    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 27 19:30:10.205373 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.205020    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 27 19:30:10.205373 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.205029    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 27 19:30:10.205373 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.205057    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 27 19:30:10.205373 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.205212    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-r7l9x\""
Jul 27 19:30:10.206349 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.206313    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 27 19:30:10.216946 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.216872    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-p4c8p"]
Jul 27 19:30:10.221494 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.220314    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-pwsmc"]
Jul 27 19:30:10.223737 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.223718    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-pwsmc"
Jul 27 19:30:10.226806 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.226785    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 27 19:30:10.227112 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.227095    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-fpwbx\""
Jul 27 19:30:10.227587 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.227567    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 27 19:30:10.227761 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.227745    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-b65wp"]
Jul 27 19:30:10.228283 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.228258    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 27 19:30:10.231198 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.231162    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-b65wp"
Jul 27 19:30:10.235742 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.235302    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-pwsmc"]
Jul 27 19:30:10.235742 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.235344    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 27 19:30:10.235742 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.235448    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 27 19:30:10.235742 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.235305    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-j8v9r\""
Jul 27 19:30:10.235742 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.235610    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 27 19:30:10.268533 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.268504    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:10.330614 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.330384    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:10.354336 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.354248    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4aa68886-81b2-47c7-9aef-c6d73404122f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-b65wp\" (UID: \"4aa68886-81b2-47c7-9aef-c6d73404122f\") " pod="openshift-monitoring/node-exporter-b65wp"
Jul 27 19:30:10.354336 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.354294    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6b8fj\" (UniqueName: \"kubernetes.io/projected/9b70adeb-3b2e-41f2-a157-6b2dfe797669-kube-api-access-6b8fj\") pod \"kube-state-metrics-56d9f944b4-pwsmc\" (UID: \"9b70adeb-3b2e-41f2-a157-6b2dfe797669\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-pwsmc"
Jul 27 19:30:10.354336 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.354320    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4aa68886-81b2-47c7-9aef-c6d73404122f-node-exporter-accelerators-collector-config\") pod \"node-exporter-b65wp\" (UID: \"4aa68886-81b2-47c7-9aef-c6d73404122f\") " pod="openshift-monitoring/node-exporter-b65wp"
Jul 27 19:30:10.354601 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.354354    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42nxq\" (UniqueName: \"kubernetes.io/projected/4aa68886-81b2-47c7-9aef-c6d73404122f-kube-api-access-42nxq\") pod \"node-exporter-b65wp\" (UID: \"4aa68886-81b2-47c7-9aef-c6d73404122f\") " pod="openshift-monitoring/node-exporter-b65wp"
Jul 27 19:30:10.354601 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.354384    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b70adeb-3b2e-41f2-a157-6b2dfe797669-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-pwsmc\" (UID: \"9b70adeb-3b2e-41f2-a157-6b2dfe797669\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-pwsmc"
Jul 27 19:30:10.354601 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.354440    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4aa68886-81b2-47c7-9aef-c6d73404122f-node-exporter-wtmp\") pod \"node-exporter-b65wp\" (UID: \"4aa68886-81b2-47c7-9aef-c6d73404122f\") " pod="openshift-monitoring/node-exporter-b65wp"
Jul 27 19:30:10.354601 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.354472    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4aa68886-81b2-47c7-9aef-c6d73404122f-sys\") pod \"node-exporter-b65wp\" (UID: \"4aa68886-81b2-47c7-9aef-c6d73404122f\") " pod="openshift-monitoring/node-exporter-b65wp"
Jul 27 19:30:10.354601 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.354494    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqmcj\" (UniqueName: \"kubernetes.io/projected/3a5c72a0-8769-4caf-80e4-cfc8db418c38-kube-api-access-wqmcj\") pod \"openshift-state-metrics-b8f49f5c6-p4c8p\" (UID: \"3a5c72a0-8769-4caf-80e4-cfc8db418c38\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-p4c8p"
Jul 27 19:30:10.354601 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.354522    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3a5c72a0-8769-4caf-80e4-cfc8db418c38-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-p4c8p\" (UID: \"3a5c72a0-8769-4caf-80e4-cfc8db418c38\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-p4c8p"
Jul 27 19:30:10.354601 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.354550    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9b70adeb-3b2e-41f2-a157-6b2dfe797669-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-pwsmc\" (UID: \"9b70adeb-3b2e-41f2-a157-6b2dfe797669\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-pwsmc"
Jul 27 19:30:10.355157 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.354605    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9b70adeb-3b2e-41f2-a157-6b2dfe797669-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-pwsmc\" (UID: \"9b70adeb-3b2e-41f2-a157-6b2dfe797669\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-pwsmc"
Jul 27 19:30:10.355157 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.354637    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4aa68886-81b2-47c7-9aef-c6d73404122f-metrics-client-ca\") pod \"node-exporter-b65wp\" (UID: \"4aa68886-81b2-47c7-9aef-c6d73404122f\") " pod="openshift-monitoring/node-exporter-b65wp"
Jul 27 19:30:10.355157 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.354664    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/9b70adeb-3b2e-41f2-a157-6b2dfe797669-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-pwsmc\" (UID: \"9b70adeb-3b2e-41f2-a157-6b2dfe797669\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-pwsmc"
Jul 27 19:30:10.355157 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.354701    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3a5c72a0-8769-4caf-80e4-cfc8db418c38-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-p4c8p\" (UID: \"3a5c72a0-8769-4caf-80e4-cfc8db418c38\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-p4c8p"
Jul 27 19:30:10.355157 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.354730    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4aa68886-81b2-47c7-9aef-c6d73404122f-node-exporter-textfile\") pod \"node-exporter-b65wp\" (UID: \"4aa68886-81b2-47c7-9aef-c6d73404122f\") " pod="openshift-monitoring/node-exporter-b65wp"
Jul 27 19:30:10.355157 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.354758    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4aa68886-81b2-47c7-9aef-c6d73404122f-root\") pod \"node-exporter-b65wp\" (UID: \"4aa68886-81b2-47c7-9aef-c6d73404122f\") " pod="openshift-monitoring/node-exporter-b65wp"
Jul 27 19:30:10.355157 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.354781    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a5c72a0-8769-4caf-80e4-cfc8db418c38-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-p4c8p\" (UID: \"3a5c72a0-8769-4caf-80e4-cfc8db418c38\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-p4c8p"
Jul 27 19:30:10.355157 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.354810    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4aa68886-81b2-47c7-9aef-c6d73404122f-node-exporter-tls\") pod \"node-exporter-b65wp\" (UID: \"4aa68886-81b2-47c7-9aef-c6d73404122f\") " pod="openshift-monitoring/node-exporter-b65wp"
Jul 27 19:30:10.355157 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.354863    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/9b70adeb-3b2e-41f2-a157-6b2dfe797669-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-pwsmc\" (UID: \"9b70adeb-3b2e-41f2-a157-6b2dfe797669\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-pwsmc"
Jul 27 19:30:10.456166 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.456126    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4aa68886-81b2-47c7-9aef-c6d73404122f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-b65wp\" (UID: \"4aa68886-81b2-47c7-9aef-c6d73404122f\") " pod="openshift-monitoring/node-exporter-b65wp"
Jul 27 19:30:10.456166 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.456169    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6b8fj\" (UniqueName: \"kubernetes.io/projected/9b70adeb-3b2e-41f2-a157-6b2dfe797669-kube-api-access-6b8fj\") pod \"kube-state-metrics-56d9f944b4-pwsmc\" (UID: \"9b70adeb-3b2e-41f2-a157-6b2dfe797669\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-pwsmc"
Jul 27 19:30:10.456392 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.456190    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4aa68886-81b2-47c7-9aef-c6d73404122f-node-exporter-accelerators-collector-config\") pod \"node-exporter-b65wp\" (UID: \"4aa68886-81b2-47c7-9aef-c6d73404122f\") " pod="openshift-monitoring/node-exporter-b65wp"
Jul 27 19:30:10.456392 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.456225    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42nxq\" (UniqueName: \"kubernetes.io/projected/4aa68886-81b2-47c7-9aef-c6d73404122f-kube-api-access-42nxq\") pod \"node-exporter-b65wp\" (UID: \"4aa68886-81b2-47c7-9aef-c6d73404122f\") " pod="openshift-monitoring/node-exporter-b65wp"
Jul 27 19:30:10.456392 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.456251    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b70adeb-3b2e-41f2-a157-6b2dfe797669-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-pwsmc\" (UID: \"9b70adeb-3b2e-41f2-a157-6b2dfe797669\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-pwsmc"
Jul 27 19:30:10.456392 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.456290    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4aa68886-81b2-47c7-9aef-c6d73404122f-node-exporter-wtmp\") pod \"node-exporter-b65wp\" (UID: \"4aa68886-81b2-47c7-9aef-c6d73404122f\") " pod="openshift-monitoring/node-exporter-b65wp"
Jul 27 19:30:10.456392 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.456319    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4aa68886-81b2-47c7-9aef-c6d73404122f-sys\") pod \"node-exporter-b65wp\" (UID: \"4aa68886-81b2-47c7-9aef-c6d73404122f\") " pod="openshift-monitoring/node-exporter-b65wp"
Jul 27 19:30:10.456392 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.456342    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqmcj\" (UniqueName: \"kubernetes.io/projected/3a5c72a0-8769-4caf-80e4-cfc8db418c38-kube-api-access-wqmcj\") pod \"openshift-state-metrics-b8f49f5c6-p4c8p\" (UID: \"3a5c72a0-8769-4caf-80e4-cfc8db418c38\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-p4c8p"
Jul 27 19:30:10.456392 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.456368    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3a5c72a0-8769-4caf-80e4-cfc8db418c38-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-p4c8p\" (UID: \"3a5c72a0-8769-4caf-80e4-cfc8db418c38\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-p4c8p"
Jul 27 19:30:10.457954 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.456396    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9b70adeb-3b2e-41f2-a157-6b2dfe797669-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-pwsmc\" (UID: \"9b70adeb-3b2e-41f2-a157-6b2dfe797669\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-pwsmc"
Jul 27 19:30:10.457954 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.456462    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9b70adeb-3b2e-41f2-a157-6b2dfe797669-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-pwsmc\" (UID: \"9b70adeb-3b2e-41f2-a157-6b2dfe797669\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-pwsmc"
Jul 27 19:30:10.457954 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.456492    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4aa68886-81b2-47c7-9aef-c6d73404122f-metrics-client-ca\") pod \"node-exporter-b65wp\" (UID: \"4aa68886-81b2-47c7-9aef-c6d73404122f\") " pod="openshift-monitoring/node-exporter-b65wp"
Jul 27 19:30:10.457954 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.456512    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4aa68886-81b2-47c7-9aef-c6d73404122f-sys\") pod \"node-exporter-b65wp\" (UID: \"4aa68886-81b2-47c7-9aef-c6d73404122f\") " pod="openshift-monitoring/node-exporter-b65wp"
Jul 27 19:30:10.457954 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.456520    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/9b70adeb-3b2e-41f2-a157-6b2dfe797669-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-pwsmc\" (UID: \"9b70adeb-3b2e-41f2-a157-6b2dfe797669\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-pwsmc"
Jul 27 19:30:10.457954 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.456595    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3a5c72a0-8769-4caf-80e4-cfc8db418c38-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-p4c8p\" (UID: \"3a5c72a0-8769-4caf-80e4-cfc8db418c38\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-p4c8p"
Jul 27 19:30:10.457954 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.456630    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4aa68886-81b2-47c7-9aef-c6d73404122f-node-exporter-wtmp\") pod \"node-exporter-b65wp\" (UID: \"4aa68886-81b2-47c7-9aef-c6d73404122f\") " pod="openshift-monitoring/node-exporter-b65wp"
Jul 27 19:30:10.457954 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.456669    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4aa68886-81b2-47c7-9aef-c6d73404122f-node-exporter-textfile\") pod \"node-exporter-b65wp\" (UID: \"4aa68886-81b2-47c7-9aef-c6d73404122f\") " pod="openshift-monitoring/node-exporter-b65wp"
Jul 27 19:30:10.457954 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.456707    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4aa68886-81b2-47c7-9aef-c6d73404122f-root\") pod \"node-exporter-b65wp\" (UID: \"4aa68886-81b2-47c7-9aef-c6d73404122f\") " pod="openshift-monitoring/node-exporter-b65wp"
Jul 27 19:30:10.457954 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.456730    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a5c72a0-8769-4caf-80e4-cfc8db418c38-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-p4c8p\" (UID: \"3a5c72a0-8769-4caf-80e4-cfc8db418c38\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-p4c8p"
Jul 27 19:30:10.457954 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.456762    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4aa68886-81b2-47c7-9aef-c6d73404122f-node-exporter-tls\") pod \"node-exporter-b65wp\" (UID: \"4aa68886-81b2-47c7-9aef-c6d73404122f\") " pod="openshift-monitoring/node-exporter-b65wp"
Jul 27 19:30:10.457954 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.456794    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/9b70adeb-3b2e-41f2-a157-6b2dfe797669-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-pwsmc\" (UID: \"9b70adeb-3b2e-41f2-a157-6b2dfe797669\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-pwsmc"
Jul 27 19:30:10.457954 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.456834    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/9b70adeb-3b2e-41f2-a157-6b2dfe797669-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-pwsmc\" (UID: \"9b70adeb-3b2e-41f2-a157-6b2dfe797669\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-pwsmc"
Jul 27 19:30:10.457954 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.457113    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b70adeb-3b2e-41f2-a157-6b2dfe797669-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-pwsmc\" (UID: \"9b70adeb-3b2e-41f2-a157-6b2dfe797669\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-pwsmc"
Jul 27 19:30:10.457954 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.457135    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4aa68886-81b2-47c7-9aef-c6d73404122f-node-exporter-accelerators-collector-config\") pod \"node-exporter-b65wp\" (UID: \"4aa68886-81b2-47c7-9aef-c6d73404122f\") " pod="openshift-monitoring/node-exporter-b65wp"
Jul 27 19:30:10.457954 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.457388    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4aa68886-81b2-47c7-9aef-c6d73404122f-metrics-client-ca\") pod \"node-exporter-b65wp\" (UID: \"4aa68886-81b2-47c7-9aef-c6d73404122f\") " pod="openshift-monitoring/node-exporter-b65wp"
Jul 27 19:30:10.458721 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.457396    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4aa68886-81b2-47c7-9aef-c6d73404122f-node-exporter-textfile\") pod \"node-exporter-b65wp\" (UID: \"4aa68886-81b2-47c7-9aef-c6d73404122f\") " pod="openshift-monitoring/node-exporter-b65wp"
Jul 27 19:30:10.458721 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.457488    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/9b70adeb-3b2e-41f2-a157-6b2dfe797669-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-pwsmc\" (UID: \"9b70adeb-3b2e-41f2-a157-6b2dfe797669\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-pwsmc"
Jul 27 19:30:10.458721 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.457635    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4aa68886-81b2-47c7-9aef-c6d73404122f-root\") pod \"node-exporter-b65wp\" (UID: \"4aa68886-81b2-47c7-9aef-c6d73404122f\") " pod="openshift-monitoring/node-exporter-b65wp"
Jul 27 19:30:10.458721 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.457774    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3a5c72a0-8769-4caf-80e4-cfc8db418c38-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-p4c8p\" (UID: \"3a5c72a0-8769-4caf-80e4-cfc8db418c38\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-p4c8p"
Jul 27 19:30:10.458721 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:30:10.457852    2558 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Jul 27 19:30:10.458721 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:30:10.457930    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a5c72a0-8769-4caf-80e4-cfc8db418c38-openshift-state-metrics-tls podName:3a5c72a0-8769-4caf-80e4-cfc8db418c38 nodeName:}" failed. No retries permitted until 2026-07-27 19:30:10.957910334 +0000 UTC m=+223.830102286 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/3a5c72a0-8769-4caf-80e4-cfc8db418c38-openshift-state-metrics-tls") pod "openshift-state-metrics-b8f49f5c6-p4c8p" (UID: "3a5c72a0-8769-4caf-80e4-cfc8db418c38") : secret "openshift-state-metrics-tls" not found
Jul 27 19:30:10.458721 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:30:10.457940    2558 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 27 19:30:10.458721 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:30:10.457990    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4aa68886-81b2-47c7-9aef-c6d73404122f-node-exporter-tls podName:4aa68886-81b2-47c7-9aef-c6d73404122f nodeName:}" failed. No retries permitted until 2026-07-27 19:30:10.957971825 +0000 UTC m=+223.830163782 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/4aa68886-81b2-47c7-9aef-c6d73404122f-node-exporter-tls") pod "node-exporter-b65wp" (UID: "4aa68886-81b2-47c7-9aef-c6d73404122f") : secret "node-exporter-tls" not found
Jul 27 19:30:10.460229 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.460110    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3a5c72a0-8769-4caf-80e4-cfc8db418c38-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-p4c8p\" (UID: \"3a5c72a0-8769-4caf-80e4-cfc8db418c38\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-p4c8p"
Jul 27 19:30:10.460424 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.460384    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9b70adeb-3b2e-41f2-a157-6b2dfe797669-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-pwsmc\" (UID: \"9b70adeb-3b2e-41f2-a157-6b2dfe797669\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-pwsmc"
Jul 27 19:30:10.460586 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.460567    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4aa68886-81b2-47c7-9aef-c6d73404122f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-b65wp\" (UID: \"4aa68886-81b2-47c7-9aef-c6d73404122f\") " pod="openshift-monitoring/node-exporter-b65wp"
Jul 27 19:30:10.462754 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.462684    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9b70adeb-3b2e-41f2-a157-6b2dfe797669-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-pwsmc\" (UID: \"9b70adeb-3b2e-41f2-a157-6b2dfe797669\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-pwsmc"
Jul 27 19:30:10.465194 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.465133    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-42nxq\" (UniqueName: \"kubernetes.io/projected/4aa68886-81b2-47c7-9aef-c6d73404122f-kube-api-access-42nxq\") pod \"node-exporter-b65wp\" (UID: \"4aa68886-81b2-47c7-9aef-c6d73404122f\") " pod="openshift-monitoring/node-exporter-b65wp"
Jul 27 19:30:10.465704 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.465660    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6b8fj\" (UniqueName: \"kubernetes.io/projected/9b70adeb-3b2e-41f2-a157-6b2dfe797669-kube-api-access-6b8fj\") pod \"kube-state-metrics-56d9f944b4-pwsmc\" (UID: \"9b70adeb-3b2e-41f2-a157-6b2dfe797669\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-pwsmc"
Jul 27 19:30:10.465807 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.465717    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqmcj\" (UniqueName: \"kubernetes.io/projected/3a5c72a0-8769-4caf-80e4-cfc8db418c38-kube-api-access-wqmcj\") pod \"openshift-state-metrics-b8f49f5c6-p4c8p\" (UID: \"3a5c72a0-8769-4caf-80e4-cfc8db418c38\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-p4c8p"
Jul 27 19:30:10.466868 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.466844    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:10.540495 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.540394    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-pwsmc"
Jul 27 19:30:10.564610 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.564570    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:10.646615 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.646504    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:10.690084 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:30:10.690040    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-zbjtr" podUID="229036e0-7551-4987-98d3-0ccef9c511e5"
Jul 27 19:30:10.960851 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.960821    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a5c72a0-8769-4caf-80e4-cfc8db418c38-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-p4c8p\" (UID: \"3a5c72a0-8769-4caf-80e4-cfc8db418c38\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-p4c8p"
Jul 27 19:30:10.960996 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.960868    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4aa68886-81b2-47c7-9aef-c6d73404122f-node-exporter-tls\") pod \"node-exporter-b65wp\" (UID: \"4aa68886-81b2-47c7-9aef-c6d73404122f\") " pod="openshift-monitoring/node-exporter-b65wp"
Jul 27 19:30:10.963651 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.963625    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a5c72a0-8769-4caf-80e4-cfc8db418c38-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-p4c8p\" (UID: \"3a5c72a0-8769-4caf-80e4-cfc8db418c38\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-p4c8p"
Jul 27 19:30:10.963759 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:10.963625    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4aa68886-81b2-47c7-9aef-c6d73404122f-node-exporter-tls\") pod \"node-exporter-b65wp\" (UID: \"4aa68886-81b2-47c7-9aef-c6d73404122f\") " pod="openshift-monitoring/node-exporter-b65wp"
Jul 27 19:30:11.116279 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:11.116235    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-p4c8p"
Jul 27 19:30:11.149709 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:11.149536    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-b65wp"
Jul 27 19:30:11.181939 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:11.181910    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:11.260955 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:11.260877    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 27 19:30:11.262711 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:11.262682    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:11.264529 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:11.264505    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:11.267431 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:11.267118    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 27 19:30:11.267431 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:11.267118    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 27 19:30:11.267431 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:11.267401    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 27 19:30:11.267639 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:11.267402    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 27 19:30:11.267901 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:11.267872    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-25z2x\""
Jul 27 19:30:11.268946 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:11.268751    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 27 19:30:11.268946 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:11.268776    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 27 19:30:11.268946 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:11.268809    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 27 19:30:11.268946 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:11.268938    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 27 19:30:11.269341 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:11.269150    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 27 19:30:11.277280 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:11.277257    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 27 19:30:11.364635 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:11.364588    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/dbb93860-c0e8-4d5c-8844-1142f7a7d187-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"dbb93860-c0e8-4d5c-8844-1142f7a7d187\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:11.364805 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:11.364666    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dbb93860-c0e8-4d5c-8844-1142f7a7d187-config-out\") pod \"alertmanager-main-0\" (UID: \"dbb93860-c0e8-4d5c-8844-1142f7a7d187\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:11.364805 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:11.364735    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dbb93860-c0e8-4d5c-8844-1142f7a7d187-tls-assets\") pod \"alertmanager-main-0\" (UID: \"dbb93860-c0e8-4d5c-8844-1142f7a7d187\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:11.364805 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:11.364762    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dbb93860-c0e8-4d5c-8844-1142f7a7d187-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"dbb93860-c0e8-4d5c-8844-1142f7a7d187\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:11.364979 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:11.364870    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/dbb93860-c0e8-4d5c-8844-1142f7a7d187-config-volume\") pod \"alertmanager-main-0\" (UID: \"dbb93860-c0e8-4d5c-8844-1142f7a7d187\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:11.364979 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:11.364908    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dbb93860-c0e8-4d5c-8844-1142f7a7d187-web-config\") pod \"alertmanager-main-0\" (UID: \"dbb93860-c0e8-4d5c-8844-1142f7a7d187\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:11.364979 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:11.364935    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dbb93860-c0e8-4d5c-8844-1142f7a7d187-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"dbb93860-c0e8-4d5c-8844-1142f7a7d187\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:11.364979 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:11.364961    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dbb93860-c0e8-4d5c-8844-1142f7a7d187-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"dbb93860-c0e8-4d5c-8844-1142f7a7d187\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:11.365160 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:11.364998    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/dbb93860-c0e8-4d5c-8844-1142f7a7d187-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"dbb93860-c0e8-4d5c-8844-1142f7a7d187\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:11.365160 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:11.365029    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbxh8\" (UniqueName: \"kubernetes.io/projected/dbb93860-c0e8-4d5c-8844-1142f7a7d187-kube-api-access-mbxh8\") pod \"alertmanager-main-0\" (UID: \"dbb93860-c0e8-4d5c-8844-1142f7a7d187\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:11.365160 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:11.365114    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/dbb93860-c0e8-4d5c-8844-1142f7a7d187-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"dbb93860-c0e8-4d5c-8844-1142f7a7d187\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:11.365160 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:11.365150    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/dbb93860-c0e8-4d5c-8844-1142f7a7d187-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"dbb93860-c0e8-4d5c-8844-1142f7a7d187\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:11.365288 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:11.365185    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dbb93860-c0e8-4d5c-8844-1142f7a7d187-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"dbb93860-c0e8-4d5c-8844-1142f7a7d187\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:11.397057 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:11.397024    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:11.466507 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:11.466462    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/dbb93860-c0e8-4d5c-8844-1142f7a7d187-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"dbb93860-c0e8-4d5c-8844-1142f7a7d187\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:11.466507 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:11.466504    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/dbb93860-c0e8-4d5c-8844-1142f7a7d187-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"dbb93860-c0e8-4d5c-8844-1142f7a7d187\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:11.466757 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:11.466537    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dbb93860-c0e8-4d5c-8844-1142f7a7d187-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"dbb93860-c0e8-4d5c-8844-1142f7a7d187\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:11.466757 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:11.466559    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/dbb93860-c0e8-4d5c-8844-1142f7a7d187-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"dbb93860-c0e8-4d5c-8844-1142f7a7d187\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:11.466757 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:11.466584    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dbb93860-c0e8-4d5c-8844-1142f7a7d187-config-out\") pod \"alertmanager-main-0\" (UID: \"dbb93860-c0e8-4d5c-8844-1142f7a7d187\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:11.466757 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:11.466638    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dbb93860-c0e8-4d5c-8844-1142f7a7d187-tls-assets\") pod \"alertmanager-main-0\" (UID: \"dbb93860-c0e8-4d5c-8844-1142f7a7d187\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:11.466757 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:11.466661    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dbb93860-c0e8-4d5c-8844-1142f7a7d187-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"dbb93860-c0e8-4d5c-8844-1142f7a7d187\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:11.466757 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:11.466703    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/dbb93860-c0e8-4d5c-8844-1142f7a7d187-config-volume\") pod \"alertmanager-main-0\" (UID: \"dbb93860-c0e8-4d5c-8844-1142f7a7d187\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:11.467046 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:11.466778    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dbb93860-c0e8-4d5c-8844-1142f7a7d187-web-config\") pod \"alertmanager-main-0\" (UID: \"dbb93860-c0e8-4d5c-8844-1142f7a7d187\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:11.467046 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:11.466823    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dbb93860-c0e8-4d5c-8844-1142f7a7d187-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"dbb93860-c0e8-4d5c-8844-1142f7a7d187\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:11.467046 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:11.466853    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dbb93860-c0e8-4d5c-8844-1142f7a7d187-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"dbb93860-c0e8-4d5c-8844-1142f7a7d187\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:11.467046 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:11.466896    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/dbb93860-c0e8-4d5c-8844-1142f7a7d187-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"dbb93860-c0e8-4d5c-8844-1142f7a7d187\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:11.467046 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:11.466927    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbxh8\" (UniqueName: \"kubernetes.io/projected/dbb93860-c0e8-4d5c-8844-1142f7a7d187-kube-api-access-mbxh8\") pod \"alertmanager-main-0\" (UID: \"dbb93860-c0e8-4d5c-8844-1142f7a7d187\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:11.467563 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:11.467320    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/dbb93860-c0e8-4d5c-8844-1142f7a7d187-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"dbb93860-c0e8-4d5c-8844-1142f7a7d187\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:11.467950 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:11.467887    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dbb93860-c0e8-4d5c-8844-1142f7a7d187-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"dbb93860-c0e8-4d5c-8844-1142f7a7d187\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:11.468067 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:11.468033    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dbb93860-c0e8-4d5c-8844-1142f7a7d187-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"dbb93860-c0e8-4d5c-8844-1142f7a7d187\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:11.469988 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:11.469945    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/dbb93860-c0e8-4d5c-8844-1142f7a7d187-config-volume\") pod \"alertmanager-main-0\" (UID: \"dbb93860-c0e8-4d5c-8844-1142f7a7d187\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:11.470938 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:11.470915    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dbb93860-c0e8-4d5c-8844-1142f7a7d187-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"dbb93860-c0e8-4d5c-8844-1142f7a7d187\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:11.471035 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:11.471007    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dbb93860-c0e8-4d5c-8844-1142f7a7d187-tls-assets\") pod \"alertmanager-main-0\" (UID: \"dbb93860-c0e8-4d5c-8844-1142f7a7d187\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:11.471144 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:11.471084    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dbb93860-c0e8-4d5c-8844-1142f7a7d187-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"dbb93860-c0e8-4d5c-8844-1142f7a7d187\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:11.471443 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:11.471381    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dbb93860-c0e8-4d5c-8844-1142f7a7d187-config-out\") pod \"alertmanager-main-0\" (UID: \"dbb93860-c0e8-4d5c-8844-1142f7a7d187\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:11.471545 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:11.471445    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/dbb93860-c0e8-4d5c-8844-1142f7a7d187-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"dbb93860-c0e8-4d5c-8844-1142f7a7d187\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:11.471545 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:11.471479    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dbb93860-c0e8-4d5c-8844-1142f7a7d187-web-config\") pod \"alertmanager-main-0\" (UID: \"dbb93860-c0e8-4d5c-8844-1142f7a7d187\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:11.472728 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:11.472706    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/dbb93860-c0e8-4d5c-8844-1142f7a7d187-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"dbb93860-c0e8-4d5c-8844-1142f7a7d187\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:11.472728 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:11.472718    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/dbb93860-c0e8-4d5c-8844-1142f7a7d187-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"dbb93860-c0e8-4d5c-8844-1142f7a7d187\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:11.474589 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:11.474569    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbxh8\" (UniqueName: \"kubernetes.io/projected/dbb93860-c0e8-4d5c-8844-1142f7a7d187-kube-api-access-mbxh8\") pod \"alertmanager-main-0\" (UID: \"dbb93860-c0e8-4d5c-8844-1142f7a7d187\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:11.497704 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:11.497675    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:11.578778 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:11.578701    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:13.130759 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:13.130718    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:13.205558 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:13.205523    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:13.500139 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:13.500067    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:13.596083 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:13.596055    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:14.008207 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:14.008164    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:14.090198 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:14.090166    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:14.611369 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:14.611329    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-55cdff9587-pc94t"]
Jul 27 19:30:14.614764 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:14.614741    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-55cdff9587-pc94t"
Jul 27 19:30:14.617947 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:14.617919    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 27 19:30:14.619371 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:14.619346    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 27 19:30:14.619707 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:14.619679    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-6o88p33iobs8c\""
Jul 27 19:30:14.619707 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:14.619704    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-zh4wt\""
Jul 27 19:30:14.619843 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:14.619683    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 27 19:30:14.619843 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:14.619804    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 27 19:30:14.625165 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:14.625135    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-55cdff9587-pc94t"]
Jul 27 19:30:14.699526 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:14.699485    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/94c83f10-a0af-4c99-b91f-14bac2e99cfd-metrics-server-audit-profiles\") pod \"metrics-server-55cdff9587-pc94t\" (UID: \"94c83f10-a0af-4c99-b91f-14bac2e99cfd\") " pod="openshift-monitoring/metrics-server-55cdff9587-pc94t"
Jul 27 19:30:14.699526 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:14.699530    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/94c83f10-a0af-4c99-b91f-14bac2e99cfd-secret-metrics-server-tls\") pod \"metrics-server-55cdff9587-pc94t\" (UID: \"94c83f10-a0af-4c99-b91f-14bac2e99cfd\") " pod="openshift-monitoring/metrics-server-55cdff9587-pc94t"
Jul 27 19:30:14.699723 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:14.699616    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/94c83f10-a0af-4c99-b91f-14bac2e99cfd-audit-log\") pod \"metrics-server-55cdff9587-pc94t\" (UID: \"94c83f10-a0af-4c99-b91f-14bac2e99cfd\") " pod="openshift-monitoring/metrics-server-55cdff9587-pc94t"
Jul 27 19:30:14.699723 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:14.699668    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86ssx\" (UniqueName: \"kubernetes.io/projected/94c83f10-a0af-4c99-b91f-14bac2e99cfd-kube-api-access-86ssx\") pod \"metrics-server-55cdff9587-pc94t\" (UID: \"94c83f10-a0af-4c99-b91f-14bac2e99cfd\") " pod="openshift-monitoring/metrics-server-55cdff9587-pc94t"
Jul 27 19:30:14.699723 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:14.699706    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/94c83f10-a0af-4c99-b91f-14bac2e99cfd-secret-metrics-server-client-certs\") pod \"metrics-server-55cdff9587-pc94t\" (UID: \"94c83f10-a0af-4c99-b91f-14bac2e99cfd\") " pod="openshift-monitoring/metrics-server-55cdff9587-pc94t"
Jul 27 19:30:14.699869 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:14.699789    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/94c83f10-a0af-4c99-b91f-14bac2e99cfd-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-55cdff9587-pc94t\" (UID: \"94c83f10-a0af-4c99-b91f-14bac2e99cfd\") " pod="openshift-monitoring/metrics-server-55cdff9587-pc94t"
Jul 27 19:30:14.699869 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:14.699834    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/94c83f10-a0af-4c99-b91f-14bac2e99cfd-client-ca-bundle\") pod \"metrics-server-55cdff9587-pc94t\" (UID: \"94c83f10-a0af-4c99-b91f-14bac2e99cfd\") " pod="openshift-monitoring/metrics-server-55cdff9587-pc94t"
Jul 27 19:30:14.800889 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:14.800850    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/94c83f10-a0af-4c99-b91f-14bac2e99cfd-audit-log\") pod \"metrics-server-55cdff9587-pc94t\" (UID: \"94c83f10-a0af-4c99-b91f-14bac2e99cfd\") " pod="openshift-monitoring/metrics-server-55cdff9587-pc94t"
Jul 27 19:30:14.801081 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:14.800924    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86ssx\" (UniqueName: \"kubernetes.io/projected/94c83f10-a0af-4c99-b91f-14bac2e99cfd-kube-api-access-86ssx\") pod \"metrics-server-55cdff9587-pc94t\" (UID: \"94c83f10-a0af-4c99-b91f-14bac2e99cfd\") " pod="openshift-monitoring/metrics-server-55cdff9587-pc94t"
Jul 27 19:30:14.801081 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:14.800972    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/94c83f10-a0af-4c99-b91f-14bac2e99cfd-secret-metrics-server-client-certs\") pod \"metrics-server-55cdff9587-pc94t\" (UID: \"94c83f10-a0af-4c99-b91f-14bac2e99cfd\") " pod="openshift-monitoring/metrics-server-55cdff9587-pc94t"
Jul 27 19:30:14.801187 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:14.801074    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/94c83f10-a0af-4c99-b91f-14bac2e99cfd-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-55cdff9587-pc94t\" (UID: \"94c83f10-a0af-4c99-b91f-14bac2e99cfd\") " pod="openshift-monitoring/metrics-server-55cdff9587-pc94t"
Jul 27 19:30:14.801187 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:14.801128    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/94c83f10-a0af-4c99-b91f-14bac2e99cfd-client-ca-bundle\") pod \"metrics-server-55cdff9587-pc94t\" (UID: \"94c83f10-a0af-4c99-b91f-14bac2e99cfd\") " pod="openshift-monitoring/metrics-server-55cdff9587-pc94t"
Jul 27 19:30:14.801305 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:14.801185    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/94c83f10-a0af-4c99-b91f-14bac2e99cfd-metrics-server-audit-profiles\") pod \"metrics-server-55cdff9587-pc94t\" (UID: \"94c83f10-a0af-4c99-b91f-14bac2e99cfd\") " pod="openshift-monitoring/metrics-server-55cdff9587-pc94t"
Jul 27 19:30:14.801305 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:14.801233    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/94c83f10-a0af-4c99-b91f-14bac2e99cfd-secret-metrics-server-tls\") pod \"metrics-server-55cdff9587-pc94t\" (UID: \"94c83f10-a0af-4c99-b91f-14bac2e99cfd\") " pod="openshift-monitoring/metrics-server-55cdff9587-pc94t"
Jul 27 19:30:14.801305 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:14.801261    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/94c83f10-a0af-4c99-b91f-14bac2e99cfd-audit-log\") pod \"metrics-server-55cdff9587-pc94t\" (UID: \"94c83f10-a0af-4c99-b91f-14bac2e99cfd\") " pod="openshift-monitoring/metrics-server-55cdff9587-pc94t"
Jul 27 19:30:14.802192 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:14.802154    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/94c83f10-a0af-4c99-b91f-14bac2e99cfd-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-55cdff9587-pc94t\" (UID: \"94c83f10-a0af-4c99-b91f-14bac2e99cfd\") " pod="openshift-monitoring/metrics-server-55cdff9587-pc94t"
Jul 27 19:30:14.802360 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:14.802312    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/94c83f10-a0af-4c99-b91f-14bac2e99cfd-metrics-server-audit-profiles\") pod \"metrics-server-55cdff9587-pc94t\" (UID: \"94c83f10-a0af-4c99-b91f-14bac2e99cfd\") " pod="openshift-monitoring/metrics-server-55cdff9587-pc94t"
Jul 27 19:30:14.804051 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:14.804027    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/94c83f10-a0af-4c99-b91f-14bac2e99cfd-client-ca-bundle\") pod \"metrics-server-55cdff9587-pc94t\" (UID: \"94c83f10-a0af-4c99-b91f-14bac2e99cfd\") " pod="openshift-monitoring/metrics-server-55cdff9587-pc94t"
Jul 27 19:30:14.804180 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:14.804157    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/94c83f10-a0af-4c99-b91f-14bac2e99cfd-secret-metrics-server-client-certs\") pod \"metrics-server-55cdff9587-pc94t\" (UID: \"94c83f10-a0af-4c99-b91f-14bac2e99cfd\") " pod="openshift-monitoring/metrics-server-55cdff9587-pc94t"
Jul 27 19:30:14.804661 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:14.804638    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/94c83f10-a0af-4c99-b91f-14bac2e99cfd-secret-metrics-server-tls\") pod \"metrics-server-55cdff9587-pc94t\" (UID: \"94c83f10-a0af-4c99-b91f-14bac2e99cfd\") " pod="openshift-monitoring/metrics-server-55cdff9587-pc94t"
Jul 27 19:30:14.809599 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:14.809577    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-86ssx\" (UniqueName: \"kubernetes.io/projected/94c83f10-a0af-4c99-b91f-14bac2e99cfd-kube-api-access-86ssx\") pod \"metrics-server-55cdff9587-pc94t\" (UID: \"94c83f10-a0af-4c99-b91f-14bac2e99cfd\") " pod="openshift-monitoring/metrics-server-55cdff9587-pc94t"
Jul 27 19:30:14.927301 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:14.927261    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-55cdff9587-pc94t"
Jul 27 19:30:14.982963 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:14.982932    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:15.049894 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:15.049857    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:15.132169 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:15.132128    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:15.413336 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:15.413301    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-65b64784b7-vnbrq"]
Jul 27 19:30:15.416783 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:15.416759    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-65b64784b7-vnbrq"
Jul 27 19:30:15.419137 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:15.419111    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 27 19:30:15.419285 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:15.419162    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 27 19:30:15.419285 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:15.419258    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 27 19:30:15.419680 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:15.419643    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 27 19:30:15.419813 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:15.419679    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 27 19:30:15.419813 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:15.419789    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-q75sp\""
Jul 27 19:30:15.424428 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:15.424396    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 27 19:30:15.428868 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:15.428843    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-65b64784b7-vnbrq"]
Jul 27 19:30:15.507506 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:15.507470    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/489d2925-8a91-49b6-8a13-0280ea8b8bc2-metrics-client-ca\") pod \"telemeter-client-65b64784b7-vnbrq\" (UID: \"489d2925-8a91-49b6-8a13-0280ea8b8bc2\") " pod="openshift-monitoring/telemeter-client-65b64784b7-vnbrq"
Jul 27 19:30:15.507706 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:15.507521    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/489d2925-8a91-49b6-8a13-0280ea8b8bc2-serving-certs-ca-bundle\") pod \"telemeter-client-65b64784b7-vnbrq\" (UID: \"489d2925-8a91-49b6-8a13-0280ea8b8bc2\") " pod="openshift-monitoring/telemeter-client-65b64784b7-vnbrq"
Jul 27 19:30:15.507706 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:15.507552    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/489d2925-8a91-49b6-8a13-0280ea8b8bc2-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-65b64784b7-vnbrq\" (UID: \"489d2925-8a91-49b6-8a13-0280ea8b8bc2\") " pod="openshift-monitoring/telemeter-client-65b64784b7-vnbrq"
Jul 27 19:30:15.507706 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:15.507584    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/489d2925-8a91-49b6-8a13-0280ea8b8bc2-telemeter-client-tls\") pod \"telemeter-client-65b64784b7-vnbrq\" (UID: \"489d2925-8a91-49b6-8a13-0280ea8b8bc2\") " pod="openshift-monitoring/telemeter-client-65b64784b7-vnbrq"
Jul 27 19:30:15.507706 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:15.507620    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/489d2925-8a91-49b6-8a13-0280ea8b8bc2-secret-telemeter-client\") pod \"telemeter-client-65b64784b7-vnbrq\" (UID: \"489d2925-8a91-49b6-8a13-0280ea8b8bc2\") " pod="openshift-monitoring/telemeter-client-65b64784b7-vnbrq"
Jul 27 19:30:15.507706 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:15.507652    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/489d2925-8a91-49b6-8a13-0280ea8b8bc2-telemeter-trusted-ca-bundle\") pod \"telemeter-client-65b64784b7-vnbrq\" (UID: \"489d2925-8a91-49b6-8a13-0280ea8b8bc2\") " pod="openshift-monitoring/telemeter-client-65b64784b7-vnbrq"
Jul 27 19:30:15.507955 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:15.507723    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/489d2925-8a91-49b6-8a13-0280ea8b8bc2-federate-client-tls\") pod \"telemeter-client-65b64784b7-vnbrq\" (UID: \"489d2925-8a91-49b6-8a13-0280ea8b8bc2\") " pod="openshift-monitoring/telemeter-client-65b64784b7-vnbrq"
Jul 27 19:30:15.507955 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:15.507758    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29cxx\" (UniqueName: \"kubernetes.io/projected/489d2925-8a91-49b6-8a13-0280ea8b8bc2-kube-api-access-29cxx\") pod \"telemeter-client-65b64784b7-vnbrq\" (UID: \"489d2925-8a91-49b6-8a13-0280ea8b8bc2\") " pod="openshift-monitoring/telemeter-client-65b64784b7-vnbrq"
Jul 27 19:30:15.608803 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:15.608743    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/489d2925-8a91-49b6-8a13-0280ea8b8bc2-metrics-client-ca\") pod \"telemeter-client-65b64784b7-vnbrq\" (UID: \"489d2925-8a91-49b6-8a13-0280ea8b8bc2\") " pod="openshift-monitoring/telemeter-client-65b64784b7-vnbrq"
Jul 27 19:30:15.608803 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:15.608806    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/489d2925-8a91-49b6-8a13-0280ea8b8bc2-serving-certs-ca-bundle\") pod \"telemeter-client-65b64784b7-vnbrq\" (UID: \"489d2925-8a91-49b6-8a13-0280ea8b8bc2\") " pod="openshift-monitoring/telemeter-client-65b64784b7-vnbrq"
Jul 27 19:30:15.609072 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:15.608834    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/489d2925-8a91-49b6-8a13-0280ea8b8bc2-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-65b64784b7-vnbrq\" (UID: \"489d2925-8a91-49b6-8a13-0280ea8b8bc2\") " pod="openshift-monitoring/telemeter-client-65b64784b7-vnbrq"
Jul 27 19:30:15.609072 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:15.608855    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/489d2925-8a91-49b6-8a13-0280ea8b8bc2-telemeter-client-tls\") pod \"telemeter-client-65b64784b7-vnbrq\" (UID: \"489d2925-8a91-49b6-8a13-0280ea8b8bc2\") " pod="openshift-monitoring/telemeter-client-65b64784b7-vnbrq"
Jul 27 19:30:15.609072 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:15.608887    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/489d2925-8a91-49b6-8a13-0280ea8b8bc2-secret-telemeter-client\") pod \"telemeter-client-65b64784b7-vnbrq\" (UID: \"489d2925-8a91-49b6-8a13-0280ea8b8bc2\") " pod="openshift-monitoring/telemeter-client-65b64784b7-vnbrq"
Jul 27 19:30:15.609072 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:15.608920    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/489d2925-8a91-49b6-8a13-0280ea8b8bc2-telemeter-trusted-ca-bundle\") pod \"telemeter-client-65b64784b7-vnbrq\" (UID: \"489d2925-8a91-49b6-8a13-0280ea8b8bc2\") " pod="openshift-monitoring/telemeter-client-65b64784b7-vnbrq"
Jul 27 19:30:15.609072 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:15.608983    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/489d2925-8a91-49b6-8a13-0280ea8b8bc2-federate-client-tls\") pod \"telemeter-client-65b64784b7-vnbrq\" (UID: \"489d2925-8a91-49b6-8a13-0280ea8b8bc2\") " pod="openshift-monitoring/telemeter-client-65b64784b7-vnbrq"
Jul 27 19:30:15.609072 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:15.609011    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29cxx\" (UniqueName: \"kubernetes.io/projected/489d2925-8a91-49b6-8a13-0280ea8b8bc2-kube-api-access-29cxx\") pod \"telemeter-client-65b64784b7-vnbrq\" (UID: \"489d2925-8a91-49b6-8a13-0280ea8b8bc2\") " pod="openshift-monitoring/telemeter-client-65b64784b7-vnbrq"
Jul 27 19:30:15.609827 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:15.609748    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/489d2925-8a91-49b6-8a13-0280ea8b8bc2-metrics-client-ca\") pod \"telemeter-client-65b64784b7-vnbrq\" (UID: \"489d2925-8a91-49b6-8a13-0280ea8b8bc2\") " pod="openshift-monitoring/telemeter-client-65b64784b7-vnbrq"
Jul 27 19:30:15.610132 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:15.610092    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/489d2925-8a91-49b6-8a13-0280ea8b8bc2-serving-certs-ca-bundle\") pod \"telemeter-client-65b64784b7-vnbrq\" (UID: \"489d2925-8a91-49b6-8a13-0280ea8b8bc2\") " pod="openshift-monitoring/telemeter-client-65b64784b7-vnbrq"
Jul 27 19:30:15.610311 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:15.610288    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/489d2925-8a91-49b6-8a13-0280ea8b8bc2-telemeter-trusted-ca-bundle\") pod \"telemeter-client-65b64784b7-vnbrq\" (UID: \"489d2925-8a91-49b6-8a13-0280ea8b8bc2\") " pod="openshift-monitoring/telemeter-client-65b64784b7-vnbrq"
Jul 27 19:30:15.612041 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:15.611994    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/489d2925-8a91-49b6-8a13-0280ea8b8bc2-secret-telemeter-client\") pod \"telemeter-client-65b64784b7-vnbrq\" (UID: \"489d2925-8a91-49b6-8a13-0280ea8b8bc2\") " pod="openshift-monitoring/telemeter-client-65b64784b7-vnbrq"
Jul 27 19:30:15.612475 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:15.612047    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/489d2925-8a91-49b6-8a13-0280ea8b8bc2-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-65b64784b7-vnbrq\" (UID: \"489d2925-8a91-49b6-8a13-0280ea8b8bc2\") " pod="openshift-monitoring/telemeter-client-65b64784b7-vnbrq"
Jul 27 19:30:15.612475 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:15.612302    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/489d2925-8a91-49b6-8a13-0280ea8b8bc2-telemeter-client-tls\") pod \"telemeter-client-65b64784b7-vnbrq\" (UID: \"489d2925-8a91-49b6-8a13-0280ea8b8bc2\") " pod="openshift-monitoring/telemeter-client-65b64784b7-vnbrq"
Jul 27 19:30:15.612609 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:15.612479    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/489d2925-8a91-49b6-8a13-0280ea8b8bc2-federate-client-tls\") pod \"telemeter-client-65b64784b7-vnbrq\" (UID: \"489d2925-8a91-49b6-8a13-0280ea8b8bc2\") " pod="openshift-monitoring/telemeter-client-65b64784b7-vnbrq"
Jul 27 19:30:15.617468 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:15.617373    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-29cxx\" (UniqueName: \"kubernetes.io/projected/489d2925-8a91-49b6-8a13-0280ea8b8bc2-kube-api-access-29cxx\") pod \"telemeter-client-65b64784b7-vnbrq\" (UID: \"489d2925-8a91-49b6-8a13-0280ea8b8bc2\") " pod="openshift-monitoring/telemeter-client-65b64784b7-vnbrq"
Jul 27 19:30:15.729582 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:15.729500    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-65b64784b7-vnbrq"
Jul 27 19:30:16.144913 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:16.144879    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:16.240704 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:16.240665    2558 patch_prober.go:28] interesting pod/image-registry-7f4bbb7d8b-kjzcn container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 27 19:30:16.240908 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:16.240740    2558 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-7f4bbb7d8b-kjzcn" podUID="a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 27 19:30:16.245637 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:16.245612    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:16.402810 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:16.402718    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:17.207435 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:17.207391    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6cc4fc4b9c-mnk9w"]
Jul 27 19:30:17.218696 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:17.218667    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cc4fc4b9c-mnk9w"
Jul 27 19:30:17.221860 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:17.221828    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6cc4fc4b9c-mnk9w"]
Jul 27 19:30:17.226178 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:17.226153    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 27 19:30:17.323131 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:17.323099    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/67d6f95c-e959-40a5-8dfa-39228c3b963f-console-config\") pod \"console-6cc4fc4b9c-mnk9w\" (UID: \"67d6f95c-e959-40a5-8dfa-39228c3b963f\") " pod="openshift-console/console-6cc4fc4b9c-mnk9w"
Jul 27 19:30:17.323328 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:17.323156    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/67d6f95c-e959-40a5-8dfa-39228c3b963f-oauth-serving-cert\") pod \"console-6cc4fc4b9c-mnk9w\" (UID: \"67d6f95c-e959-40a5-8dfa-39228c3b963f\") " pod="openshift-console/console-6cc4fc4b9c-mnk9w"
Jul 27 19:30:17.323328 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:17.323259    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/67d6f95c-e959-40a5-8dfa-39228c3b963f-service-ca\") pod \"console-6cc4fc4b9c-mnk9w\" (UID: \"67d6f95c-e959-40a5-8dfa-39228c3b963f\") " pod="openshift-console/console-6cc4fc4b9c-mnk9w"
Jul 27 19:30:17.323328 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:17.323302    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrd62\" (UniqueName: \"kubernetes.io/projected/67d6f95c-e959-40a5-8dfa-39228c3b963f-kube-api-access-qrd62\") pod \"console-6cc4fc4b9c-mnk9w\" (UID: \"67d6f95c-e959-40a5-8dfa-39228c3b963f\") " pod="openshift-console/console-6cc4fc4b9c-mnk9w"
Jul 27 19:30:17.323520 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:17.323394    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/67d6f95c-e959-40a5-8dfa-39228c3b963f-console-serving-cert\") pod \"console-6cc4fc4b9c-mnk9w\" (UID: \"67d6f95c-e959-40a5-8dfa-39228c3b963f\") " pod="openshift-console/console-6cc4fc4b9c-mnk9w"
Jul 27 19:30:17.323520 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:17.323447    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/67d6f95c-e959-40a5-8dfa-39228c3b963f-console-oauth-config\") pod \"console-6cc4fc4b9c-mnk9w\" (UID: \"67d6f95c-e959-40a5-8dfa-39228c3b963f\") " pod="openshift-console/console-6cc4fc4b9c-mnk9w"
Jul 27 19:30:17.323520 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:17.323469    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67d6f95c-e959-40a5-8dfa-39228c3b963f-trusted-ca-bundle\") pod \"console-6cc4fc4b9c-mnk9w\" (UID: \"67d6f95c-e959-40a5-8dfa-39228c3b963f\") " pod="openshift-console/console-6cc4fc4b9c-mnk9w"
Jul 27 19:30:17.424260 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:17.424223    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/67d6f95c-e959-40a5-8dfa-39228c3b963f-console-serving-cert\") pod \"console-6cc4fc4b9c-mnk9w\" (UID: \"67d6f95c-e959-40a5-8dfa-39228c3b963f\") " pod="openshift-console/console-6cc4fc4b9c-mnk9w"
Jul 27 19:30:17.424475 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:17.424268    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/67d6f95c-e959-40a5-8dfa-39228c3b963f-console-oauth-config\") pod \"console-6cc4fc4b9c-mnk9w\" (UID: \"67d6f95c-e959-40a5-8dfa-39228c3b963f\") " pod="openshift-console/console-6cc4fc4b9c-mnk9w"
Jul 27 19:30:17.424475 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:17.424299    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67d6f95c-e959-40a5-8dfa-39228c3b963f-trusted-ca-bundle\") pod \"console-6cc4fc4b9c-mnk9w\" (UID: \"67d6f95c-e959-40a5-8dfa-39228c3b963f\") " pod="openshift-console/console-6cc4fc4b9c-mnk9w"
Jul 27 19:30:17.424475 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:17.424381    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/67d6f95c-e959-40a5-8dfa-39228c3b963f-console-config\") pod \"console-6cc4fc4b9c-mnk9w\" (UID: \"67d6f95c-e959-40a5-8dfa-39228c3b963f\") " pod="openshift-console/console-6cc4fc4b9c-mnk9w"
Jul 27 19:30:17.424475 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:17.424446    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/67d6f95c-e959-40a5-8dfa-39228c3b963f-oauth-serving-cert\") pod \"console-6cc4fc4b9c-mnk9w\" (UID: \"67d6f95c-e959-40a5-8dfa-39228c3b963f\") " pod="openshift-console/console-6cc4fc4b9c-mnk9w"
Jul 27 19:30:17.424735 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:17.424486    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/67d6f95c-e959-40a5-8dfa-39228c3b963f-service-ca\") pod \"console-6cc4fc4b9c-mnk9w\" (UID: \"67d6f95c-e959-40a5-8dfa-39228c3b963f\") " pod="openshift-console/console-6cc4fc4b9c-mnk9w"
Jul 27 19:30:17.424735 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:17.424515    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrd62\" (UniqueName: \"kubernetes.io/projected/67d6f95c-e959-40a5-8dfa-39228c3b963f-kube-api-access-qrd62\") pod \"console-6cc4fc4b9c-mnk9w\" (UID: \"67d6f95c-e959-40a5-8dfa-39228c3b963f\") " pod="openshift-console/console-6cc4fc4b9c-mnk9w"
Jul 27 19:30:17.425320 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:17.425274    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67d6f95c-e959-40a5-8dfa-39228c3b963f-trusted-ca-bundle\") pod \"console-6cc4fc4b9c-mnk9w\" (UID: \"67d6f95c-e959-40a5-8dfa-39228c3b963f\") " pod="openshift-console/console-6cc4fc4b9c-mnk9w"
Jul 27 19:30:17.425320 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:17.425289    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/67d6f95c-e959-40a5-8dfa-39228c3b963f-console-config\") pod \"console-6cc4fc4b9c-mnk9w\" (UID: \"67d6f95c-e959-40a5-8dfa-39228c3b963f\") " pod="openshift-console/console-6cc4fc4b9c-mnk9w"
Jul 27 19:30:17.425504 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:17.425447    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/67d6f95c-e959-40a5-8dfa-39228c3b963f-service-ca\") pod \"console-6cc4fc4b9c-mnk9w\" (UID: \"67d6f95c-e959-40a5-8dfa-39228c3b963f\") " pod="openshift-console/console-6cc4fc4b9c-mnk9w"
Jul 27 19:30:17.425556 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:17.425512    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/67d6f95c-e959-40a5-8dfa-39228c3b963f-oauth-serving-cert\") pod \"console-6cc4fc4b9c-mnk9w\" (UID: \"67d6f95c-e959-40a5-8dfa-39228c3b963f\") " pod="openshift-console/console-6cc4fc4b9c-mnk9w"
Jul 27 19:30:17.427145 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:17.427126    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/67d6f95c-e959-40a5-8dfa-39228c3b963f-console-serving-cert\") pod \"console-6cc4fc4b9c-mnk9w\" (UID: \"67d6f95c-e959-40a5-8dfa-39228c3b963f\") " pod="openshift-console/console-6cc4fc4b9c-mnk9w"
Jul 27 19:30:17.427438 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:17.427400    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/67d6f95c-e959-40a5-8dfa-39228c3b963f-console-oauth-config\") pod \"console-6cc4fc4b9c-mnk9w\" (UID: \"67d6f95c-e959-40a5-8dfa-39228c3b963f\") " pod="openshift-console/console-6cc4fc4b9c-mnk9w"
Jul 27 19:30:17.432845 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:17.432822    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrd62\" (UniqueName: \"kubernetes.io/projected/67d6f95c-e959-40a5-8dfa-39228c3b963f-kube-api-access-qrd62\") pod \"console-6cc4fc4b9c-mnk9w\" (UID: \"67d6f95c-e959-40a5-8dfa-39228c3b963f\") " pod="openshift-console/console-6cc4fc4b9c-mnk9w"
Jul 27 19:30:17.531737 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:17.531653    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cc4fc4b9c-mnk9w"
Jul 27 19:30:18.107685 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:18.107620    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:18.174302 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:18.174249    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:18.273915 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:18.273869    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7f4bbb7d8b-kjzcn"
Jul 27 19:30:18.420619 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:18.420320    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-65b64784b7-vnbrq"]
Jul 27 19:30:18.546917 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:18.546812    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:18.615256 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:18.615217    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:18.634444 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:30:18.634188    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod489d2925_8a91_49b6_8a13_0280ea8b8bc2.slice/crio-550e694ba7bb8d18bfa6e2866e7999af0907f5148ae1094f0c20bd9cd80c9fe7 WatchSource:0}: Error finding container 550e694ba7bb8d18bfa6e2866e7999af0907f5148ae1094f0c20bd9cd80c9fe7: Status 404 returned error can't find the container with id 550e694ba7bb8d18bfa6e2866e7999af0907f5148ae1094f0c20bd9cd80c9fe7
Jul 27 19:30:18.638661 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:30:18.638511    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4aa68886_81b2_47c7_9aef_c6d73404122f.slice/crio-1e17242c277ccd7eba6546db9b0b914aa92bf9c6fb2c49e64ff36c4b15556f1d WatchSource:0}: Error finding container 1e17242c277ccd7eba6546db9b0b914aa92bf9c6fb2c49e64ff36c4b15556f1d: Status 404 returned error can't find the container with id 1e17242c277ccd7eba6546db9b0b914aa92bf9c6fb2c49e64ff36c4b15556f1d
Jul 27 19:30:19.056091 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:19.056018    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-564f6df4f-d4kcc"]
Jul 27 19:30:19.059365 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:19.059291    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-pwsmc"]
Jul 27 19:30:19.064477 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:19.063910    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 27 19:30:19.091793 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:19.091761    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-55cdff9587-pc94t"]
Jul 27 19:30:19.093699 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:19.093644    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6cc4fc4b9c-mnk9w"]
Jul 27 19:30:19.097360 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:30:19.096714    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94c83f10_a0af_4c99_b91f_14bac2e99cfd.slice/crio-7c40505dccbae63b49a5307e9de1f8d29708e7dda8fd37dc1ef5e057a857dc05 WatchSource:0}: Error finding container 7c40505dccbae63b49a5307e9de1f8d29708e7dda8fd37dc1ef5e057a857dc05: Status 404 returned error can't find the container with id 7c40505dccbae63b49a5307e9de1f8d29708e7dda8fd37dc1ef5e057a857dc05
Jul 27 19:30:19.097360 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:19.096842    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-p4c8p"]
Jul 27 19:30:19.097760 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:30:19.097732    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67d6f95c_e959_40a5_8dfa_39228c3b963f.slice/crio-f215442834ecd01042123100392046f6205e6d9f6dab1297ee703b920dcf1ff9 WatchSource:0}: Error finding container f215442834ecd01042123100392046f6205e6d9f6dab1297ee703b920dcf1ff9: Status 404 returned error can't find the container with id f215442834ecd01042123100392046f6205e6d9f6dab1297ee703b920dcf1ff9
Jul 27 19:30:19.100143 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:30:19.100110    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a5c72a0_8769_4caf_80e4_cfc8db418c38.slice/crio-3bcd163c99f7b5828dc5e41ea8dfdf62b31ef892ba6257246d456cd095b87e3b WatchSource:0}: Error finding container 3bcd163c99f7b5828dc5e41ea8dfdf62b31ef892ba6257246d456cd095b87e3b: Status 404 returned error can't find the container with id 3bcd163c99f7b5828dc5e41ea8dfdf62b31ef892ba6257246d456cd095b87e3b
Jul 27 19:30:19.369623 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:19.369532    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jscrz" event={"ID":"8bb804bd-b63d-43b7-8159-80cd1abfab55","Type":"ContainerStarted","Data":"b6753961add9ff50dc2daa32091b3b546a085257967c0185fdcd2431cc1a6727"}
Jul 27 19:30:19.372570 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:19.372502    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dbb93860-c0e8-4d5c-8844-1142f7a7d187","Type":"ContainerStarted","Data":"4804dc8da4b19bee736657771879c34dcf69a1f022cae81978c958c2f2a6c979"}
Jul 27 19:30:19.379144 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:19.375903    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-qm44q" event={"ID":"212c42c2-b2a1-4082-8ea8-2a7f867bd45a","Type":"ContainerStarted","Data":"cecd5af21e2d66b81a7a6b48dea25b5b989de8b8ff9289c305a058e1ad9f5137"}
Jul 27 19:30:19.379144 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:19.378832    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-7c97dc4555-qm44q"
Jul 27 19:30:19.380947 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:19.380891    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-564f6df4f-d4kcc" event={"ID":"8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f","Type":"ContainerStarted","Data":"b098c5ee698e2d655b8c5563e5e035d53ccee4a169a6ebdfe70b1e020e6e2231"}
Jul 27 19:30:19.383664 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:19.383639    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-65b64784b7-vnbrq" event={"ID":"489d2925-8a91-49b6-8a13-0280ea8b8bc2","Type":"ContainerStarted","Data":"550e694ba7bb8d18bfa6e2866e7999af0907f5148ae1094f0c20bd9cd80c9fe7"}
Jul 27 19:30:19.386827 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:19.386800    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-p4c8p" event={"ID":"3a5c72a0-8769-4caf-80e4-cfc8db418c38","Type":"ContainerStarted","Data":"6aa151a4c3ee90082c7c0c11fd381e6500fe98855cdbb7eb4fe4f342e961fa67"}
Jul 27 19:30:19.386916 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:19.386859    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-p4c8p" event={"ID":"3a5c72a0-8769-4caf-80e4-cfc8db418c38","Type":"ContainerStarted","Data":"b073063c3c5129117da22138c9f4aafbecc095b5d488510d71d34d29df32fc6d"}
Jul 27 19:30:19.386916 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:19.386889    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-p4c8p" event={"ID":"3a5c72a0-8769-4caf-80e4-cfc8db418c38","Type":"ContainerStarted","Data":"3bcd163c99f7b5828dc5e41ea8dfdf62b31ef892ba6257246d456cd095b87e3b"}
Jul 27 19:30:19.388398 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:19.388375    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cc4fc4b9c-mnk9w" event={"ID":"67d6f95c-e959-40a5-8dfa-39228c3b963f","Type":"ContainerStarted","Data":"f215442834ecd01042123100392046f6205e6d9f6dab1297ee703b920dcf1ff9"}
Jul 27 19:30:19.391530 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:19.391505    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-55cdff9587-pc94t" event={"ID":"94c83f10-a0af-4c99-b91f-14bac2e99cfd","Type":"ContainerStarted","Data":"7c40505dccbae63b49a5307e9de1f8d29708e7dda8fd37dc1ef5e057a857dc05"}
Jul 27 19:30:19.393328 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:19.393277    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-7c97dc4555-qm44q" podStartSLOduration=2.087510346 podStartE2EDuration="21.393260095s" totalImagesPullingTime="19.305749749s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:29:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:30:19.39204232 +0000 UTC m=+232.264234297" watchObservedRunningTime="2026-07-27 19:30:19.393260095 +0000 UTC m=+232.265452065"
Jul 27 19:30:19.395277 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:19.395253    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-pwsmc" event={"ID":"9b70adeb-3b2e-41f2-a157-6b2dfe797669","Type":"ContainerStarted","Data":"d9dd076c2e70a452be8551f34081f791a3087e72c51fd1bb63120bab0af2d065"}
Jul 27 19:30:19.396996 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:19.396930    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b65wp" event={"ID":"4aa68886-81b2-47c7-9aef-c6d73404122f","Type":"ContainerStarted","Data":"1e17242c277ccd7eba6546db9b0b914aa92bf9c6fb2c49e64ff36c4b15556f1d"}
Jul 27 19:30:19.402320 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:19.402274    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-7c97dc4555-qm44q"
Jul 27 19:30:20.418779 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:20.418745    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:21.044229 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:21.044193    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7f4bbb7d8b-kjzcn"]
Jul 27 19:30:21.123447 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:21.123387    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:21.219834 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:21.219765    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:23.778493 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:23.778399    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:23.839925 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:23.839885    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:24.668342 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:24.668309    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zbjtr"
Jul 27 19:30:25.409686 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:25.409656    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:25.800474 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:25.800384    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:25.866948 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:25.866914    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:26.431677 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:26.431644    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:26.532040 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:26.532000    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:27.138631 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:27.138596    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:27.239827 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:27.239800    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:30.456880 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:30.456800    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jscrz" event={"ID":"8bb804bd-b63d-43b7-8159-80cd1abfab55","Type":"ContainerStarted","Data":"64aa81b9337e57b60af5647471f74f1598c86eb8c1dad1d7527104b41d6bddeb"}
Jul 27 19:30:30.458264 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:30.458240    2558 generic.go:356] "Generic (PLEG): container finished" podID="dbb93860-c0e8-4d5c-8844-1142f7a7d187" containerID="61e6492e60e09c1d082eb2d1d947538c85b634fe0e269717d94dd93f425ab4b7" exitCode=0
Jul 27 19:30:30.458382 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:30.458330    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dbb93860-c0e8-4d5c-8844-1142f7a7d187","Type":"ContainerDied","Data":"61e6492e60e09c1d082eb2d1d947538c85b634fe0e269717d94dd93f425ab4b7"}
Jul 27 19:30:30.460062 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:30.460003    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-564f6df4f-d4kcc" event={"ID":"8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f","Type":"ContainerStarted","Data":"13e1c9219ffe4125051337171b57b20d5d7a95b5920ae36f16373b4fbccdf2fc"}
Jul 27 19:30:30.462545 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:30.462526    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-65b64784b7-vnbrq" event={"ID":"489d2925-8a91-49b6-8a13-0280ea8b8bc2","Type":"ContainerStarted","Data":"a073d135c7ade990f85fa0d62360018bb81c8aeeec29767134e12557ba218514"}
Jul 27 19:30:30.462655 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:30.462566    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-65b64784b7-vnbrq" event={"ID":"489d2925-8a91-49b6-8a13-0280ea8b8bc2","Type":"ContainerStarted","Data":"4f8592b686fee41cfb13469c593d52039b44c120ad70b71808caa96e9280d9a4"}
Jul 27 19:30:30.462655 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:30.462593    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-65b64784b7-vnbrq" event={"ID":"489d2925-8a91-49b6-8a13-0280ea8b8bc2","Type":"ContainerStarted","Data":"473ffe686e38cfeec326cf68188ec370cde2e7a4f2756e93645e12ae71483e93"}
Jul 27 19:30:30.464723 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:30.464704    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-p4c8p" event={"ID":"3a5c72a0-8769-4caf-80e4-cfc8db418c38","Type":"ContainerStarted","Data":"5c02922a91ac4ce969704c91d9dbfed29bdab462ab0de8643a42f8db040dddcb"}
Jul 27 19:30:30.466391 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:30.466373    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cc4fc4b9c-mnk9w" event={"ID":"67d6f95c-e959-40a5-8dfa-39228c3b963f","Type":"ContainerStarted","Data":"8388a47740ba18c4ea434d482ea6b292a5fd6a8dd85571762c5dd787c58d00de"}
Jul 27 19:30:30.472484 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:30.472455    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-55cdff9587-pc94t" event={"ID":"94c83f10-a0af-4c99-b91f-14bac2e99cfd","Type":"ContainerStarted","Data":"773bfe8802092b1797fad9c652d0d9b4cac513d3167a85d1d3ad5b38eff9b473"}
Jul 27 19:30:30.475937 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:30.475913    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-pwsmc" event={"ID":"9b70adeb-3b2e-41f2-a157-6b2dfe797669","Type":"ContainerStarted","Data":"1c8541d9177107c07470831d502e0e9b30029287a16b4f2a51a549bec66e8087"}
Jul 27 19:30:30.476043 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:30.475966    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-pwsmc" event={"ID":"9b70adeb-3b2e-41f2-a157-6b2dfe797669","Type":"ContainerStarted","Data":"751fdeae57b24a0763560ec36070b1bdd48d38e5e8e8673854387fed2c9df5f6"}
Jul 27 19:30:30.476043 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:30.475988    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-pwsmc" event={"ID":"9b70adeb-3b2e-41f2-a157-6b2dfe797669","Type":"ContainerStarted","Data":"c8e24c3f6fffb7f2c5eb0bf693f9dd37076c05d59c0edad857540be7e7acc4a4"}
Jul 27 19:30:30.477875 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:30.477849    2558 generic.go:356] "Generic (PLEG): container finished" podID="4aa68886-81b2-47c7-9aef-c6d73404122f" containerID="dd4ca41d4c62c5bc6ce028a100b8cbe1ab7f60a909e84e1ac86e7818e7b4dde6" exitCode=0
Jul 27 19:30:30.477972 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:30.477885    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b65wp" event={"ID":"4aa68886-81b2-47c7-9aef-c6d73404122f","Type":"ContainerDied","Data":"dd4ca41d4c62c5bc6ce028a100b8cbe1ab7f60a909e84e1ac86e7818e7b4dde6"}
Jul 27 19:30:30.504683 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:30.504626    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-65b64784b7-vnbrq" podStartSLOduration=4.498011592 podStartE2EDuration="15.50460028s" totalImagesPullingTime="11.006588688s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:30:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:30:30.503454247 +0000 UTC m=+243.375646220" watchObservedRunningTime="2026-07-27 19:30:30.50460028 +0000 UTC m=+243.376792235"
Jul 27 19:30:30.538221 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:30.538173    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-564f6df4f-d4kcc" podStartSLOduration=11.926402786 podStartE2EDuration="22.538156315s" totalImagesPullingTime="10.611753529s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:30:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:30:30.520328859 +0000 UTC m=+243.392520833" watchObservedRunningTime="2026-07-27 19:30:30.538156315 +0000 UTC m=+243.410348294"
Jul 27 19:30:30.556422 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:30.556355    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-pwsmc" podStartSLOduration=9.952429207 podStartE2EDuration="20.556332764s" totalImagesPullingTime="10.603903557s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:30:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:30:30.554887921 +0000 UTC m=+243.427079899" watchObservedRunningTime="2026-07-27 19:30:30.556332764 +0000 UTC m=+243.428524739"
Jul 27 19:30:30.572384 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:30.572333    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-p4c8p" podStartSLOduration=10.14979001 podStartE2EDuration="20.572311994s" totalImagesPullingTime="10.422521984s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:30:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:30:30.571014327 +0000 UTC m=+243.443206300" watchObservedRunningTime="2026-07-27 19:30:30.572311994 +0000 UTC m=+243.444503970"
Jul 27 19:30:30.588936 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:30.588885    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-55cdff9587-pc94t" podStartSLOduration=6.017811402 podStartE2EDuration="16.588867567s" totalImagesPullingTime="10.571056165s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:30:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:30:30.586980824 +0000 UTC m=+243.459172799" watchObservedRunningTime="2026-07-27 19:30:30.588867567 +0000 UTC m=+243.461059615"
Jul 27 19:30:30.604899 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:30.604847    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6cc4fc4b9c-mnk9w" podStartSLOduration=3.025824261 podStartE2EDuration="13.604824817s" totalImagesPullingTime="10.579000556s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:30:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:30:30.603526215 +0000 UTC m=+243.475718192" watchObservedRunningTime="2026-07-27 19:30:30.604824817 +0000 UTC m=+243.477016793"
Jul 27 19:30:31.436993 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:31.436956    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-564f6df4f-d4kcc"]
Jul 27 19:30:31.476448 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:31.475387    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6c49bdcbd7-xc4js"]
Jul 27 19:30:31.501906 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:31.501793    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c49bdcbd7-xc4js"]
Jul 27 19:30:31.501906 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:31.501833    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b65wp" event={"ID":"4aa68886-81b2-47c7-9aef-c6d73404122f","Type":"ContainerStarted","Data":"5c36daad4db794e80b18be6620a594ba412094e7274d8d6c7d6cb307779ecbee"}
Jul 27 19:30:31.501906 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:31.501865    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b65wp" event={"ID":"4aa68886-81b2-47c7-9aef-c6d73404122f","Type":"ContainerStarted","Data":"15cc7f9645e915ecd9b34375248d4b60753c5250001ab46e53696cf27f2ef892"}
Jul 27 19:30:31.502167 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:31.501918    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c49bdcbd7-xc4js"
Jul 27 19:30:31.510437 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:31.509882    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-b65wp" podStartSLOduration=11.357035102 podStartE2EDuration="21.509863137s" totalImagesPullingTime="10.043335029s" totalInitContainerRuntime="109.493006ms" isStatefulPod=true podCreationTimestamp="2026-07-27 19:30:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:30:31.508332314 +0000 UTC m=+244.380524291" watchObservedRunningTime="2026-07-27 19:30:31.509863137 +0000 UTC m=+244.382055112"
Jul 27 19:30:31.579298 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:31.579262    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5-oauth-serving-cert\") pod \"console-6c49bdcbd7-xc4js\" (UID: \"fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5\") " pod="openshift-console/console-6c49bdcbd7-xc4js"
Jul 27 19:30:31.580305 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:31.579757    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5-console-serving-cert\") pod \"console-6c49bdcbd7-xc4js\" (UID: \"fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5\") " pod="openshift-console/console-6c49bdcbd7-xc4js"
Jul 27 19:30:31.580305 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:31.579807    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5-console-oauth-config\") pod \"console-6c49bdcbd7-xc4js\" (UID: \"fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5\") " pod="openshift-console/console-6c49bdcbd7-xc4js"
Jul 27 19:30:31.580305 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:31.580040    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5-trusted-ca-bundle\") pod \"console-6c49bdcbd7-xc4js\" (UID: \"fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5\") " pod="openshift-console/console-6c49bdcbd7-xc4js"
Jul 27 19:30:31.580305 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:31.580261    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5-service-ca\") pod \"console-6c49bdcbd7-xc4js\" (UID: \"fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5\") " pod="openshift-console/console-6c49bdcbd7-xc4js"
Jul 27 19:30:31.580643 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:31.580515    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5b5m4\" (UniqueName: \"kubernetes.io/projected/fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5-kube-api-access-5b5m4\") pod \"console-6c49bdcbd7-xc4js\" (UID: \"fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5\") " pod="openshift-console/console-6c49bdcbd7-xc4js"
Jul 27 19:30:31.581156 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:31.580768    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5-console-config\") pod \"console-6c49bdcbd7-xc4js\" (UID: \"fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5\") " pod="openshift-console/console-6c49bdcbd7-xc4js"
Jul 27 19:30:31.681864 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:31.681824    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5-console-serving-cert\") pod \"console-6c49bdcbd7-xc4js\" (UID: \"fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5\") " pod="openshift-console/console-6c49bdcbd7-xc4js"
Jul 27 19:30:31.681864 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:31.681866    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5-console-oauth-config\") pod \"console-6c49bdcbd7-xc4js\" (UID: \"fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5\") " pod="openshift-console/console-6c49bdcbd7-xc4js"
Jul 27 19:30:31.682102 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:31.681917    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5-trusted-ca-bundle\") pod \"console-6c49bdcbd7-xc4js\" (UID: \"fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5\") " pod="openshift-console/console-6c49bdcbd7-xc4js"
Jul 27 19:30:31.682102 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:31.681961    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5-service-ca\") pod \"console-6c49bdcbd7-xc4js\" (UID: \"fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5\") " pod="openshift-console/console-6c49bdcbd7-xc4js"
Jul 27 19:30:31.682102 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:31.681994    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5b5m4\" (UniqueName: \"kubernetes.io/projected/fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5-kube-api-access-5b5m4\") pod \"console-6c49bdcbd7-xc4js\" (UID: \"fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5\") " pod="openshift-console/console-6c49bdcbd7-xc4js"
Jul 27 19:30:31.682102 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:31.682053    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5-console-config\") pod \"console-6c49bdcbd7-xc4js\" (UID: \"fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5\") " pod="openshift-console/console-6c49bdcbd7-xc4js"
Jul 27 19:30:31.682102 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:31.682097    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5-oauth-serving-cert\") pod \"console-6c49bdcbd7-xc4js\" (UID: \"fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5\") " pod="openshift-console/console-6c49bdcbd7-xc4js"
Jul 27 19:30:31.683444 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:31.683370    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5-console-config\") pod \"console-6c49bdcbd7-xc4js\" (UID: \"fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5\") " pod="openshift-console/console-6c49bdcbd7-xc4js"
Jul 27 19:30:31.683444 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:31.683370    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5-service-ca\") pod \"console-6c49bdcbd7-xc4js\" (UID: \"fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5\") " pod="openshift-console/console-6c49bdcbd7-xc4js"
Jul 27 19:30:31.683645 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:31.683487    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5-oauth-serving-cert\") pod \"console-6c49bdcbd7-xc4js\" (UID: \"fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5\") " pod="openshift-console/console-6c49bdcbd7-xc4js"
Jul 27 19:30:31.685444 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:31.685399    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5-trusted-ca-bundle\") pod \"console-6c49bdcbd7-xc4js\" (UID: \"fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5\") " pod="openshift-console/console-6c49bdcbd7-xc4js"
Jul 27 19:30:31.690555 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:31.690475    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5-console-oauth-config\") pod \"console-6c49bdcbd7-xc4js\" (UID: \"fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5\") " pod="openshift-console/console-6c49bdcbd7-xc4js"
Jul 27 19:30:31.692261 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:31.692235    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5-console-serving-cert\") pod \"console-6c49bdcbd7-xc4js\" (UID: \"fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5\") " pod="openshift-console/console-6c49bdcbd7-xc4js"
Jul 27 19:30:31.692372 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:31.692261    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5b5m4\" (UniqueName: \"kubernetes.io/projected/fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5-kube-api-access-5b5m4\") pod \"console-6c49bdcbd7-xc4js\" (UID: \"fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5\") " pod="openshift-console/console-6c49bdcbd7-xc4js"
Jul 27 19:30:31.814717 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:31.814674    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c49bdcbd7-xc4js"
Jul 27 19:30:31.985541 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:31.985308    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c49bdcbd7-xc4js"]
Jul 27 19:30:32.359794 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:30:32.359712    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe93b772_7ae3_4f3b_ba41_01cd07a5f9f5.slice/crio-36d9b56c1e74898e3b864919dd83396f9b070eafa4c099b9993a6415536b4554 WatchSource:0}: Error finding container 36d9b56c1e74898e3b864919dd83396f9b070eafa4c099b9993a6415536b4554: Status 404 returned error can't find the container with id 36d9b56c1e74898e3b864919dd83396f9b070eafa4c099b9993a6415536b4554
Jul 27 19:30:32.492449 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:32.492398    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c49bdcbd7-xc4js" event={"ID":"fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5","Type":"ContainerStarted","Data":"36d9b56c1e74898e3b864919dd83396f9b070eafa4c099b9993a6415536b4554"}
Jul 27 19:30:32.860459 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:32.860404    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:32.926316 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:32.925824    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:34.278675 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:34.278637    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:34.927757 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:34.927684    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-55cdff9587-pc94t"
Jul 27 19:30:34.927757 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:34.927756    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-55cdff9587-pc94t"
Jul 27 19:30:35.505426 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:35.505330    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jscrz" event={"ID":"8bb804bd-b63d-43b7-8159-80cd1abfab55","Type":"ContainerStarted","Data":"de795ba74aeb5ef84f440e49548f3161e722b3e6dbcff12492a7cc1c0a3e16be"}
Jul 27 19:30:35.508366 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:35.508338    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dbb93860-c0e8-4d5c-8844-1142f7a7d187","Type":"ContainerStarted","Data":"739d6c59010607ca4447e341f0241ccaab34b8cfd2977b36e0786e2ef2d742e4"}
Jul 27 19:30:35.508530 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:35.508431    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dbb93860-c0e8-4d5c-8844-1142f7a7d187","Type":"ContainerStarted","Data":"ccfd8d4bfa9aa9c2ad0b6fa9bea4e877e4ec0ba814a2b093ff255d1420d2be62"}
Jul 27 19:30:35.508530 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:35.508472    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dbb93860-c0e8-4d5c-8844-1142f7a7d187","Type":"ContainerStarted","Data":"87adc48d29aebe74aec150cf7f525fb04b39d152c761c051debc531d2cc8920f"}
Jul 27 19:30:35.508530 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:35.508509    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dbb93860-c0e8-4d5c-8844-1142f7a7d187","Type":"ContainerStarted","Data":"e8fd0f725d10315f19df2e17e79be0653712edaf457e5eb9adf8b8588080744d"}
Jul 27 19:30:35.508678 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:35.508545    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dbb93860-c0e8-4d5c-8844-1142f7a7d187","Type":"ContainerStarted","Data":"8855e85fdc59ba86ab264527b046d6f96cebe18f8684aaf3d7f38f928903dcde"}
Jul 27 19:30:35.509826 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:35.509809    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c49bdcbd7-xc4js" event={"ID":"fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5","Type":"ContainerStarted","Data":"c6250472b46311a98faad58869e0008b0271dec3ce68bb97e9390a12e7571f03"}
Jul 27 19:30:35.521959 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:35.521911    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-jscrz" podStartSLOduration=33.63422918 podStartE2EDuration="1m0.521898268s" totalImagesPullingTime="26.887669088s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 19:29:35 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:30:35.520792718 +0000 UTC m=+248.392984694" watchObservedRunningTime="2026-07-27 19:30:35.521898268 +0000 UTC m=+248.394090242"
Jul 27 19:30:35.537184 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:35.537140    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6c49bdcbd7-xc4js" podStartSLOduration=4.537124231 podStartE2EDuration="4.537124231s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:30:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:30:35.535288726 +0000 UTC m=+248.407480701" watchObservedRunningTime="2026-07-27 19:30:35.537124231 +0000 UTC m=+248.409316204"
Jul 27 19:30:37.522196 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:37.522162    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dbb93860-c0e8-4d5c-8844-1142f7a7d187","Type":"ContainerStarted","Data":"e667c829ad34c078d62cb2ba95216c8b19ef24878a4c73cb892adf2797b92cb2"}
Jul 27 19:30:37.532073 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:37.532052    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6cc4fc4b9c-mnk9w"
Jul 27 19:30:37.532203 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:37.532089    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6cc4fc4b9c-mnk9w"
Jul 27 19:30:37.536865 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:37.536845    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6cc4fc4b9c-mnk9w"
Jul 27 19:30:37.548009 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:37.547967    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=11.22516684 podStartE2EDuration="26.547954536s" totalImagesPullingTime="15.300269619s" totalInitContainerRuntime="22.518077ms" isStatefulPod=false podCreationTimestamp="2026-07-27 19:30:11 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:30:37.545689754 +0000 UTC m=+250.417881729" watchObservedRunningTime="2026-07-27 19:30:37.547954536 +0000 UTC m=+250.420146509"
Jul 27 19:30:38.529752 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:38.529724    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6cc4fc4b9c-mnk9w"
Jul 27 19:30:38.654742 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:38.654714    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:38.736936 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:38.736908    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-564f6df4f-d4kcc"
Jul 27 19:30:38.753892 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:38.753837    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:40.195710 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:40.195683    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_dbb93860-c0e8-4d5c-8844-1142f7a7d187/init-config-reloader/0.log"
Jul 27 19:30:40.201252 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:40.201231    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_dbb93860-c0e8-4d5c-8844-1142f7a7d187/alertmanager/0.log"
Jul 27 19:30:40.359424 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:40.359388    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_dbb93860-c0e8-4d5c-8844-1142f7a7d187/config-reloader/0.log"
Jul 27 19:30:40.558708 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:40.558637    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_dbb93860-c0e8-4d5c-8844-1142f7a7d187/kube-rbac-proxy-web/0.log"
Jul 27 19:30:40.759298 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:40.759273    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_dbb93860-c0e8-4d5c-8844-1142f7a7d187/kube-rbac-proxy/0.log"
Jul 27 19:30:40.806575 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:40.806550    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:40.909541 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:40.909508    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:40.958543 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:40.958514    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_dbb93860-c0e8-4d5c-8844-1142f7a7d187/kube-rbac-proxy-metric/0.log"
Jul 27 19:30:41.158989 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:41.158963    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_dbb93860-c0e8-4d5c-8844-1142f7a7d187/prom-label-proxy/0.log"
Jul 27 19:30:41.559447 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:41.559382    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-pwsmc_9b70adeb-3b2e-41f2-a157-6b2dfe797669/kube-state-metrics/0.log"
Jul 27 19:30:41.759006 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:41.758940    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-pwsmc_9b70adeb-3b2e-41f2-a157-6b2dfe797669/kube-rbac-proxy-main/0.log"
Jul 27 19:30:41.815236 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:41.815161    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6c49bdcbd7-xc4js"
Jul 27 19:30:41.815236 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:41.815217    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6c49bdcbd7-xc4js"
Jul 27 19:30:41.820273 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:41.820250    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6c49bdcbd7-xc4js"
Jul 27 19:30:41.959002 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:41.958968    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-pwsmc_9b70adeb-3b2e-41f2-a157-6b2dfe797669/kube-rbac-proxy-self/0.log"
Jul 27 19:30:42.158829 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:42.158802    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-55cdff9587-pc94t_94c83f10-a0af-4c99-b91f-14bac2e99cfd/metrics-server/0.log"
Jul 27 19:30:42.544502 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:42.544423    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6c49bdcbd7-xc4js"
Jul 27 19:30:42.588206 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:42.588167    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6cc4fc4b9c-mnk9w"]
Jul 27 19:30:42.684933 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:42.684906    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:42.786815 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:42.786781    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:42.849735 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:42.849664    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:43.158500 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:43.158477    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b65wp_4aa68886-81b2-47c7-9aef-c6d73404122f/init-textfile/0.log"
Jul 27 19:30:43.360109 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:43.360081    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b65wp_4aa68886-81b2-47c7-9aef-c6d73404122f/node-exporter/0.log"
Jul 27 19:30:43.560912 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:43.560842    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b65wp_4aa68886-81b2-47c7-9aef-c6d73404122f/kube-rbac-proxy/0.log"
Jul 27 19:30:44.359239 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:44.359213    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-p4c8p_3a5c72a0-8769-4caf-80e4-cfc8db418c38/kube-rbac-proxy-main/0.log"
Jul 27 19:30:44.558837 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:44.558803    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-p4c8p_3a5c72a0-8769-4caf-80e4-cfc8db418c38/kube-rbac-proxy-self/0.log"
Jul 27 19:30:44.758990 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:44.758962    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-p4c8p_3a5c72a0-8769-4caf-80e4-cfc8db418c38/openshift-state-metrics/0.log"
Jul 27 19:30:46.090564 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:46.090531    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7f4bbb7d8b-kjzcn" podUID="a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6" containerName="registry" containerID="cri-o://5eea4f12c152892d9b493303d402ddf619f10dc681341f350c6a1baeebd09083" gracePeriod=30
Jul 27 19:30:46.390195 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:46.390168    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f4bbb7d8b-kjzcn"
Jul 27 19:30:46.517225 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:46.517189    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6-trusted-ca\") pod \"a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6\" (UID: \"a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6\") "
Jul 27 19:30:46.517225 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:46.517222    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6-ca-trust-extracted\") pod \"a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6\" (UID: \"a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6\") "
Jul 27 19:30:46.517478 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:46.517250    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6-bound-sa-token\") pod \"a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6\" (UID: \"a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6\") "
Jul 27 19:30:46.517478 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:46.517290    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vngjq\" (UniqueName: \"kubernetes.io/projected/a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6-kube-api-access-vngjq\") pod \"a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6\" (UID: \"a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6\") "
Jul 27 19:30:46.517478 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:46.517324    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6-registry-tls\") pod \"a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6\" (UID: \"a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6\") "
Jul 27 19:30:46.517478 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:46.517360    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6-registry-certificates\") pod \"a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6\" (UID: \"a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6\") "
Jul 27 19:30:46.517478 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:46.517437    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6-installation-pull-secrets\") pod \"a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6\" (UID: \"a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6\") "
Jul 27 19:30:46.517728 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:46.517490    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6-image-registry-private-configuration\") pod \"a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6\" (UID: \"a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6\") "
Jul 27 19:30:46.517728 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:46.517663    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6" (UID: "a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:30:46.517911 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:46.517823    2558 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6-trusted-ca\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:30:46.518196 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:46.518168    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6" (UID: "a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:30:46.520007 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:46.519971    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6" (UID: "a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:30:46.520111 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:46.520078    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6" (UID: "a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:30:46.520158 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:46.520113    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6" (UID: "a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:30:46.520158 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:46.520130    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6-kube-api-access-vngjq" (OuterVolumeSpecName: "kube-api-access-vngjq") pod "a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6" (UID: "a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6"). InnerVolumeSpecName "kube-api-access-vngjq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:30:46.520222 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:46.520157    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6" (UID: "a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:30:46.526436 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:46.526396    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6" (UID: "a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:30:46.557846 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:46.557813    2558 generic.go:356] "Generic (PLEG): container finished" podID="a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6" containerID="5eea4f12c152892d9b493303d402ddf619f10dc681341f350c6a1baeebd09083" exitCode=0
Jul 27 19:30:46.557942 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:46.557864    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f4bbb7d8b-kjzcn"
Jul 27 19:30:46.557942 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:46.557899    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7f4bbb7d8b-kjzcn" event={"ID":"a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6","Type":"ContainerDied","Data":"5eea4f12c152892d9b493303d402ddf619f10dc681341f350c6a1baeebd09083"}
Jul 27 19:30:46.558013 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:46.557950    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7f4bbb7d8b-kjzcn" event={"ID":"a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6","Type":"ContainerDied","Data":"c5e9cdbd95b9e6dbf015117436a93f4609d7ada88637f522cd95f7b2c60eaa7a"}
Jul 27 19:30:46.558013 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:46.557971    2558 scope.go:117] "RemoveContainer" containerID="5eea4f12c152892d9b493303d402ddf619f10dc681341f350c6a1baeebd09083"
Jul 27 19:30:46.566267 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:46.566251    2558 scope.go:117] "RemoveContainer" containerID="5eea4f12c152892d9b493303d402ddf619f10dc681341f350c6a1baeebd09083"
Jul 27 19:30:46.566540 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:30:46.566520    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5eea4f12c152892d9b493303d402ddf619f10dc681341f350c6a1baeebd09083\": container with ID starting with 5eea4f12c152892d9b493303d402ddf619f10dc681341f350c6a1baeebd09083 not found: ID does not exist" containerID="5eea4f12c152892d9b493303d402ddf619f10dc681341f350c6a1baeebd09083"
Jul 27 19:30:46.566609 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:46.566551    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5eea4f12c152892d9b493303d402ddf619f10dc681341f350c6a1baeebd09083"} err="failed to get container status \"5eea4f12c152892d9b493303d402ddf619f10dc681341f350c6a1baeebd09083\": rpc error: code = NotFound desc = could not find container \"5eea4f12c152892d9b493303d402ddf619f10dc681341f350c6a1baeebd09083\": container with ID starting with 5eea4f12c152892d9b493303d402ddf619f10dc681341f350c6a1baeebd09083 not found: ID does not exist"
Jul 27 19:30:46.577123 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:46.577101    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7f4bbb7d8b-kjzcn"]
Jul 27 19:30:46.580543 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:46.580525    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7f4bbb7d8b-kjzcn"]
Jul 27 19:30:46.619165 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:46.619142    2558 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6-bound-sa-token\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:30:46.619165 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:46.619163    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vngjq\" (UniqueName: \"kubernetes.io/projected/a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6-kube-api-access-vngjq\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:30:46.619313 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:46.619172    2558 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6-registry-tls\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:30:46.619313 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:46.619181    2558 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6-registry-certificates\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:30:46.619313 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:46.619190    2558 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6-installation-pull-secrets\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:30:46.619313 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:46.619199    2558 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6-image-registry-private-configuration\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:30:46.619313 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:46.619207    2558 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6-ca-trust-extracted\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:30:46.959468 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:46.959341    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-65b64784b7-vnbrq_489d2925-8a91-49b6-8a13-0280ea8b8bc2/telemeter-client/0.log"
Jul 27 19:30:47.159533 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:47.159492    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-65b64784b7-vnbrq_489d2925-8a91-49b6-8a13-0280ea8b8bc2/reload/0.log"
Jul 27 19:30:47.359463 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:47.359324    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-65b64784b7-vnbrq_489d2925-8a91-49b6-8a13-0280ea8b8bc2/kube-rbac-proxy/0.log"
Jul 27 19:30:47.673325 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:47.673256    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6" path="/var/lib/kubelet/pods/a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6/volumes"
Jul 27 19:30:49.359178 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:49.359136    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-564f6df4f-d4kcc_8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f/console/0.log"
Jul 27 19:30:49.559239 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:49.559196    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6c49bdcbd7-xc4js_fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5/console/0.log"
Jul 27 19:30:49.758908 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:49.758881    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6cc4fc4b9c-mnk9w_67d6f95c-e959-40a5-8dfa-39228c3b963f/console/0.log"
Jul 27 19:30:49.960323 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:49.960297    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-7c97dc4555-qm44q_212c42c2-b2a1-4082-8ea8-2a7f867bd45a/download-server/0.log"
Jul 27 19:30:51.957795 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:51.957765    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:54.933274 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:54.933245    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-55cdff9587-pc94t"
Jul 27 19:30:54.937008 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:54.936988    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-55cdff9587-pc94t"
Jul 27 19:30:55.866021 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:55.865988    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:55.965869 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:55.965840    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:57.513307 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:57.513266    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-564f6df4f-d4kcc" podUID="8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f" containerName="console" containerID="cri-o://13e1c9219ffe4125051337171b57b20d5d7a95b5920ae36f16373b4fbccdf2fc" gracePeriod=15
Jul 27 19:30:58.308499 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:58.308465    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-564f6df4f-d4kcc"
Jul 27 19:30:58.325224 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:58.325192    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f-console-oauth-config\") pod \"8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f\" (UID: \"8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f\") "
Jul 27 19:30:58.325389 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:58.325232    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f-service-ca\") pod \"8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f\" (UID: \"8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f\") "
Jul 27 19:30:58.325389 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:58.325260    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f-console-config\") pod \"8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f\" (UID: \"8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f\") "
Jul 27 19:30:58.325542 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:58.325380    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6k2kt\" (UniqueName: \"kubernetes.io/projected/8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f-kube-api-access-6k2kt\") pod \"8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f\" (UID: \"8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f\") "
Jul 27 19:30:58.325542 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:58.325475    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f-oauth-serving-cert\") pod \"8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f\" (UID: \"8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f\") "
Jul 27 19:30:58.325542 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:58.325506    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f-console-serving-cert\") pod \"8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f\" (UID: \"8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f\") "
Jul 27 19:30:58.325685 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:58.325650    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f-service-ca" (OuterVolumeSpecName: "service-ca") pod "8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f" (UID: "8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:30:58.325685 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:58.325656    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f-console-config" (OuterVolumeSpecName: "console-config") pod "8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f" (UID: "8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:30:58.325832 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:58.325813    2558 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f-service-ca\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:30:58.325907 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:58.325837    2558 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f-console-config\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:30:58.326006 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:58.325977    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f" (UID: "8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:30:58.328053 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:58.328029    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f" (UID: "8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:30:58.328165 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:58.328081    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f-kube-api-access-6k2kt" (OuterVolumeSpecName: "kube-api-access-6k2kt") pod "8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f" (UID: "8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f"). InnerVolumeSpecName "kube-api-access-6k2kt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:30:58.328165 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:58.328151    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f" (UID: "8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:30:58.426288 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:58.426210    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6k2kt\" (UniqueName: \"kubernetes.io/projected/8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f-kube-api-access-6k2kt\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:30:58.426288 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:58.426239    2558 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f-oauth-serving-cert\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:30:58.426288 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:58.426250    2558 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f-console-serving-cert\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:30:58.426288 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:58.426259    2558 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f-console-oauth-config\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:30:58.608607 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:58.608575    2558 generic.go:356] "Generic (PLEG): container finished" podID="8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f" containerID="13e1c9219ffe4125051337171b57b20d5d7a95b5920ae36f16373b4fbccdf2fc" exitCode=0
Jul 27 19:30:58.609091 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:58.608636    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-564f6df4f-d4kcc"
Jul 27 19:30:58.609091 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:58.608660    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-564f6df4f-d4kcc" event={"ID":"8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f","Type":"ContainerDied","Data":"13e1c9219ffe4125051337171b57b20d5d7a95b5920ae36f16373b4fbccdf2fc"}
Jul 27 19:30:58.609091 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:58.608709    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-564f6df4f-d4kcc" event={"ID":"8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f","Type":"ContainerDied","Data":"b098c5ee698e2d655b8c5563e5e035d53ccee4a169a6ebdfe70b1e020e6e2231"}
Jul 27 19:30:58.609091 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:58.608730    2558 scope.go:117] "RemoveContainer" containerID="13e1c9219ffe4125051337171b57b20d5d7a95b5920ae36f16373b4fbccdf2fc"
Jul 27 19:30:58.619352 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:58.619328    2558 scope.go:117] "RemoveContainer" containerID="13e1c9219ffe4125051337171b57b20d5d7a95b5920ae36f16373b4fbccdf2fc"
Jul 27 19:30:58.619683 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:30:58.619656    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13e1c9219ffe4125051337171b57b20d5d7a95b5920ae36f16373b4fbccdf2fc\": container with ID starting with 13e1c9219ffe4125051337171b57b20d5d7a95b5920ae36f16373b4fbccdf2fc not found: ID does not exist" containerID="13e1c9219ffe4125051337171b57b20d5d7a95b5920ae36f16373b4fbccdf2fc"
Jul 27 19:30:58.619746 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:58.619696    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13e1c9219ffe4125051337171b57b20d5d7a95b5920ae36f16373b4fbccdf2fc"} err="failed to get container status \"13e1c9219ffe4125051337171b57b20d5d7a95b5920ae36f16373b4fbccdf2fc\": rpc error: code = NotFound desc = could not find container \"13e1c9219ffe4125051337171b57b20d5d7a95b5920ae36f16373b4fbccdf2fc\": container with ID starting with 13e1c9219ffe4125051337171b57b20d5d7a95b5920ae36f16373b4fbccdf2fc not found: ID does not exist"
Jul 27 19:30:58.630677 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:58.630652    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-564f6df4f-d4kcc"]
Jul 27 19:30:58.634036 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:58.634014    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-564f6df4f-d4kcc"]
Jul 27 19:30:59.674018 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:30:59.673979    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f" path="/var/lib/kubelet/pods/8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f/volumes"
Jul 27 19:31:07.608723 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:07.608669    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6cc4fc4b9c-mnk9w" podUID="67d6f95c-e959-40a5-8dfa-39228c3b963f" containerName="console" containerID="cri-o://8388a47740ba18c4ea434d482ea6b292a5fd6a8dd85571762c5dd787c58d00de" gracePeriod=15
Jul 27 19:31:07.909272 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:07.909237    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cc4fc4b9c-mnk9w"
Jul 27 19:31:08.002669 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:08.002633    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/67d6f95c-e959-40a5-8dfa-39228c3b963f-console-serving-cert\") pod \"67d6f95c-e959-40a5-8dfa-39228c3b963f\" (UID: \"67d6f95c-e959-40a5-8dfa-39228c3b963f\") "
Jul 27 19:31:08.002669 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:08.002669    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/67d6f95c-e959-40a5-8dfa-39228c3b963f-oauth-serving-cert\") pod \"67d6f95c-e959-40a5-8dfa-39228c3b963f\" (UID: \"67d6f95c-e959-40a5-8dfa-39228c3b963f\") "
Jul 27 19:31:08.002908 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:08.002694    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/67d6f95c-e959-40a5-8dfa-39228c3b963f-console-oauth-config\") pod \"67d6f95c-e959-40a5-8dfa-39228c3b963f\" (UID: \"67d6f95c-e959-40a5-8dfa-39228c3b963f\") "
Jul 27 19:31:08.002908 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:08.002723    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67d6f95c-e959-40a5-8dfa-39228c3b963f-trusted-ca-bundle\") pod \"67d6f95c-e959-40a5-8dfa-39228c3b963f\" (UID: \"67d6f95c-e959-40a5-8dfa-39228c3b963f\") "
Jul 27 19:31:08.002908 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:08.002748    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qrd62\" (UniqueName: \"kubernetes.io/projected/67d6f95c-e959-40a5-8dfa-39228c3b963f-kube-api-access-qrd62\") pod \"67d6f95c-e959-40a5-8dfa-39228c3b963f\" (UID: \"67d6f95c-e959-40a5-8dfa-39228c3b963f\") "
Jul 27 19:31:08.002908 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:08.002797    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/67d6f95c-e959-40a5-8dfa-39228c3b963f-console-config\") pod \"67d6f95c-e959-40a5-8dfa-39228c3b963f\" (UID: \"67d6f95c-e959-40a5-8dfa-39228c3b963f\") "
Jul 27 19:31:08.002908 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:08.002845    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/67d6f95c-e959-40a5-8dfa-39228c3b963f-service-ca\") pod \"67d6f95c-e959-40a5-8dfa-39228c3b963f\" (UID: \"67d6f95c-e959-40a5-8dfa-39228c3b963f\") "
Jul 27 19:31:08.003154 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:08.003085    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67d6f95c-e959-40a5-8dfa-39228c3b963f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "67d6f95c-e959-40a5-8dfa-39228c3b963f" (UID: "67d6f95c-e959-40a5-8dfa-39228c3b963f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:31:08.003273 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:08.003246    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67d6f95c-e959-40a5-8dfa-39228c3b963f-console-config" (OuterVolumeSpecName: "console-config") pod "67d6f95c-e959-40a5-8dfa-39228c3b963f" (UID: "67d6f95c-e959-40a5-8dfa-39228c3b963f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:31:08.003273 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:08.003242    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67d6f95c-e959-40a5-8dfa-39228c3b963f-service-ca" (OuterVolumeSpecName: "service-ca") pod "67d6f95c-e959-40a5-8dfa-39228c3b963f" (UID: "67d6f95c-e959-40a5-8dfa-39228c3b963f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:31:08.003500 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:08.003317    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67d6f95c-e959-40a5-8dfa-39228c3b963f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "67d6f95c-e959-40a5-8dfa-39228c3b963f" (UID: "67d6f95c-e959-40a5-8dfa-39228c3b963f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:31:08.005038 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:08.005016    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/67d6f95c-e959-40a5-8dfa-39228c3b963f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "67d6f95c-e959-40a5-8dfa-39228c3b963f" (UID: "67d6f95c-e959-40a5-8dfa-39228c3b963f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:31:08.005038 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:08.005022    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67d6f95c-e959-40a5-8dfa-39228c3b963f-kube-api-access-qrd62" (OuterVolumeSpecName: "kube-api-access-qrd62") pod "67d6f95c-e959-40a5-8dfa-39228c3b963f" (UID: "67d6f95c-e959-40a5-8dfa-39228c3b963f"). InnerVolumeSpecName "kube-api-access-qrd62". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:31:08.005149 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:08.005073    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/67d6f95c-e959-40a5-8dfa-39228c3b963f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "67d6f95c-e959-40a5-8dfa-39228c3b963f" (UID: "67d6f95c-e959-40a5-8dfa-39228c3b963f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:31:08.104350 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:08.104314    2558 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/67d6f95c-e959-40a5-8dfa-39228c3b963f-service-ca\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:31:08.104350 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:08.104343    2558 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/67d6f95c-e959-40a5-8dfa-39228c3b963f-console-serving-cert\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:31:08.104350 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:08.104352    2558 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/67d6f95c-e959-40a5-8dfa-39228c3b963f-oauth-serving-cert\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:31:08.104350 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:08.104361    2558 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/67d6f95c-e959-40a5-8dfa-39228c3b963f-console-oauth-config\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:31:08.104666 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:08.104369    2558 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67d6f95c-e959-40a5-8dfa-39228c3b963f-trusted-ca-bundle\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:31:08.104666 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:08.104378    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qrd62\" (UniqueName: \"kubernetes.io/projected/67d6f95c-e959-40a5-8dfa-39228c3b963f-kube-api-access-qrd62\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:31:08.104666 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:08.104386    2558 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/67d6f95c-e959-40a5-8dfa-39228c3b963f-console-config\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:31:08.652866 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:08.652831    2558 generic.go:356] "Generic (PLEG): container finished" podID="67d6f95c-e959-40a5-8dfa-39228c3b963f" containerID="8388a47740ba18c4ea434d482ea6b292a5fd6a8dd85571762c5dd787c58d00de" exitCode=0
Jul 27 19:31:08.652866 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:08.652871    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cc4fc4b9c-mnk9w"
Jul 27 19:31:08.653294 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:08.652888    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cc4fc4b9c-mnk9w" event={"ID":"67d6f95c-e959-40a5-8dfa-39228c3b963f","Type":"ContainerDied","Data":"8388a47740ba18c4ea434d482ea6b292a5fd6a8dd85571762c5dd787c58d00de"}
Jul 27 19:31:08.653294 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:08.652934    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cc4fc4b9c-mnk9w" event={"ID":"67d6f95c-e959-40a5-8dfa-39228c3b963f","Type":"ContainerDied","Data":"f215442834ecd01042123100392046f6205e6d9f6dab1297ee703b920dcf1ff9"}
Jul 27 19:31:08.653294 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:08.652958    2558 scope.go:117] "RemoveContainer" containerID="8388a47740ba18c4ea434d482ea6b292a5fd6a8dd85571762c5dd787c58d00de"
Jul 27 19:31:08.661278 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:08.661260    2558 scope.go:117] "RemoveContainer" containerID="8388a47740ba18c4ea434d482ea6b292a5fd6a8dd85571762c5dd787c58d00de"
Jul 27 19:31:08.661584 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:31:08.661564    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8388a47740ba18c4ea434d482ea6b292a5fd6a8dd85571762c5dd787c58d00de\": container with ID starting with 8388a47740ba18c4ea434d482ea6b292a5fd6a8dd85571762c5dd787c58d00de not found: ID does not exist" containerID="8388a47740ba18c4ea434d482ea6b292a5fd6a8dd85571762c5dd787c58d00de"
Jul 27 19:31:08.661669 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:08.661590    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8388a47740ba18c4ea434d482ea6b292a5fd6a8dd85571762c5dd787c58d00de"} err="failed to get container status \"8388a47740ba18c4ea434d482ea6b292a5fd6a8dd85571762c5dd787c58d00de\": rpc error: code = NotFound desc = could not find container \"8388a47740ba18c4ea434d482ea6b292a5fd6a8dd85571762c5dd787c58d00de\": container with ID starting with 8388a47740ba18c4ea434d482ea6b292a5fd6a8dd85571762c5dd787c58d00de not found: ID does not exist"
Jul 27 19:31:08.672052 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:08.672021    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6cc4fc4b9c-mnk9w"]
Jul 27 19:31:08.679263 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:08.679225    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6cc4fc4b9c-mnk9w"]
Jul 27 19:31:09.673222 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:09.673185    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67d6f95c-e959-40a5-8dfa-39228c3b963f" path="/var/lib/kubelet/pods/67d6f95c-e959-40a5-8dfa-39228c3b963f/volumes"
Jul 27 19:31:26.268794 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:26.268758    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:31:26.369572 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:26.369541    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:31:27.487210 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:27.487181    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5dfc2_103bab23-bca3-472a-974b-7731f83c5bfb/ovn-acl-logging/0.log"
Jul 27 19:31:27.493736 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:27.493709    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5dfc2_103bab23-bca3-472a-974b-7731f83c5bfb/ovn-acl-logging/0.log"
Jul 27 19:31:27.495203 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:27.495180    2558 kubelet.go:1648] "Image garbage collection succeeded"
Jul 27 19:31:30.499968 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:30.499937    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 27 19:31:30.501755 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:30.500231    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="dbb93860-c0e8-4d5c-8844-1142f7a7d187" containerName="alertmanager" containerID="cri-o://8855e85fdc59ba86ab264527b046d6f96cebe18f8684aaf3d7f38f928903dcde" gracePeriod=120
Jul 27 19:31:30.501755 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:30.500310    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="dbb93860-c0e8-4d5c-8844-1142f7a7d187" containerName="kube-rbac-proxy-metric" containerID="cri-o://739d6c59010607ca4447e341f0241ccaab34b8cfd2977b36e0786e2ef2d742e4" gracePeriod=120
Jul 27 19:31:30.501755 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:30.500361    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="dbb93860-c0e8-4d5c-8844-1142f7a7d187" containerName="prom-label-proxy" containerID="cri-o://e667c829ad34c078d62cb2ba95216c8b19ef24878a4c73cb892adf2797b92cb2" gracePeriod=120
Jul 27 19:31:30.501755 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:30.500359    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="dbb93860-c0e8-4d5c-8844-1142f7a7d187" containerName="kube-rbac-proxy" containerID="cri-o://ccfd8d4bfa9aa9c2ad0b6fa9bea4e877e4ec0ba814a2b093ff255d1420d2be62" gracePeriod=120
Jul 27 19:31:30.501755 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:30.500320    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="dbb93860-c0e8-4d5c-8844-1142f7a7d187" containerName="kube-rbac-proxy-web" containerID="cri-o://87adc48d29aebe74aec150cf7f525fb04b39d152c761c051debc531d2cc8920f" gracePeriod=120
Jul 27 19:31:30.501755 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:30.500366    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="dbb93860-c0e8-4d5c-8844-1142f7a7d187" containerName="config-reloader" containerID="cri-o://e8fd0f725d10315f19df2e17e79be0653712edaf457e5eb9adf8b8588080744d" gracePeriod=120
Jul 27 19:31:30.740239 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:30.740202    2558 generic.go:356] "Generic (PLEG): container finished" podID="dbb93860-c0e8-4d5c-8844-1142f7a7d187" containerID="e667c829ad34c078d62cb2ba95216c8b19ef24878a4c73cb892adf2797b92cb2" exitCode=0
Jul 27 19:31:30.740239 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:30.740230    2558 generic.go:356] "Generic (PLEG): container finished" podID="dbb93860-c0e8-4d5c-8844-1142f7a7d187" containerID="739d6c59010607ca4447e341f0241ccaab34b8cfd2977b36e0786e2ef2d742e4" exitCode=0
Jul 27 19:31:30.740239 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:30.740237    2558 generic.go:356] "Generic (PLEG): container finished" podID="dbb93860-c0e8-4d5c-8844-1142f7a7d187" containerID="ccfd8d4bfa9aa9c2ad0b6fa9bea4e877e4ec0ba814a2b093ff255d1420d2be62" exitCode=0
Jul 27 19:31:30.740239 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:30.740243    2558 generic.go:356] "Generic (PLEG): container finished" podID="dbb93860-c0e8-4d5c-8844-1142f7a7d187" containerID="e8fd0f725d10315f19df2e17e79be0653712edaf457e5eb9adf8b8588080744d" exitCode=0
Jul 27 19:31:30.740239 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:30.740248    2558 generic.go:356] "Generic (PLEG): container finished" podID="dbb93860-c0e8-4d5c-8844-1142f7a7d187" containerID="8855e85fdc59ba86ab264527b046d6f96cebe18f8684aaf3d7f38f928903dcde" exitCode=0
Jul 27 19:31:30.740645 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:30.740275    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dbb93860-c0e8-4d5c-8844-1142f7a7d187","Type":"ContainerDied","Data":"e667c829ad34c078d62cb2ba95216c8b19ef24878a4c73cb892adf2797b92cb2"}
Jul 27 19:31:30.740645 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:30.740356    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dbb93860-c0e8-4d5c-8844-1142f7a7d187","Type":"ContainerDied","Data":"739d6c59010607ca4447e341f0241ccaab34b8cfd2977b36e0786e2ef2d742e4"}
Jul 27 19:31:30.740645 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:30.740395    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dbb93860-c0e8-4d5c-8844-1142f7a7d187","Type":"ContainerDied","Data":"ccfd8d4bfa9aa9c2ad0b6fa9bea4e877e4ec0ba814a2b093ff255d1420d2be62"}
Jul 27 19:31:30.740645 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:30.740451    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dbb93860-c0e8-4d5c-8844-1142f7a7d187","Type":"ContainerDied","Data":"e8fd0f725d10315f19df2e17e79be0653712edaf457e5eb9adf8b8588080744d"}
Jul 27 19:31:30.740645 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:30.740491    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dbb93860-c0e8-4d5c-8844-1142f7a7d187","Type":"ContainerDied","Data":"8855e85fdc59ba86ab264527b046d6f96cebe18f8684aaf3d7f38f928903dcde"}
Jul 27 19:31:31.748875 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:31.748842    2558 generic.go:356] "Generic (PLEG): container finished" podID="dbb93860-c0e8-4d5c-8844-1142f7a7d187" containerID="87adc48d29aebe74aec150cf7f525fb04b39d152c761c051debc531d2cc8920f" exitCode=0
Jul 27 19:31:31.749229 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:31.748900    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dbb93860-c0e8-4d5c-8844-1142f7a7d187","Type":"ContainerDied","Data":"87adc48d29aebe74aec150cf7f525fb04b39d152c761c051debc531d2cc8920f"}
Jul 27 19:31:31.749229 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:31.748971    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dbb93860-c0e8-4d5c-8844-1142f7a7d187","Type":"ContainerDied","Data":"4804dc8da4b19bee736657771879c34dcf69a1f022cae81978c958c2f2a6c979"}
Jul 27 19:31:31.749229 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:31.748998    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4804dc8da4b19bee736657771879c34dcf69a1f022cae81978c958c2f2a6c979"
Jul 27 19:31:31.757884 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:31.757860    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:31:31.911001 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:31.910962    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dbb93860-c0e8-4d5c-8844-1142f7a7d187-web-config\") pod \"dbb93860-c0e8-4d5c-8844-1142f7a7d187\" (UID: \"dbb93860-c0e8-4d5c-8844-1142f7a7d187\") "
Jul 27 19:31:31.911001 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:31.911003    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/dbb93860-c0e8-4d5c-8844-1142f7a7d187-secret-alertmanager-main-tls\") pod \"dbb93860-c0e8-4d5c-8844-1142f7a7d187\" (UID: \"dbb93860-c0e8-4d5c-8844-1142f7a7d187\") "
Jul 27 19:31:31.911229 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:31.911038    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mbxh8\" (UniqueName: \"kubernetes.io/projected/dbb93860-c0e8-4d5c-8844-1142f7a7d187-kube-api-access-mbxh8\") pod \"dbb93860-c0e8-4d5c-8844-1142f7a7d187\" (UID: \"dbb93860-c0e8-4d5c-8844-1142f7a7d187\") "
Jul 27 19:31:31.911229 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:31.911064    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dbb93860-c0e8-4d5c-8844-1142f7a7d187-metrics-client-ca\") pod \"dbb93860-c0e8-4d5c-8844-1142f7a7d187\" (UID: \"dbb93860-c0e8-4d5c-8844-1142f7a7d187\") "
Jul 27 19:31:31.911229 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:31.911096    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dbb93860-c0e8-4d5c-8844-1142f7a7d187-tls-assets\") pod \"dbb93860-c0e8-4d5c-8844-1142f7a7d187\" (UID: \"dbb93860-c0e8-4d5c-8844-1142f7a7d187\") "
Jul 27 19:31:31.911229 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:31.911119    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/dbb93860-c0e8-4d5c-8844-1142f7a7d187-secret-alertmanager-kube-rbac-proxy-metric\") pod \"dbb93860-c0e8-4d5c-8844-1142f7a7d187\" (UID: \"dbb93860-c0e8-4d5c-8844-1142f7a7d187\") "
Jul 27 19:31:31.911229 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:31.911148    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/dbb93860-c0e8-4d5c-8844-1142f7a7d187-alertmanager-main-db\") pod \"dbb93860-c0e8-4d5c-8844-1142f7a7d187\" (UID: \"dbb93860-c0e8-4d5c-8844-1142f7a7d187\") "
Jul 27 19:31:31.911229 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:31.911173    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/dbb93860-c0e8-4d5c-8844-1142f7a7d187-cluster-tls-config\") pod \"dbb93860-c0e8-4d5c-8844-1142f7a7d187\" (UID: \"dbb93860-c0e8-4d5c-8844-1142f7a7d187\") "
Jul 27 19:31:31.911229 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:31.911201    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dbb93860-c0e8-4d5c-8844-1142f7a7d187-secret-alertmanager-kube-rbac-proxy\") pod \"dbb93860-c0e8-4d5c-8844-1142f7a7d187\" (UID: \"dbb93860-c0e8-4d5c-8844-1142f7a7d187\") "
Jul 27 19:31:31.911229 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:31.911228    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dbb93860-c0e8-4d5c-8844-1142f7a7d187-secret-alertmanager-kube-rbac-proxy-web\") pod \"dbb93860-c0e8-4d5c-8844-1142f7a7d187\" (UID: \"dbb93860-c0e8-4d5c-8844-1142f7a7d187\") "
Jul 27 19:31:31.911663 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:31.911278    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dbb93860-c0e8-4d5c-8844-1142f7a7d187-config-out\") pod \"dbb93860-c0e8-4d5c-8844-1142f7a7d187\" (UID: \"dbb93860-c0e8-4d5c-8844-1142f7a7d187\") "
Jul 27 19:31:31.911663 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:31.911332    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dbb93860-c0e8-4d5c-8844-1142f7a7d187-alertmanager-trusted-ca-bundle\") pod \"dbb93860-c0e8-4d5c-8844-1142f7a7d187\" (UID: \"dbb93860-c0e8-4d5c-8844-1142f7a7d187\") "
Jul 27 19:31:31.911663 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:31.911360    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/dbb93860-c0e8-4d5c-8844-1142f7a7d187-config-volume\") pod \"dbb93860-c0e8-4d5c-8844-1142f7a7d187\" (UID: \"dbb93860-c0e8-4d5c-8844-1142f7a7d187\") "
Jul 27 19:31:31.911663 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:31.911507    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbb93860-c0e8-4d5c-8844-1142f7a7d187-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "dbb93860-c0e8-4d5c-8844-1142f7a7d187" (UID: "dbb93860-c0e8-4d5c-8844-1142f7a7d187"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:31:31.911663 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:31.911542    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dbb93860-c0e8-4d5c-8844-1142f7a7d187-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "dbb93860-c0e8-4d5c-8844-1142f7a7d187" (UID: "dbb93860-c0e8-4d5c-8844-1142f7a7d187"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:31:31.912081 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:31.911678    2558 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dbb93860-c0e8-4d5c-8844-1142f7a7d187-metrics-client-ca\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:31:31.912081 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:31.911698    2558 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/dbb93860-c0e8-4d5c-8844-1142f7a7d187-alertmanager-main-db\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:31:31.914566 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:31.913877    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dbb93860-c0e8-4d5c-8844-1142f7a7d187-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "dbb93860-c0e8-4d5c-8844-1142f7a7d187" (UID: "dbb93860-c0e8-4d5c-8844-1142f7a7d187"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:31:31.914566 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:31.913980    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dbb93860-c0e8-4d5c-8844-1142f7a7d187-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "dbb93860-c0e8-4d5c-8844-1142f7a7d187" (UID: "dbb93860-c0e8-4d5c-8844-1142f7a7d187"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:31:31.914566 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:31.914133    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbb93860-c0e8-4d5c-8844-1142f7a7d187-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "dbb93860-c0e8-4d5c-8844-1142f7a7d187" (UID: "dbb93860-c0e8-4d5c-8844-1142f7a7d187"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:31:31.915510 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:31.915477    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dbb93860-c0e8-4d5c-8844-1142f7a7d187-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "dbb93860-c0e8-4d5c-8844-1142f7a7d187" (UID: "dbb93860-c0e8-4d5c-8844-1142f7a7d187"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:31:31.915636 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:31.915593    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dbb93860-c0e8-4d5c-8844-1142f7a7d187-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "dbb93860-c0e8-4d5c-8844-1142f7a7d187" (UID: "dbb93860-c0e8-4d5c-8844-1142f7a7d187"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:31:31.915783 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:31.915765    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbb93860-c0e8-4d5c-8844-1142f7a7d187-kube-api-access-mbxh8" (OuterVolumeSpecName: "kube-api-access-mbxh8") pod "dbb93860-c0e8-4d5c-8844-1142f7a7d187" (UID: "dbb93860-c0e8-4d5c-8844-1142f7a7d187"). InnerVolumeSpecName "kube-api-access-mbxh8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:31:31.915853 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:31.915827    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dbb93860-c0e8-4d5c-8844-1142f7a7d187-config-volume" (OuterVolumeSpecName: "config-volume") pod "dbb93860-c0e8-4d5c-8844-1142f7a7d187" (UID: "dbb93860-c0e8-4d5c-8844-1142f7a7d187"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:31:31.915922 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:31.915898    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbb93860-c0e8-4d5c-8844-1142f7a7d187-config-out" (OuterVolumeSpecName: "config-out") pod "dbb93860-c0e8-4d5c-8844-1142f7a7d187" (UID: "dbb93860-c0e8-4d5c-8844-1142f7a7d187"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:31:31.916113 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:31.916093    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dbb93860-c0e8-4d5c-8844-1142f7a7d187-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "dbb93860-c0e8-4d5c-8844-1142f7a7d187" (UID: "dbb93860-c0e8-4d5c-8844-1142f7a7d187"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:31:31.919219 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:31.919186    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dbb93860-c0e8-4d5c-8844-1142f7a7d187-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "dbb93860-c0e8-4d5c-8844-1142f7a7d187" (UID: "dbb93860-c0e8-4d5c-8844-1142f7a7d187"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:31:31.925835 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:31.925809    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dbb93860-c0e8-4d5c-8844-1142f7a7d187-web-config" (OuterVolumeSpecName: "web-config") pod "dbb93860-c0e8-4d5c-8844-1142f7a7d187" (UID: "dbb93860-c0e8-4d5c-8844-1142f7a7d187"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:31:32.012590 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.012538    2558 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dbb93860-c0e8-4d5c-8844-1142f7a7d187-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:31:32.012590 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.012582    2558 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/dbb93860-c0e8-4d5c-8844-1142f7a7d187-config-volume\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:31:32.012590 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.012593    2558 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dbb93860-c0e8-4d5c-8844-1142f7a7d187-web-config\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:31:32.012590 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.012603    2558 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/dbb93860-c0e8-4d5c-8844-1142f7a7d187-secret-alertmanager-main-tls\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:31:32.012590 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.012612    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mbxh8\" (UniqueName: \"kubernetes.io/projected/dbb93860-c0e8-4d5c-8844-1142f7a7d187-kube-api-access-mbxh8\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:31:32.012940 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.012620    2558 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dbb93860-c0e8-4d5c-8844-1142f7a7d187-tls-assets\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:31:32.012940 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.012637    2558 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/dbb93860-c0e8-4d5c-8844-1142f7a7d187-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:31:32.012940 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.012649    2558 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/dbb93860-c0e8-4d5c-8844-1142f7a7d187-cluster-tls-config\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:31:32.012940 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.012659    2558 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dbb93860-c0e8-4d5c-8844-1142f7a7d187-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:31:32.012940 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.012668    2558 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dbb93860-c0e8-4d5c-8844-1142f7a7d187-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:31:32.012940 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.012676    2558 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dbb93860-c0e8-4d5c-8844-1142f7a7d187-config-out\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:31:32.094091 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.094053    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-56ff6c45bc-kcmv8"]
Jul 27 19:31:32.094323 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.094311    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dbb93860-c0e8-4d5c-8844-1142f7a7d187" containerName="alertmanager"
Jul 27 19:31:32.094323 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.094325    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="dbb93860-c0e8-4d5c-8844-1142f7a7d187" containerName="alertmanager"
Jul 27 19:31:32.094436 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.094333    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f" containerName="console"
Jul 27 19:31:32.094436 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.094339    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f" containerName="console"
Jul 27 19:31:32.094436 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.094350    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6" containerName="registry"
Jul 27 19:31:32.094436 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.094355    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6" containerName="registry"
Jul 27 19:31:32.094436 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.094361    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dbb93860-c0e8-4d5c-8844-1142f7a7d187" containerName="kube-rbac-proxy-web"
Jul 27 19:31:32.094436 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.094366    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="dbb93860-c0e8-4d5c-8844-1142f7a7d187" containerName="kube-rbac-proxy-web"
Jul 27 19:31:32.094436 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.094374    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dbb93860-c0e8-4d5c-8844-1142f7a7d187" containerName="kube-rbac-proxy-metric"
Jul 27 19:31:32.094436 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.094379    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="dbb93860-c0e8-4d5c-8844-1142f7a7d187" containerName="kube-rbac-proxy-metric"
Jul 27 19:31:32.094436 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.094387    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dbb93860-c0e8-4d5c-8844-1142f7a7d187" containerName="config-reloader"
Jul 27 19:31:32.094436 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.094392    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="dbb93860-c0e8-4d5c-8844-1142f7a7d187" containerName="config-reloader"
Jul 27 19:31:32.094436 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.094398    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dbb93860-c0e8-4d5c-8844-1142f7a7d187" containerName="init-config-reloader"
Jul 27 19:31:32.094436 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.094403    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="dbb93860-c0e8-4d5c-8844-1142f7a7d187" containerName="init-config-reloader"
Jul 27 19:31:32.094436 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.094423    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67d6f95c-e959-40a5-8dfa-39228c3b963f" containerName="console"
Jul 27 19:31:32.094436 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.094428    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="67d6f95c-e959-40a5-8dfa-39228c3b963f" containerName="console"
Jul 27 19:31:32.094436 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.094436    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dbb93860-c0e8-4d5c-8844-1142f7a7d187" containerName="kube-rbac-proxy"
Jul 27 19:31:32.094436 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.094441    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="dbb93860-c0e8-4d5c-8844-1142f7a7d187" containerName="kube-rbac-proxy"
Jul 27 19:31:32.094436 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.094447    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dbb93860-c0e8-4d5c-8844-1142f7a7d187" containerName="prom-label-proxy"
Jul 27 19:31:32.094931 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.094452    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="dbb93860-c0e8-4d5c-8844-1142f7a7d187" containerName="prom-label-proxy"
Jul 27 19:31:32.094931 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.094493    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="67d6f95c-e959-40a5-8dfa-39228c3b963f" containerName="console"
Jul 27 19:31:32.094931 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.094499    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d9e9cb6-e041-4bbe-b8d2-670c6ac2ca1f" containerName="console"
Jul 27 19:31:32.094931 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.094505    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="dbb93860-c0e8-4d5c-8844-1142f7a7d187" containerName="prom-label-proxy"
Jul 27 19:31:32.094931 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.094511    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3d3ea53-7b03-4acf-ae92-d4a7d762dbf6" containerName="registry"
Jul 27 19:31:32.094931 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.094516    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="dbb93860-c0e8-4d5c-8844-1142f7a7d187" containerName="kube-rbac-proxy-web"
Jul 27 19:31:32.094931 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.094538    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="dbb93860-c0e8-4d5c-8844-1142f7a7d187" containerName="kube-rbac-proxy"
Jul 27 19:31:32.094931 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.094545    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="dbb93860-c0e8-4d5c-8844-1142f7a7d187" containerName="kube-rbac-proxy-metric"
Jul 27 19:31:32.094931 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.094551    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="dbb93860-c0e8-4d5c-8844-1142f7a7d187" containerName="alertmanager"
Jul 27 19:31:32.094931 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.094556    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="dbb93860-c0e8-4d5c-8844-1142f7a7d187" containerName="config-reloader"
Jul 27 19:31:32.097937 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.097901    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56ff6c45bc-kcmv8"
Jul 27 19:31:32.110286 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.110259    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56ff6c45bc-kcmv8"]
Jul 27 19:31:32.148701 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.148671    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:31:32.213909 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.213807    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7643b91f-b8d8-4e7a-aefb-8fb2531b7029-console-serving-cert\") pod \"console-56ff6c45bc-kcmv8\" (UID: \"7643b91f-b8d8-4e7a-aefb-8fb2531b7029\") " pod="openshift-console/console-56ff6c45bc-kcmv8"
Jul 27 19:31:32.213909 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.213848    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7643b91f-b8d8-4e7a-aefb-8fb2531b7029-console-oauth-config\") pod \"console-56ff6c45bc-kcmv8\" (UID: \"7643b91f-b8d8-4e7a-aefb-8fb2531b7029\") " pod="openshift-console/console-56ff6c45bc-kcmv8"
Jul 27 19:31:32.213909 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.213873    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7643b91f-b8d8-4e7a-aefb-8fb2531b7029-trusted-ca-bundle\") pod \"console-56ff6c45bc-kcmv8\" (UID: \"7643b91f-b8d8-4e7a-aefb-8fb2531b7029\") " pod="openshift-console/console-56ff6c45bc-kcmv8"
Jul 27 19:31:32.213909 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.213900    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2pvn\" (UniqueName: \"kubernetes.io/projected/7643b91f-b8d8-4e7a-aefb-8fb2531b7029-kube-api-access-r2pvn\") pod \"console-56ff6c45bc-kcmv8\" (UID: \"7643b91f-b8d8-4e7a-aefb-8fb2531b7029\") " pod="openshift-console/console-56ff6c45bc-kcmv8"
Jul 27 19:31:32.214196 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.213926    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7643b91f-b8d8-4e7a-aefb-8fb2531b7029-console-config\") pod \"console-56ff6c45bc-kcmv8\" (UID: \"7643b91f-b8d8-4e7a-aefb-8fb2531b7029\") " pod="openshift-console/console-56ff6c45bc-kcmv8"
Jul 27 19:31:32.214196 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.213955    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7643b91f-b8d8-4e7a-aefb-8fb2531b7029-service-ca\") pod \"console-56ff6c45bc-kcmv8\" (UID: \"7643b91f-b8d8-4e7a-aefb-8fb2531b7029\") " pod="openshift-console/console-56ff6c45bc-kcmv8"
Jul 27 19:31:32.214196 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.213989    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7643b91f-b8d8-4e7a-aefb-8fb2531b7029-oauth-serving-cert\") pod \"console-56ff6c45bc-kcmv8\" (UID: \"7643b91f-b8d8-4e7a-aefb-8fb2531b7029\") " pod="openshift-console/console-56ff6c45bc-kcmv8"
Jul 27 19:31:32.315254 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.315214    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7643b91f-b8d8-4e7a-aefb-8fb2531b7029-console-config\") pod \"console-56ff6c45bc-kcmv8\" (UID: \"7643b91f-b8d8-4e7a-aefb-8fb2531b7029\") " pod="openshift-console/console-56ff6c45bc-kcmv8"
Jul 27 19:31:32.315254 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.315257    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7643b91f-b8d8-4e7a-aefb-8fb2531b7029-service-ca\") pod \"console-56ff6c45bc-kcmv8\" (UID: \"7643b91f-b8d8-4e7a-aefb-8fb2531b7029\") " pod="openshift-console/console-56ff6c45bc-kcmv8"
Jul 27 19:31:32.315494 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.315282    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7643b91f-b8d8-4e7a-aefb-8fb2531b7029-oauth-serving-cert\") pod \"console-56ff6c45bc-kcmv8\" (UID: \"7643b91f-b8d8-4e7a-aefb-8fb2531b7029\") " pod="openshift-console/console-56ff6c45bc-kcmv8"
Jul 27 19:31:32.315494 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.315450    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7643b91f-b8d8-4e7a-aefb-8fb2531b7029-console-serving-cert\") pod \"console-56ff6c45bc-kcmv8\" (UID: \"7643b91f-b8d8-4e7a-aefb-8fb2531b7029\") " pod="openshift-console/console-56ff6c45bc-kcmv8"
Jul 27 19:31:32.315494 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.315488    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7643b91f-b8d8-4e7a-aefb-8fb2531b7029-console-oauth-config\") pod \"console-56ff6c45bc-kcmv8\" (UID: \"7643b91f-b8d8-4e7a-aefb-8fb2531b7029\") " pod="openshift-console/console-56ff6c45bc-kcmv8"
Jul 27 19:31:32.315660 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.315510    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7643b91f-b8d8-4e7a-aefb-8fb2531b7029-trusted-ca-bundle\") pod \"console-56ff6c45bc-kcmv8\" (UID: \"7643b91f-b8d8-4e7a-aefb-8fb2531b7029\") " pod="openshift-console/console-56ff6c45bc-kcmv8"
Jul 27 19:31:32.315660 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.315537    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2pvn\" (UniqueName: \"kubernetes.io/projected/7643b91f-b8d8-4e7a-aefb-8fb2531b7029-kube-api-access-r2pvn\") pod \"console-56ff6c45bc-kcmv8\" (UID: \"7643b91f-b8d8-4e7a-aefb-8fb2531b7029\") " pod="openshift-console/console-56ff6c45bc-kcmv8"
Jul 27 19:31:32.316049 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.316026    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7643b91f-b8d8-4e7a-aefb-8fb2531b7029-console-config\") pod \"console-56ff6c45bc-kcmv8\" (UID: \"7643b91f-b8d8-4e7a-aefb-8fb2531b7029\") " pod="openshift-console/console-56ff6c45bc-kcmv8"
Jul 27 19:31:32.316153 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.316101    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7643b91f-b8d8-4e7a-aefb-8fb2531b7029-oauth-serving-cert\") pod \"console-56ff6c45bc-kcmv8\" (UID: \"7643b91f-b8d8-4e7a-aefb-8fb2531b7029\") " pod="openshift-console/console-56ff6c45bc-kcmv8"
Jul 27 19:31:32.316153 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.316135    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7643b91f-b8d8-4e7a-aefb-8fb2531b7029-service-ca\") pod \"console-56ff6c45bc-kcmv8\" (UID: \"7643b91f-b8d8-4e7a-aefb-8fb2531b7029\") " pod="openshift-console/console-56ff6c45bc-kcmv8"
Jul 27 19:31:32.316342 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.316322    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7643b91f-b8d8-4e7a-aefb-8fb2531b7029-trusted-ca-bundle\") pod \"console-56ff6c45bc-kcmv8\" (UID: \"7643b91f-b8d8-4e7a-aefb-8fb2531b7029\") " pod="openshift-console/console-56ff6c45bc-kcmv8"
Jul 27 19:31:32.318075 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.318050    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7643b91f-b8d8-4e7a-aefb-8fb2531b7029-console-serving-cert\") pod \"console-56ff6c45bc-kcmv8\" (UID: \"7643b91f-b8d8-4e7a-aefb-8fb2531b7029\") " pod="openshift-console/console-56ff6c45bc-kcmv8"
Jul 27 19:31:32.318075 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.318069    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7643b91f-b8d8-4e7a-aefb-8fb2531b7029-console-oauth-config\") pod \"console-56ff6c45bc-kcmv8\" (UID: \"7643b91f-b8d8-4e7a-aefb-8fb2531b7029\") " pod="openshift-console/console-56ff6c45bc-kcmv8"
Jul 27 19:31:32.323851 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.323821    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2pvn\" (UniqueName: \"kubernetes.io/projected/7643b91f-b8d8-4e7a-aefb-8fb2531b7029-kube-api-access-r2pvn\") pod \"console-56ff6c45bc-kcmv8\" (UID: \"7643b91f-b8d8-4e7a-aefb-8fb2531b7029\") " pod="openshift-console/console-56ff6c45bc-kcmv8"
Jul 27 19:31:32.407430 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.407390    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56ff6c45bc-kcmv8"
Jul 27 19:31:32.547384 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.547351    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56ff6c45bc-kcmv8"]
Jul 27 19:31:32.550208 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:31:32.550180    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7643b91f_b8d8_4e7a_aefb_8fb2531b7029.slice/crio-4418609dedf616ad6dd0e0ef9889cb55a5388fcb8b325f334fef69667eea2f3c WatchSource:0}: Error finding container 4418609dedf616ad6dd0e0ef9889cb55a5388fcb8b325f334fef69667eea2f3c: Status 404 returned error can't find the container with id 4418609dedf616ad6dd0e0ef9889cb55a5388fcb8b325f334fef69667eea2f3c
Jul 27 19:31:32.756255 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.756159    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:31:32.756780 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.756158    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56ff6c45bc-kcmv8" event={"ID":"7643b91f-b8d8-4e7a-aefb-8fb2531b7029","Type":"ContainerStarted","Data":"1589a28bf9c203cf3d0ebef5e8cf0141feb7a0f4162af7d97580d879578ed93a"}
Jul 27 19:31:32.756780 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.756311    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56ff6c45bc-kcmv8" event={"ID":"7643b91f-b8d8-4e7a-aefb-8fb2531b7029","Type":"ContainerStarted","Data":"4418609dedf616ad6dd0e0ef9889cb55a5388fcb8b325f334fef69667eea2f3c"}
Jul 27 19:31:32.773799 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.773737    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-56ff6c45bc-kcmv8" podStartSLOduration=0.77371633 podStartE2EDuration="773.71633ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:31:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:31:32.773056865 +0000 UTC m=+305.645248866" watchObservedRunningTime="2026-07-27 19:31:32.77371633 +0000 UTC m=+305.645908313"
Jul 27 19:31:32.789980 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.789942    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 27 19:31:32.793691 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.793647    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 27 19:31:32.819899 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.819865    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 27 19:31:32.824068 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.824040    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:31:32.826337 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.826305    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 27 19:31:32.826568 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.826547    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 27 19:31:32.826649 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.826585    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 27 19:31:32.826649 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.826596    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 27 19:31:32.826649 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.826605    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 27 19:31:32.826649 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.826617    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 27 19:31:32.826649 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.826629    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-25z2x\""
Jul 27 19:31:32.826958 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.826762    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 27 19:31:32.827040 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.826994    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 27 19:31:32.836189 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.836000    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 27 19:31:32.840682 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.840655    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 27 19:31:32.919945 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.919909    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a36a8d24-7a5f-4435-b2e3-5f019285639b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a36a8d24-7a5f-4435-b2e3-5f019285639b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:31:32.919945 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.919952    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a36a8d24-7a5f-4435-b2e3-5f019285639b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a36a8d24-7a5f-4435-b2e3-5f019285639b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:31:32.920180 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.919978    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a36a8d24-7a5f-4435-b2e3-5f019285639b-config-volume\") pod \"alertmanager-main-0\" (UID: \"a36a8d24-7a5f-4435-b2e3-5f019285639b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:31:32.920180 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.920054    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a36a8d24-7a5f-4435-b2e3-5f019285639b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a36a8d24-7a5f-4435-b2e3-5f019285639b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:31:32.920180 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.920123    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a36a8d24-7a5f-4435-b2e3-5f019285639b-web-config\") pod \"alertmanager-main-0\" (UID: \"a36a8d24-7a5f-4435-b2e3-5f019285639b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:31:32.920180 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.920153    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a36a8d24-7a5f-4435-b2e3-5f019285639b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a36a8d24-7a5f-4435-b2e3-5f019285639b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:31:32.920180 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.920174    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a36a8d24-7a5f-4435-b2e3-5f019285639b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a36a8d24-7a5f-4435-b2e3-5f019285639b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:31:32.920384 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.920191    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a36a8d24-7a5f-4435-b2e3-5f019285639b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a36a8d24-7a5f-4435-b2e3-5f019285639b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:31:32.920384 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.920230    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjxwq\" (UniqueName: \"kubernetes.io/projected/a36a8d24-7a5f-4435-b2e3-5f019285639b-kube-api-access-kjxwq\") pod \"alertmanager-main-0\" (UID: \"a36a8d24-7a5f-4435-b2e3-5f019285639b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:31:32.920384 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.920246    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a36a8d24-7a5f-4435-b2e3-5f019285639b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a36a8d24-7a5f-4435-b2e3-5f019285639b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:31:32.920384 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.920265    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a36a8d24-7a5f-4435-b2e3-5f019285639b-config-out\") pod \"alertmanager-main-0\" (UID: \"a36a8d24-7a5f-4435-b2e3-5f019285639b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:31:32.920384 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.920285    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a36a8d24-7a5f-4435-b2e3-5f019285639b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a36a8d24-7a5f-4435-b2e3-5f019285639b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:31:32.920384 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:32.920337    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a36a8d24-7a5f-4435-b2e3-5f019285639b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a36a8d24-7a5f-4435-b2e3-5f019285639b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:31:33.021121 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:33.021015    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a36a8d24-7a5f-4435-b2e3-5f019285639b-web-config\") pod \"alertmanager-main-0\" (UID: \"a36a8d24-7a5f-4435-b2e3-5f019285639b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:31:33.021121 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:33.021064    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a36a8d24-7a5f-4435-b2e3-5f019285639b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a36a8d24-7a5f-4435-b2e3-5f019285639b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:31:33.021121 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:33.021092    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a36a8d24-7a5f-4435-b2e3-5f019285639b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a36a8d24-7a5f-4435-b2e3-5f019285639b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:31:33.021384 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:33.021219    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a36a8d24-7a5f-4435-b2e3-5f019285639b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a36a8d24-7a5f-4435-b2e3-5f019285639b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:31:33.021384 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:33.021277    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjxwq\" (UniqueName: \"kubernetes.io/projected/a36a8d24-7a5f-4435-b2e3-5f019285639b-kube-api-access-kjxwq\") pod \"alertmanager-main-0\" (UID: \"a36a8d24-7a5f-4435-b2e3-5f019285639b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:31:33.021384 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:33.021307    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a36a8d24-7a5f-4435-b2e3-5f019285639b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a36a8d24-7a5f-4435-b2e3-5f019285639b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:31:33.021384 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:33.021337    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a36a8d24-7a5f-4435-b2e3-5f019285639b-config-out\") pod \"alertmanager-main-0\" (UID: \"a36a8d24-7a5f-4435-b2e3-5f019285639b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:31:33.021384 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:33.021363    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a36a8d24-7a5f-4435-b2e3-5f019285639b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a36a8d24-7a5f-4435-b2e3-5f019285639b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:31:33.021671 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:33.021390    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a36a8d24-7a5f-4435-b2e3-5f019285639b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a36a8d24-7a5f-4435-b2e3-5f019285639b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:31:33.021671 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:33.021568    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a36a8d24-7a5f-4435-b2e3-5f019285639b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a36a8d24-7a5f-4435-b2e3-5f019285639b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:31:33.021842 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:33.021684    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a36a8d24-7a5f-4435-b2e3-5f019285639b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a36a8d24-7a5f-4435-b2e3-5f019285639b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:31:33.021949 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:33.021884    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a36a8d24-7a5f-4435-b2e3-5f019285639b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a36a8d24-7a5f-4435-b2e3-5f019285639b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:31:33.021949 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:33.021926    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a36a8d24-7a5f-4435-b2e3-5f019285639b-config-volume\") pod \"alertmanager-main-0\" (UID: \"a36a8d24-7a5f-4435-b2e3-5f019285639b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:31:33.022174 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:33.022025    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a36a8d24-7a5f-4435-b2e3-5f019285639b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a36a8d24-7a5f-4435-b2e3-5f019285639b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:31:33.022717 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:33.022650    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a36a8d24-7a5f-4435-b2e3-5f019285639b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a36a8d24-7a5f-4435-b2e3-5f019285639b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:31:33.023025 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:33.022995    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a36a8d24-7a5f-4435-b2e3-5f019285639b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a36a8d24-7a5f-4435-b2e3-5f019285639b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:31:33.024798 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:33.024773    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a36a8d24-7a5f-4435-b2e3-5f019285639b-web-config\") pod \"alertmanager-main-0\" (UID: \"a36a8d24-7a5f-4435-b2e3-5f019285639b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:31:33.025163 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:33.025121    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a36a8d24-7a5f-4435-b2e3-5f019285639b-config-out\") pod \"alertmanager-main-0\" (UID: \"a36a8d24-7a5f-4435-b2e3-5f019285639b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:31:33.025289 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:33.025236    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a36a8d24-7a5f-4435-b2e3-5f019285639b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a36a8d24-7a5f-4435-b2e3-5f019285639b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:31:33.025289 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:33.025270    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a36a8d24-7a5f-4435-b2e3-5f019285639b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a36a8d24-7a5f-4435-b2e3-5f019285639b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:31:33.025439 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:33.025301    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a36a8d24-7a5f-4435-b2e3-5f019285639b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a36a8d24-7a5f-4435-b2e3-5f019285639b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:31:33.025527 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:33.025508    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a36a8d24-7a5f-4435-b2e3-5f019285639b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a36a8d24-7a5f-4435-b2e3-5f019285639b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:31:33.025965 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:33.025944    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a36a8d24-7a5f-4435-b2e3-5f019285639b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a36a8d24-7a5f-4435-b2e3-5f019285639b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:31:33.026028 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:33.025969    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a36a8d24-7a5f-4435-b2e3-5f019285639b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a36a8d24-7a5f-4435-b2e3-5f019285639b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:31:33.026028 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:33.025971    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a36a8d24-7a5f-4435-b2e3-5f019285639b-config-volume\") pod \"alertmanager-main-0\" (UID: \"a36a8d24-7a5f-4435-b2e3-5f019285639b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:31:33.029628 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:33.029606    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjxwq\" (UniqueName: \"kubernetes.io/projected/a36a8d24-7a5f-4435-b2e3-5f019285639b-kube-api-access-kjxwq\") pod \"alertmanager-main-0\" (UID: \"a36a8d24-7a5f-4435-b2e3-5f019285639b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:31:33.139119 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:33.139071    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:31:33.283665 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:33.283555    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 27 19:31:33.286090 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:31:33.286060    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda36a8d24_7a5f_4435_b2e3_5f019285639b.slice/crio-7df77e9869f69de250d3b1c1d0d33fb044ac72ee7690d6244d68418d4c0c474b WatchSource:0}: Error finding container 7df77e9869f69de250d3b1c1d0d33fb044ac72ee7690d6244d68418d4c0c474b: Status 404 returned error can't find the container with id 7df77e9869f69de250d3b1c1d0d33fb044ac72ee7690d6244d68418d4c0c474b
Jul 27 19:31:33.673769 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:33.673731    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dbb93860-c0e8-4d5c-8844-1142f7a7d187" path="/var/lib/kubelet/pods/dbb93860-c0e8-4d5c-8844-1142f7a7d187/volumes"
Jul 27 19:31:33.762066 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:33.762029    2558 generic.go:356] "Generic (PLEG): container finished" podID="a36a8d24-7a5f-4435-b2e3-5f019285639b" containerID="92ed69da67c24793244334d6dbebb1c384742d91b841ee0382fc7c2b8170aed1" exitCode=0
Jul 27 19:31:33.762551 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:33.762120    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a36a8d24-7a5f-4435-b2e3-5f019285639b","Type":"ContainerDied","Data":"92ed69da67c24793244334d6dbebb1c384742d91b841ee0382fc7c2b8170aed1"}
Jul 27 19:31:33.762551 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:33.762188    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a36a8d24-7a5f-4435-b2e3-5f019285639b","Type":"ContainerStarted","Data":"7df77e9869f69de250d3b1c1d0d33fb044ac72ee7690d6244d68418d4c0c474b"}
Jul 27 19:31:34.770719 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:34.770679    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a36a8d24-7a5f-4435-b2e3-5f019285639b","Type":"ContainerStarted","Data":"fef292c039aa4de5da57d22b8b51ffe057d0805bba549e743e0f8a11e4502024"}
Jul 27 19:31:34.771093 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:34.770763    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a36a8d24-7a5f-4435-b2e3-5f019285639b","Type":"ContainerStarted","Data":"a221ab970f9b2cf2b62b0647556cdfbe1d0ad27de18685fca2e71da194b1e40b"}
Jul 27 19:31:34.771093 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:34.770806    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a36a8d24-7a5f-4435-b2e3-5f019285639b","Type":"ContainerStarted","Data":"27ccf074da3e580de38a1454cb1e36e7279657cfb3d651d13796fff277d4d531"}
Jul 27 19:31:34.771093 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:34.770841    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a36a8d24-7a5f-4435-b2e3-5f019285639b","Type":"ContainerStarted","Data":"69de70e611f4b715069cf57dc7439a01f6b1166548ad9facbc3df8114e855391"}
Jul 27 19:31:34.771093 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:34.770875    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a36a8d24-7a5f-4435-b2e3-5f019285639b","Type":"ContainerStarted","Data":"b75f7e94bd5fd4df7728e1aeef922dfbe38d6afbd18dc056bc2386f4a9ecd297"}
Jul 27 19:31:34.771093 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:34.770910    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a36a8d24-7a5f-4435-b2e3-5f019285639b","Type":"ContainerStarted","Data":"efa7a90883a16d52f69fac6d9313ed7757b3a93fd169420b416f7ec522f9e84e"}
Jul 27 19:31:34.796933 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:34.796883    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.7829407760000002 podStartE2EDuration="2.796864718s" totalImagesPullingTime="0s" totalInitContainerRuntime="13.923942ms" isStatefulPod=false podCreationTimestamp="2026-07-27 19:31:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:31:34.795237514 +0000 UTC m=+307.667429569" watchObservedRunningTime="2026-07-27 19:31:34.796864718 +0000 UTC m=+307.669056671"
Jul 27 19:31:36.096977 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:36.096945    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:31:36.177055 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:36.177025    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:31:36.397177 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:36.397147    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:31:38.161614 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:38.161568    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:31:38.218717 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:38.218686    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:31:40.795595 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:40.795555    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:31:42.407984 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:42.407939    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-56ff6c45bc-kcmv8"
Jul 27 19:31:42.408480 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:42.408004    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-56ff6c45bc-kcmv8"
Jul 27 19:31:42.412914 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:42.412886    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-56ff6c45bc-kcmv8"
Jul 27 19:31:42.807877 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:42.807804    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-56ff6c45bc-kcmv8"
Jul 27 19:31:42.851669 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:42.851633    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c49bdcbd7-xc4js"]
Jul 27 19:31:43.271885 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:43.271854    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:31:45.534704 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:45.534656    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/229036e0-7551-4987-98d3-0ccef9c511e5-metrics-certs\") pod \"network-metrics-daemon-zbjtr\" (UID: \"229036e0-7551-4987-98d3-0ccef9c511e5\") " pod="openshift-multus/network-metrics-daemon-zbjtr"
Jul 27 19:31:45.537179 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:45.537152    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/229036e0-7551-4987-98d3-0ccef9c511e5-metrics-certs\") pod \"network-metrics-daemon-zbjtr\" (UID: \"229036e0-7551-4987-98d3-0ccef9c511e5\") " pod="openshift-multus/network-metrics-daemon-zbjtr"
Jul 27 19:31:45.671907 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:45.671879    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-lj4rj\""
Jul 27 19:31:45.679881 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:45.679859    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zbjtr"
Jul 27 19:31:45.810219 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:45.810139    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-zbjtr"]
Jul 27 19:31:45.813592 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:31:45.813562    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod229036e0_7551_4987_98d3_0ccef9c511e5.slice/crio-2bcf18868882ff6148419561ee3a6507f39e66efe351748efb61a04ce6d006f6 WatchSource:0}: Error finding container 2bcf18868882ff6148419561ee3a6507f39e66efe351748efb61a04ce6d006f6: Status 404 returned error can't find the container with id 2bcf18868882ff6148419561ee3a6507f39e66efe351748efb61a04ce6d006f6
Jul 27 19:31:45.818205 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:45.818108    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 19:31:46.821572 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:46.821536    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zbjtr" event={"ID":"229036e0-7551-4987-98d3-0ccef9c511e5","Type":"ContainerStarted","Data":"2bcf18868882ff6148419561ee3a6507f39e66efe351748efb61a04ce6d006f6"}
Jul 27 19:31:47.826572 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:47.826534    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zbjtr" event={"ID":"229036e0-7551-4987-98d3-0ccef9c511e5","Type":"ContainerStarted","Data":"946ed88c9ede68e105cf55b3f5cb2c9f9a1938f021e9d048b3d342436618d96c"}
Jul 27 19:31:47.826929 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:47.826600    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zbjtr" event={"ID":"229036e0-7551-4987-98d3-0ccef9c511e5","Type":"ContainerStarted","Data":"589b3cb99a6c577a40aadfd0518ef3c60f29168d74417e553b9f82674374901e"}
Jul 27 19:31:47.844086 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:47.844030    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-zbjtr" podStartSLOduration=271.900233257 podStartE2EDuration="4m32.844015273s" totalImagesPullingTime="943.782016ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:27:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:31:47.842338217 +0000 UTC m=+320.714530191" watchObservedRunningTime="2026-07-27 19:31:47.844015273 +0000 UTC m=+320.716207247"
Jul 27 19:31:47.945196 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:47.945158    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:31:48.002327 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:48.002268    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:31:56.083612 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:56.083563    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:31:56.169566 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:31:56.169516    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:32:07.871181 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:07.871140    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6c49bdcbd7-xc4js" podUID="fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5" containerName="console" containerID="cri-o://c6250472b46311a98faad58869e0008b0271dec3ce68bb97e9390a12e7571f03" gracePeriod=15
Jul 27 19:32:08.112218 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:08.112192    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c49bdcbd7-xc4js"
Jul 27 19:32:08.210143 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:08.210050    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5-service-ca\") pod \"fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5\" (UID: \"fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5\") "
Jul 27 19:32:08.210143 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:08.210103    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5-console-config\") pod \"fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5\" (UID: \"fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5\") "
Jul 27 19:32:08.210143 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:08.210136    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5-trusted-ca-bundle\") pod \"fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5\" (UID: \"fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5\") "
Jul 27 19:32:08.210473 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:08.210181    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5b5m4\" (UniqueName: \"kubernetes.io/projected/fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5-kube-api-access-5b5m4\") pod \"fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5\" (UID: \"fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5\") "
Jul 27 19:32:08.210473 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:08.210212    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5-console-oauth-config\") pod \"fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5\" (UID: \"fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5\") "
Jul 27 19:32:08.210473 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:08.210269    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5-oauth-serving-cert\") pod \"fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5\" (UID: \"fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5\") "
Jul 27 19:32:08.210473 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:08.210306    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5-console-serving-cert\") pod \"fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5\" (UID: \"fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5\") "
Jul 27 19:32:08.210687 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:08.210521    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5-service-ca" (OuterVolumeSpecName: "service-ca") pod "fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5" (UID: "fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:32:08.210687 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:08.210586    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5" (UID: "fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:32:08.210687 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:08.210595    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5-console-config" (OuterVolumeSpecName: "console-config") pod "fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5" (UID: "fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:32:08.210910 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:08.210890    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5" (UID: "fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:32:08.212591 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:08.212566    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5" (UID: "fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:32:08.212714 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:08.212640    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5" (UID: "fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:32:08.212775 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:08.212707    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5-kube-api-access-5b5m4" (OuterVolumeSpecName: "kube-api-access-5b5m4") pod "fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5" (UID: "fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5"). InnerVolumeSpecName "kube-api-access-5b5m4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:32:08.311778 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:08.311733    2558 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5-trusted-ca-bundle\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:32:08.311778 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:08.311764    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5b5m4\" (UniqueName: \"kubernetes.io/projected/fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5-kube-api-access-5b5m4\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:32:08.311778 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:08.311778    2558 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5-console-oauth-config\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:32:08.311778 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:08.311786    2558 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5-oauth-serving-cert\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:32:08.311778 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:08.311794    2558 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5-console-serving-cert\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:32:08.312121 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:08.311806    2558 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5-service-ca\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:32:08.312121 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:08.311814    2558 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5-console-config\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:32:08.914561 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:08.914527    2558 generic.go:356] "Generic (PLEG): container finished" podID="fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5" containerID="c6250472b46311a98faad58869e0008b0271dec3ce68bb97e9390a12e7571f03" exitCode=0
Jul 27 19:32:08.914952 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:08.914586    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c49bdcbd7-xc4js"
Jul 27 19:32:08.914952 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:08.914588    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c49bdcbd7-xc4js" event={"ID":"fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5","Type":"ContainerDied","Data":"c6250472b46311a98faad58869e0008b0271dec3ce68bb97e9390a12e7571f03"}
Jul 27 19:32:08.914952 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:08.914715    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c49bdcbd7-xc4js" event={"ID":"fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5","Type":"ContainerDied","Data":"36d9b56c1e74898e3b864919dd83396f9b070eafa4c099b9993a6415536b4554"}
Jul 27 19:32:08.914952 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:08.914739    2558 scope.go:117] "RemoveContainer" containerID="c6250472b46311a98faad58869e0008b0271dec3ce68bb97e9390a12e7571f03"
Jul 27 19:32:08.923308 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:08.923293    2558 scope.go:117] "RemoveContainer" containerID="c6250472b46311a98faad58869e0008b0271dec3ce68bb97e9390a12e7571f03"
Jul 27 19:32:08.923595 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:32:08.923572    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6250472b46311a98faad58869e0008b0271dec3ce68bb97e9390a12e7571f03\": container with ID starting with c6250472b46311a98faad58869e0008b0271dec3ce68bb97e9390a12e7571f03 not found: ID does not exist" containerID="c6250472b46311a98faad58869e0008b0271dec3ce68bb97e9390a12e7571f03"
Jul 27 19:32:08.923660 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:08.923602    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6250472b46311a98faad58869e0008b0271dec3ce68bb97e9390a12e7571f03"} err="failed to get container status \"c6250472b46311a98faad58869e0008b0271dec3ce68bb97e9390a12e7571f03\": rpc error: code = NotFound desc = could not find container \"c6250472b46311a98faad58869e0008b0271dec3ce68bb97e9390a12e7571f03\": container with ID starting with c6250472b46311a98faad58869e0008b0271dec3ce68bb97e9390a12e7571f03 not found: ID does not exist"
Jul 27 19:32:08.934440 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:08.934399    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c49bdcbd7-xc4js"]
Jul 27 19:32:08.937385 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:08.937366    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6c49bdcbd7-xc4js"]
Jul 27 19:32:09.673660 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:09.673627    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5" path="/var/lib/kubelet/pods/fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5/volumes"
Jul 27 19:32:10.749574 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:10.749537    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:32:31.910512 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:31.910458    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:32:32.006882 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:32.006848    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:32:32.835922 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:32.835891    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lmqz6"]
Jul 27 19:32:32.836190 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:32.836178    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5" containerName="console"
Jul 27 19:32:32.836233 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:32.836191    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5" containerName="console"
Jul 27 19:32:32.836264 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:32.836239    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe93b772-7ae3-4f3b-ba41-01cd07a5f9f5" containerName="console"
Jul 27 19:32:32.839306 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:32.839287    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lmqz6"
Jul 27 19:32:32.841588 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:32.841568    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-mzjlz\""
Jul 27 19:32:32.841676 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:32.841606    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 27 19:32:32.842401 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:32.842384    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 27 19:32:32.847795 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:32.847767    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lmqz6"]
Jul 27 19:32:32.901232 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:32.901201    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76l5w\" (UniqueName: \"kubernetes.io/projected/c1b8303f-6ba3-48eb-a9da-5e8614a6d11b-kube-api-access-76l5w\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lmqz6\" (UID: \"c1b8303f-6ba3-48eb-a9da-5e8614a6d11b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lmqz6"
Jul 27 19:32:32.901376 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:32.901246    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c1b8303f-6ba3-48eb-a9da-5e8614a6d11b-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lmqz6\" (UID: \"c1b8303f-6ba3-48eb-a9da-5e8614a6d11b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lmqz6"
Jul 27 19:32:32.901376 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:32.901270    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c1b8303f-6ba3-48eb-a9da-5e8614a6d11b-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lmqz6\" (UID: \"c1b8303f-6ba3-48eb-a9da-5e8614a6d11b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lmqz6"
Jul 27 19:32:33.001724 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:33.001692    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c1b8303f-6ba3-48eb-a9da-5e8614a6d11b-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lmqz6\" (UID: \"c1b8303f-6ba3-48eb-a9da-5e8614a6d11b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lmqz6"
Jul 27 19:32:33.001724 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:33.001727    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c1b8303f-6ba3-48eb-a9da-5e8614a6d11b-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lmqz6\" (UID: \"c1b8303f-6ba3-48eb-a9da-5e8614a6d11b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lmqz6"
Jul 27 19:32:33.002187 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:33.001803    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76l5w\" (UniqueName: \"kubernetes.io/projected/c1b8303f-6ba3-48eb-a9da-5e8614a6d11b-kube-api-access-76l5w\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lmqz6\" (UID: \"c1b8303f-6ba3-48eb-a9da-5e8614a6d11b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lmqz6"
Jul 27 19:32:33.002187 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:33.002107    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c1b8303f-6ba3-48eb-a9da-5e8614a6d11b-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lmqz6\" (UID: \"c1b8303f-6ba3-48eb-a9da-5e8614a6d11b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lmqz6"
Jul 27 19:32:33.002187 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:33.002155    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c1b8303f-6ba3-48eb-a9da-5e8614a6d11b-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lmqz6\" (UID: \"c1b8303f-6ba3-48eb-a9da-5e8614a6d11b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lmqz6"
Jul 27 19:32:33.010216 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:33.010194    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-76l5w\" (UniqueName: \"kubernetes.io/projected/c1b8303f-6ba3-48eb-a9da-5e8614a6d11b-kube-api-access-76l5w\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lmqz6\" (UID: \"c1b8303f-6ba3-48eb-a9da-5e8614a6d11b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lmqz6"
Jul 27 19:32:33.148988 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:33.148962    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lmqz6"
Jul 27 19:32:33.278616 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:33.278583    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lmqz6"]
Jul 27 19:32:33.281722 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:32:33.281695    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1b8303f_6ba3_48eb_a9da_5e8614a6d11b.slice/crio-d469f75ea28b96573991489c4f064d3800c3e599005a3334c882e908e293fc5f WatchSource:0}: Error finding container d469f75ea28b96573991489c4f064d3800c3e599005a3334c882e908e293fc5f: Status 404 returned error can't find the container with id d469f75ea28b96573991489c4f064d3800c3e599005a3334c882e908e293fc5f
Jul 27 19:32:34.009812 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:34.009776    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lmqz6" event={"ID":"c1b8303f-6ba3-48eb-a9da-5e8614a6d11b","Type":"ContainerStarted","Data":"d469f75ea28b96573991489c4f064d3800c3e599005a3334c882e908e293fc5f"}
Jul 27 19:32:39.033712 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:39.033680    2558 generic.go:356] "Generic (PLEG): container finished" podID="c1b8303f-6ba3-48eb-a9da-5e8614a6d11b" containerID="6b90ed7290fd577fa950ba53a34be3aad30ebb882c982cc41bed69e8ed5f158c" exitCode=0
Jul 27 19:32:39.034121 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:39.033743    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lmqz6" event={"ID":"c1b8303f-6ba3-48eb-a9da-5e8614a6d11b","Type":"ContainerDied","Data":"6b90ed7290fd577fa950ba53a34be3aad30ebb882c982cc41bed69e8ed5f158c"}
Jul 27 19:32:42.047020 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:42.046936    2558 generic.go:356] "Generic (PLEG): container finished" podID="c1b8303f-6ba3-48eb-a9da-5e8614a6d11b" containerID="c583333925f941459009a6d0b4c100b120e5e1785764348e4aee082262e6017e" exitCode=0
Jul 27 19:32:42.047020 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:42.047002    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lmqz6" event={"ID":"c1b8303f-6ba3-48eb-a9da-5e8614a6d11b","Type":"ContainerDied","Data":"c583333925f941459009a6d0b4c100b120e5e1785764348e4aee082262e6017e"}
Jul 27 19:32:49.080909 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:49.080870    2558 generic.go:356] "Generic (PLEG): container finished" podID="c1b8303f-6ba3-48eb-a9da-5e8614a6d11b" containerID="5a45baa2a4f66e9a4702fefe161cce10ef38330a6922040983b4f6b5536765a5" exitCode=0
Jul 27 19:32:49.081346 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:49.080931    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lmqz6" event={"ID":"c1b8303f-6ba3-48eb-a9da-5e8614a6d11b","Type":"ContainerDied","Data":"5a45baa2a4f66e9a4702fefe161cce10ef38330a6922040983b4f6b5536765a5"}
Jul 27 19:32:50.233470 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:50.233443    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lmqz6"
Jul 27 19:32:50.357617 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:50.357531    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c1b8303f-6ba3-48eb-a9da-5e8614a6d11b-util\") pod \"c1b8303f-6ba3-48eb-a9da-5e8614a6d11b\" (UID: \"c1b8303f-6ba3-48eb-a9da-5e8614a6d11b\") "
Jul 27 19:32:50.357617 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:50.357597    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c1b8303f-6ba3-48eb-a9da-5e8614a6d11b-bundle\") pod \"c1b8303f-6ba3-48eb-a9da-5e8614a6d11b\" (UID: \"c1b8303f-6ba3-48eb-a9da-5e8614a6d11b\") "
Jul 27 19:32:50.357805 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:50.357660    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-76l5w\" (UniqueName: \"kubernetes.io/projected/c1b8303f-6ba3-48eb-a9da-5e8614a6d11b-kube-api-access-76l5w\") pod \"c1b8303f-6ba3-48eb-a9da-5e8614a6d11b\" (UID: \"c1b8303f-6ba3-48eb-a9da-5e8614a6d11b\") "
Jul 27 19:32:50.358155 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:50.358127    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1b8303f-6ba3-48eb-a9da-5e8614a6d11b-bundle" (OuterVolumeSpecName: "bundle") pod "c1b8303f-6ba3-48eb-a9da-5e8614a6d11b" (UID: "c1b8303f-6ba3-48eb-a9da-5e8614a6d11b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:32:50.359952 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:50.359923    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1b8303f-6ba3-48eb-a9da-5e8614a6d11b-kube-api-access-76l5w" (OuterVolumeSpecName: "kube-api-access-76l5w") pod "c1b8303f-6ba3-48eb-a9da-5e8614a6d11b" (UID: "c1b8303f-6ba3-48eb-a9da-5e8614a6d11b"). InnerVolumeSpecName "kube-api-access-76l5w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:32:50.362948 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:50.362922    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1b8303f-6ba3-48eb-a9da-5e8614a6d11b-util" (OuterVolumeSpecName: "util") pod "c1b8303f-6ba3-48eb-a9da-5e8614a6d11b" (UID: "c1b8303f-6ba3-48eb-a9da-5e8614a6d11b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:32:50.458747 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:50.458714    2558 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c1b8303f-6ba3-48eb-a9da-5e8614a6d11b-util\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:32:50.458747 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:50.458743    2558 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c1b8303f-6ba3-48eb-a9da-5e8614a6d11b-bundle\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:32:50.458747 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:50.458753    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-76l5w\" (UniqueName: \"kubernetes.io/projected/c1b8303f-6ba3-48eb-a9da-5e8614a6d11b-kube-api-access-76l5w\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:32:51.090345 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:51.090311    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lmqz6" event={"ID":"c1b8303f-6ba3-48eb-a9da-5e8614a6d11b","Type":"ContainerDied","Data":"d469f75ea28b96573991489c4f064d3800c3e599005a3334c882e908e293fc5f"}
Jul 27 19:32:51.090566 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:51.090356    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lmqz6"
Jul 27 19:32:51.090566 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:51.090362    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d469f75ea28b96573991489c4f064d3800c3e599005a3334c882e908e293fc5f"
Jul 27 19:32:54.014204 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:54.014174    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:32:55.381084 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:55.381052    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:32:55.954450 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:55.954399    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-92lvq"]
Jul 27 19:32:55.954707 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:55.954695    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1b8303f-6ba3-48eb-a9da-5e8614a6d11b" containerName="pull"
Jul 27 19:32:55.954754 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:55.954709    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1b8303f-6ba3-48eb-a9da-5e8614a6d11b" containerName="pull"
Jul 27 19:32:55.954754 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:55.954721    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1b8303f-6ba3-48eb-a9da-5e8614a6d11b" containerName="extract"
Jul 27 19:32:55.954754 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:55.954727    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1b8303f-6ba3-48eb-a9da-5e8614a6d11b" containerName="extract"
Jul 27 19:32:55.954754 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:55.954734    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1b8303f-6ba3-48eb-a9da-5e8614a6d11b" containerName="util"
Jul 27 19:32:55.954754 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:55.954739    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1b8303f-6ba3-48eb-a9da-5e8614a6d11b" containerName="util"
Jul 27 19:32:55.954900 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:55.954795    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1b8303f-6ba3-48eb-a9da-5e8614a6d11b" containerName="extract"
Jul 27 19:32:55.958329 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:55.958315    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-92lvq"
Jul 27 19:32:55.960455 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:55.960429    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 27 19:32:55.960589 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:55.960477    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-j29p2\""
Jul 27 19:32:55.960655 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:55.960604    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 27 19:32:55.969360 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:55.969340    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-92lvq"]
Jul 27 19:32:56.106813 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:56.106780    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8f4j9\" (UniqueName: \"kubernetes.io/projected/0aaae5f0-6f98-468a-8d0a-3701ec7b20e0-kube-api-access-8f4j9\") pod \"cert-manager-operator-controller-manager-55c444bf86-92lvq\" (UID: \"0aaae5f0-6f98-468a-8d0a-3701ec7b20e0\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-92lvq"
Jul 27 19:32:56.106988 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:56.106839    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0aaae5f0-6f98-468a-8d0a-3701ec7b20e0-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-92lvq\" (UID: \"0aaae5f0-6f98-468a-8d0a-3701ec7b20e0\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-92lvq"
Jul 27 19:32:56.207784 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:56.207694    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8f4j9\" (UniqueName: \"kubernetes.io/projected/0aaae5f0-6f98-468a-8d0a-3701ec7b20e0-kube-api-access-8f4j9\") pod \"cert-manager-operator-controller-manager-55c444bf86-92lvq\" (UID: \"0aaae5f0-6f98-468a-8d0a-3701ec7b20e0\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-92lvq"
Jul 27 19:32:56.207784 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:56.207741    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0aaae5f0-6f98-468a-8d0a-3701ec7b20e0-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-92lvq\" (UID: \"0aaae5f0-6f98-468a-8d0a-3701ec7b20e0\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-92lvq"
Jul 27 19:32:56.208095 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:56.208078    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0aaae5f0-6f98-468a-8d0a-3701ec7b20e0-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-92lvq\" (UID: \"0aaae5f0-6f98-468a-8d0a-3701ec7b20e0\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-92lvq"
Jul 27 19:32:56.215910 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:56.215887    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8f4j9\" (UniqueName: \"kubernetes.io/projected/0aaae5f0-6f98-468a-8d0a-3701ec7b20e0-kube-api-access-8f4j9\") pod \"cert-manager-operator-controller-manager-55c444bf86-92lvq\" (UID: \"0aaae5f0-6f98-468a-8d0a-3701ec7b20e0\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-92lvq"
Jul 27 19:32:56.267461 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:56.267434    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-92lvq"
Jul 27 19:32:56.401437 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:56.401240    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-92lvq"]
Jul 27 19:32:56.404588 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:32:56.404563    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0aaae5f0_6f98_468a_8d0a_3701ec7b20e0.slice/crio-d24d4a8f37fad68406a424d2c6c50fb396896fe15952de6d621157b9b293a28a WatchSource:0}: Error finding container d24d4a8f37fad68406a424d2c6c50fb396896fe15952de6d621157b9b293a28a: Status 404 returned error can't find the container with id d24d4a8f37fad68406a424d2c6c50fb396896fe15952de6d621157b9b293a28a
Jul 27 19:32:57.114292 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:57.114253    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-92lvq" event={"ID":"0aaae5f0-6f98-468a-8d0a-3701ec7b20e0","Type":"ContainerStarted","Data":"d24d4a8f37fad68406a424d2c6c50fb396896fe15952de6d621157b9b293a28a"}
Jul 27 19:32:57.630647 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:57.630598    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:32:57.728421 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:57.728369    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:32:59.080679 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:59.080636    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:32:59.124569 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:59.124537    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-92lvq" event={"ID":"0aaae5f0-6f98-468a-8d0a-3701ec7b20e0","Type":"ContainerStarted","Data":"1e596f95976ee365f7116575de03d1933c787ebbe20dc8d324d94d78a3998ed3"}
Jul 27 19:32:59.147965 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:59.147913    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-92lvq" podStartSLOduration=2.184182851 podStartE2EDuration="4.147893799s" totalImagesPullingTime="1.963710948s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:32:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:32:59.145582628 +0000 UTC m=+392.017774601" watchObservedRunningTime="2026-07-27 19:32:59.147893799 +0000 UTC m=+392.020085767"
Jul 27 19:32:59.153536 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:59.153505    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:32:59.208788 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:59.208756    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-n7lqz"]
Jul 27 19:32:59.211122 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:59.211105    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-n7lqz"
Jul 27 19:32:59.213371 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:59.213340    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 27 19:32:59.213472 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:59.213399    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 27 19:32:59.221561 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:59.221542    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-n7lqz"]
Jul 27 19:32:59.230458 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:59.230405    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bcp2v\" (UniqueName: \"kubernetes.io/projected/c1596808-eac7-4c32-9484-5673b18e7bc7-kube-api-access-bcp2v\") pod \"cert-manager-webhook-6486ffd997-n7lqz\" (UID: \"c1596808-eac7-4c32-9484-5673b18e7bc7\") " pod="cert-manager/cert-manager-webhook-6486ffd997-n7lqz"
Jul 27 19:32:59.230579 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:59.230484    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c1596808-eac7-4c32-9484-5673b18e7bc7-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-n7lqz\" (UID: \"c1596808-eac7-4c32-9484-5673b18e7bc7\") " pod="cert-manager/cert-manager-webhook-6486ffd997-n7lqz"
Jul 27 19:32:59.330958 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:59.330874    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcp2v\" (UniqueName: \"kubernetes.io/projected/c1596808-eac7-4c32-9484-5673b18e7bc7-kube-api-access-bcp2v\") pod \"cert-manager-webhook-6486ffd997-n7lqz\" (UID: \"c1596808-eac7-4c32-9484-5673b18e7bc7\") " pod="cert-manager/cert-manager-webhook-6486ffd997-n7lqz"
Jul 27 19:32:59.330958 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:59.330921    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c1596808-eac7-4c32-9484-5673b18e7bc7-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-n7lqz\" (UID: \"c1596808-eac7-4c32-9484-5673b18e7bc7\") " pod="cert-manager/cert-manager-webhook-6486ffd997-n7lqz"
Jul 27 19:32:59.338939 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:59.338914    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bcp2v\" (UniqueName: \"kubernetes.io/projected/c1596808-eac7-4c32-9484-5673b18e7bc7-kube-api-access-bcp2v\") pod \"cert-manager-webhook-6486ffd997-n7lqz\" (UID: \"c1596808-eac7-4c32-9484-5673b18e7bc7\") " pod="cert-manager/cert-manager-webhook-6486ffd997-n7lqz"
Jul 27 19:32:59.339096 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:59.339075    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c1596808-eac7-4c32-9484-5673b18e7bc7-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-n7lqz\" (UID: \"c1596808-eac7-4c32-9484-5673b18e7bc7\") " pod="cert-manager/cert-manager-webhook-6486ffd997-n7lqz"
Jul 27 19:32:59.510950 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:59.510909    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:32:59.527630 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:59.527601    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-n7lqz"
Jul 27 19:32:59.612986 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:59.612952    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:32:59.697621 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:32:59.694988    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-n7lqz"]
Jul 27 19:32:59.697839 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:32:59.697708    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc1596808_eac7_4c32_9484_5673b18e7bc7.slice/crio-714746397d27a58b71518a26d5dfe227181fcc6e23100aae49cbf9cdac90dace WatchSource:0}: Error finding container 714746397d27a58b71518a26d5dfe227181fcc6e23100aae49cbf9cdac90dace: Status 404 returned error can't find the container with id 714746397d27a58b71518a26d5dfe227181fcc6e23100aae49cbf9cdac90dace
Jul 27 19:33:00.130545 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:00.130505    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-n7lqz" event={"ID":"c1596808-eac7-4c32-9484-5673b18e7bc7","Type":"ContainerStarted","Data":"714746397d27a58b71518a26d5dfe227181fcc6e23100aae49cbf9cdac90dace"}
Jul 27 19:33:00.641278 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:00.641250    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fr6x57"]
Jul 27 19:33:00.644480 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:00.644463    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fr6x57"
Jul 27 19:33:00.646831 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:00.646799    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 27 19:33:00.646925 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:00.646846    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-mzjlz\""
Jul 27 19:33:00.646964 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:00.646856    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 27 19:33:00.655991 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:00.655963    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fr6x57"]
Jul 27 19:33:00.742110 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:00.742060    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f11dd0d5-4d35-4e77-b97c-2d4d6e776549-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fr6x57\" (UID: \"f11dd0d5-4d35-4e77-b97c-2d4d6e776549\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fr6x57"
Jul 27 19:33:00.742301 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:00.742117    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlkpm\" (UniqueName: \"kubernetes.io/projected/f11dd0d5-4d35-4e77-b97c-2d4d6e776549-kube-api-access-xlkpm\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fr6x57\" (UID: \"f11dd0d5-4d35-4e77-b97c-2d4d6e776549\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fr6x57"
Jul 27 19:33:00.742301 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:00.742220    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f11dd0d5-4d35-4e77-b97c-2d4d6e776549-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fr6x57\" (UID: \"f11dd0d5-4d35-4e77-b97c-2d4d6e776549\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fr6x57"
Jul 27 19:33:00.843676 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:00.843634    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlkpm\" (UniqueName: \"kubernetes.io/projected/f11dd0d5-4d35-4e77-b97c-2d4d6e776549-kube-api-access-xlkpm\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fr6x57\" (UID: \"f11dd0d5-4d35-4e77-b97c-2d4d6e776549\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fr6x57"
Jul 27 19:33:00.843870 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:00.843702    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f11dd0d5-4d35-4e77-b97c-2d4d6e776549-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fr6x57\" (UID: \"f11dd0d5-4d35-4e77-b97c-2d4d6e776549\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fr6x57"
Jul 27 19:33:00.843870 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:00.843826    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f11dd0d5-4d35-4e77-b97c-2d4d6e776549-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fr6x57\" (UID: \"f11dd0d5-4d35-4e77-b97c-2d4d6e776549\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fr6x57"
Jul 27 19:33:00.844230 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:00.844197    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f11dd0d5-4d35-4e77-b97c-2d4d6e776549-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fr6x57\" (UID: \"f11dd0d5-4d35-4e77-b97c-2d4d6e776549\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fr6x57"
Jul 27 19:33:00.844362 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:00.844210    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f11dd0d5-4d35-4e77-b97c-2d4d6e776549-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fr6x57\" (UID: \"f11dd0d5-4d35-4e77-b97c-2d4d6e776549\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fr6x57"
Jul 27 19:33:00.852144 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:00.852118    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlkpm\" (UniqueName: \"kubernetes.io/projected/f11dd0d5-4d35-4e77-b97c-2d4d6e776549-kube-api-access-xlkpm\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fr6x57\" (UID: \"f11dd0d5-4d35-4e77-b97c-2d4d6e776549\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fr6x57"
Jul 27 19:33:00.953683 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:00.953548    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fr6x57"
Jul 27 19:33:01.103740 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:01.103713    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fr6x57"]
Jul 27 19:33:01.105843 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:33:01.105815    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf11dd0d5_4d35_4e77_b97c_2d4d6e776549.slice/crio-a8395ba8304b618e31e510dc12fc602ba6cd473cf4d83b487614134b0e7e0b65 WatchSource:0}: Error finding container a8395ba8304b618e31e510dc12fc602ba6cd473cf4d83b487614134b0e7e0b65: Status 404 returned error can't find the container with id a8395ba8304b618e31e510dc12fc602ba6cd473cf4d83b487614134b0e7e0b65
Jul 27 19:33:01.136394 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:01.136351    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fr6x57" event={"ID":"f11dd0d5-4d35-4e77-b97c-2d4d6e776549","Type":"ContainerStarted","Data":"a8395ba8304b618e31e510dc12fc602ba6cd473cf4d83b487614134b0e7e0b65"}
Jul 27 19:33:01.347532 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:01.347498    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:01.493603 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:01.493524    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:02.143962 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:02.143918    2558 generic.go:356] "Generic (PLEG): container finished" podID="f11dd0d5-4d35-4e77-b97c-2d4d6e776549" containerID="e23d53c625a447967ea3d52b81618e8810a90c486efca82a2456039d456cab8a" exitCode=0
Jul 27 19:33:02.144391 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:02.143980    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fr6x57" event={"ID":"f11dd0d5-4d35-4e77-b97c-2d4d6e776549","Type":"ContainerDied","Data":"e23d53c625a447967ea3d52b81618e8810a90c486efca82a2456039d456cab8a"}
Jul 27 19:33:03.150914 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:03.150883    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-n7lqz" event={"ID":"c1596808-eac7-4c32-9484-5673b18e7bc7","Type":"ContainerStarted","Data":"8d5ebba6f6f76c4b9c9db17065a55608c8bb25b9497f756db4b5bc95da8ac384"}
Jul 27 19:33:03.151336 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:03.150979    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-n7lqz"
Jul 27 19:33:03.169193 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:03.169147    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-n7lqz" podStartSLOduration=1.233083164 podStartE2EDuration="4.169132048s" totalImagesPullingTime="2.936048884s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:32:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:33:03.166637246 +0000 UTC m=+396.038829221" watchObservedRunningTime="2026-07-27 19:33:03.169132048 +0000 UTC m=+396.041324021"
Jul 27 19:33:04.164039 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:04.164008    2558 generic.go:356] "Generic (PLEG): container finished" podID="f11dd0d5-4d35-4e77-b97c-2d4d6e776549" containerID="3bcd9b1109a79a8ce6fca836d5d0987a05fd30930d9911decf6ef1fd2dcdbdc2" exitCode=0
Jul 27 19:33:04.164453 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:04.164054    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fr6x57" event={"ID":"f11dd0d5-4d35-4e77-b97c-2d4d6e776549","Type":"ContainerDied","Data":"3bcd9b1109a79a8ce6fca836d5d0987a05fd30930d9911decf6ef1fd2dcdbdc2"}
Jul 27 19:33:04.422679 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:04.422654    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:04.487927 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:04.487894    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:05.170024 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:05.169991    2558 generic.go:356] "Generic (PLEG): container finished" podID="f11dd0d5-4d35-4e77-b97c-2d4d6e776549" containerID="1772a2e17306bda7df21c69ef15b7e66bd79f89866a06c4ff6ce5890db2cd1ca" exitCode=0
Jul 27 19:33:05.170438 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:05.170035    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fr6x57" event={"ID":"f11dd0d5-4d35-4e77-b97c-2d4d6e776549","Type":"ContainerDied","Data":"1772a2e17306bda7df21c69ef15b7e66bd79f89866a06c4ff6ce5890db2cd1ca"}
Jul 27 19:33:06.296051 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:06.296029    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fr6x57"
Jul 27 19:33:06.393151 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:06.393118    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f11dd0d5-4d35-4e77-b97c-2d4d6e776549-bundle\") pod \"f11dd0d5-4d35-4e77-b97c-2d4d6e776549\" (UID: \"f11dd0d5-4d35-4e77-b97c-2d4d6e776549\") "
Jul 27 19:33:06.393326 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:06.393168    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xlkpm\" (UniqueName: \"kubernetes.io/projected/f11dd0d5-4d35-4e77-b97c-2d4d6e776549-kube-api-access-xlkpm\") pod \"f11dd0d5-4d35-4e77-b97c-2d4d6e776549\" (UID: \"f11dd0d5-4d35-4e77-b97c-2d4d6e776549\") "
Jul 27 19:33:06.393326 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:06.393195    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f11dd0d5-4d35-4e77-b97c-2d4d6e776549-util\") pod \"f11dd0d5-4d35-4e77-b97c-2d4d6e776549\" (UID: \"f11dd0d5-4d35-4e77-b97c-2d4d6e776549\") "
Jul 27 19:33:06.393578 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:06.393554    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f11dd0d5-4d35-4e77-b97c-2d4d6e776549-bundle" (OuterVolumeSpecName: "bundle") pod "f11dd0d5-4d35-4e77-b97c-2d4d6e776549" (UID: "f11dd0d5-4d35-4e77-b97c-2d4d6e776549"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:33:06.395309 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:06.395288    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f11dd0d5-4d35-4e77-b97c-2d4d6e776549-kube-api-access-xlkpm" (OuterVolumeSpecName: "kube-api-access-xlkpm") pod "f11dd0d5-4d35-4e77-b97c-2d4d6e776549" (UID: "f11dd0d5-4d35-4e77-b97c-2d4d6e776549"). InnerVolumeSpecName "kube-api-access-xlkpm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:33:06.398373 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:06.398350    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f11dd0d5-4d35-4e77-b97c-2d4d6e776549-util" (OuterVolumeSpecName: "util") pod "f11dd0d5-4d35-4e77-b97c-2d4d6e776549" (UID: "f11dd0d5-4d35-4e77-b97c-2d4d6e776549"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:33:06.494034 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:06.493974    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xlkpm\" (UniqueName: \"kubernetes.io/projected/f11dd0d5-4d35-4e77-b97c-2d4d6e776549-kube-api-access-xlkpm\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:33:06.494034 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:06.493994    2558 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f11dd0d5-4d35-4e77-b97c-2d4d6e776549-util\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:33:06.494034 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:06.494003    2558 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f11dd0d5-4d35-4e77-b97c-2d4d6e776549-bundle\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:33:07.180361 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:07.180331    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fr6x57"
Jul 27 19:33:07.180522 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:07.180336    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fr6x57" event={"ID":"f11dd0d5-4d35-4e77-b97c-2d4d6e776549","Type":"ContainerDied","Data":"a8395ba8304b618e31e510dc12fc602ba6cd473cf4d83b487614134b0e7e0b65"}
Jul 27 19:33:07.180522 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:07.180457    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a8395ba8304b618e31e510dc12fc602ba6cd473cf4d83b487614134b0e7e0b65"
Jul 27 19:33:07.275663 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:07.275635    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-q48sf"]
Jul 27 19:33:07.275924 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:07.275911    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f11dd0d5-4d35-4e77-b97c-2d4d6e776549" containerName="extract"
Jul 27 19:33:07.275924 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:07.275925    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="f11dd0d5-4d35-4e77-b97c-2d4d6e776549" containerName="extract"
Jul 27 19:33:07.276013 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:07.275940    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f11dd0d5-4d35-4e77-b97c-2d4d6e776549" containerName="pull"
Jul 27 19:33:07.276013 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:07.275947    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="f11dd0d5-4d35-4e77-b97c-2d4d6e776549" containerName="pull"
Jul 27 19:33:07.276013 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:07.275957    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f11dd0d5-4d35-4e77-b97c-2d4d6e776549" containerName="util"
Jul 27 19:33:07.276013 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:07.275963    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="f11dd0d5-4d35-4e77-b97c-2d4d6e776549" containerName="util"
Jul 27 19:33:07.276013 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:07.276007    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="f11dd0d5-4d35-4e77-b97c-2d4d6e776549" containerName="extract"
Jul 27 19:33:07.278158 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:07.278140    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-q48sf"
Jul 27 19:33:07.280402 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:07.280386    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-xz5nj\""
Jul 27 19:33:07.291112 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:07.291092    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-q48sf"]
Jul 27 19:33:07.299702 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:07.299679    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9t2gn\" (UniqueName: \"kubernetes.io/projected/b6954bd3-34d3-4cbc-8327-84cd361d82e0-kube-api-access-9t2gn\") pod \"cert-manager-cainjector-559cdd8c87-q48sf\" (UID: \"b6954bd3-34d3-4cbc-8327-84cd361d82e0\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-q48sf"
Jul 27 19:33:07.300128 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:07.299723    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b6954bd3-34d3-4cbc-8327-84cd361d82e0-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-q48sf\" (UID: \"b6954bd3-34d3-4cbc-8327-84cd361d82e0\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-q48sf"
Jul 27 19:33:07.400222 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:07.400174    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9t2gn\" (UniqueName: \"kubernetes.io/projected/b6954bd3-34d3-4cbc-8327-84cd361d82e0-kube-api-access-9t2gn\") pod \"cert-manager-cainjector-559cdd8c87-q48sf\" (UID: \"b6954bd3-34d3-4cbc-8327-84cd361d82e0\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-q48sf"
Jul 27 19:33:07.400222 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:07.400229    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b6954bd3-34d3-4cbc-8327-84cd361d82e0-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-q48sf\" (UID: \"b6954bd3-34d3-4cbc-8327-84cd361d82e0\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-q48sf"
Jul 27 19:33:07.408701 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:07.408665    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b6954bd3-34d3-4cbc-8327-84cd361d82e0-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-q48sf\" (UID: \"b6954bd3-34d3-4cbc-8327-84cd361d82e0\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-q48sf"
Jul 27 19:33:07.408953 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:07.408932    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9t2gn\" (UniqueName: \"kubernetes.io/projected/b6954bd3-34d3-4cbc-8327-84cd361d82e0-kube-api-access-9t2gn\") pod \"cert-manager-cainjector-559cdd8c87-q48sf\" (UID: \"b6954bd3-34d3-4cbc-8327-84cd361d82e0\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-q48sf"
Jul 27 19:33:07.587130 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:07.587047    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-q48sf"
Jul 27 19:33:07.718705 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:07.718677    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-q48sf"]
Jul 27 19:33:07.721802 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:33:07.721775    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb6954bd3_34d3_4cbc_8327_84cd361d82e0.slice/crio-674bc5cba0780b4b85dd9448b691b13cf634863e4f3d851ba8a6b85862137eae WatchSource:0}: Error finding container 674bc5cba0780b4b85dd9448b691b13cf634863e4f3d851ba8a6b85862137eae: Status 404 returned error can't find the container with id 674bc5cba0780b4b85dd9448b691b13cf634863e4f3d851ba8a6b85862137eae
Jul 27 19:33:08.186197 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:08.186156    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-q48sf" event={"ID":"b6954bd3-34d3-4cbc-8327-84cd361d82e0","Type":"ContainerStarted","Data":"2f74ee74f72ef65a8854fa8901b2a2fbd01d6dc263d604adb90ebce969b668f9"}
Jul 27 19:33:08.186392 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:08.186207    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-q48sf" event={"ID":"b6954bd3-34d3-4cbc-8327-84cd361d82e0","Type":"ContainerStarted","Data":"674bc5cba0780b4b85dd9448b691b13cf634863e4f3d851ba8a6b85862137eae"}
Jul 27 19:33:08.203903 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:08.203845    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-q48sf" podStartSLOduration=1.203831793 podStartE2EDuration="1.203831793s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:33:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:33:08.203126467 +0000 UTC m=+401.075318442" watchObservedRunningTime="2026-07-27 19:33:08.203831793 +0000 UTC m=+401.076023767"
Jul 27 19:33:08.512517 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:08.512440    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:08.574337 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:08.574303    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:09.166660 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:09.166626    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-n7lqz"
Jul 27 19:33:09.700565 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:09.700527    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:10.355328 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:10.355293    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-2skrw"]
Jul 27 19:33:10.357627 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:10.357609    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-2skrw"
Jul 27 19:33:10.360125 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:10.360105    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-kr5zc\""
Jul 27 19:33:10.368572 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:10.368551    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-2skrw"]
Jul 27 19:33:10.425205 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:10.425176    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dc85ec80-50a4-412d-a28f-d5822464229a-bound-sa-token\") pod \"cert-manager-55b4d954d5-2skrw\" (UID: \"dc85ec80-50a4-412d-a28f-d5822464229a\") " pod="cert-manager/cert-manager-55b4d954d5-2skrw"
Jul 27 19:33:10.425330 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:10.425221    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9qrq\" (UniqueName: \"kubernetes.io/projected/dc85ec80-50a4-412d-a28f-d5822464229a-kube-api-access-b9qrq\") pod \"cert-manager-55b4d954d5-2skrw\" (UID: \"dc85ec80-50a4-412d-a28f-d5822464229a\") " pod="cert-manager/cert-manager-55b4d954d5-2skrw"
Jul 27 19:33:10.525785 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:10.525752    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dc85ec80-50a4-412d-a28f-d5822464229a-bound-sa-token\") pod \"cert-manager-55b4d954d5-2skrw\" (UID: \"dc85ec80-50a4-412d-a28f-d5822464229a\") " pod="cert-manager/cert-manager-55b4d954d5-2skrw"
Jul 27 19:33:10.525948 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:10.525810    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9qrq\" (UniqueName: \"kubernetes.io/projected/dc85ec80-50a4-412d-a28f-d5822464229a-kube-api-access-b9qrq\") pod \"cert-manager-55b4d954d5-2skrw\" (UID: \"dc85ec80-50a4-412d-a28f-d5822464229a\") " pod="cert-manager/cert-manager-55b4d954d5-2skrw"
Jul 27 19:33:10.533962 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:10.533933    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dc85ec80-50a4-412d-a28f-d5822464229a-bound-sa-token\") pod \"cert-manager-55b4d954d5-2skrw\" (UID: \"dc85ec80-50a4-412d-a28f-d5822464229a\") " pod="cert-manager/cert-manager-55b4d954d5-2skrw"
Jul 27 19:33:10.534090 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:10.534074    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9qrq\" (UniqueName: \"kubernetes.io/projected/dc85ec80-50a4-412d-a28f-d5822464229a-kube-api-access-b9qrq\") pod \"cert-manager-55b4d954d5-2skrw\" (UID: \"dc85ec80-50a4-412d-a28f-d5822464229a\") " pod="cert-manager/cert-manager-55b4d954d5-2skrw"
Jul 27 19:33:10.667056 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:10.667025    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-2skrw"
Jul 27 19:33:10.794260 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:10.794206    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-2skrw"]
Jul 27 19:33:10.796886 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:33:10.796863    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddc85ec80_50a4_412d_a28f_d5822464229a.slice/crio-24058bf2bc31f6295a230e074249db761dfce6e084a565bb9f139fb82a4926d0 WatchSource:0}: Error finding container 24058bf2bc31f6295a230e074249db761dfce6e084a565bb9f139fb82a4926d0: Status 404 returned error can't find the container with id 24058bf2bc31f6295a230e074249db761dfce6e084a565bb9f139fb82a4926d0
Jul 27 19:33:11.201353 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:11.201317    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-2skrw" event={"ID":"dc85ec80-50a4-412d-a28f-d5822464229a","Type":"ContainerStarted","Data":"3508def36663b47cc45142c23bd5478a9870d68b5a842c4daedec3d37844e963"}
Jul 27 19:33:11.201353 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:11.201357    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-2skrw" event={"ID":"dc85ec80-50a4-412d-a28f-d5822464229a","Type":"ContainerStarted","Data":"24058bf2bc31f6295a230e074249db761dfce6e084a565bb9f139fb82a4926d0"}
Jul 27 19:33:11.219100 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:11.219005    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-2skrw" podStartSLOduration=1.218989555 podStartE2EDuration="1.218989555s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:33:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:33:11.217372204 +0000 UTC m=+404.089564179" watchObservedRunningTime="2026-07-27 19:33:11.218989555 +0000 UTC m=+404.091181529"
Jul 27 19:33:11.898895 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:11.898842    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:12.030803 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:12.030760    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:17.488941 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:17.488901    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:17.592266 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:17.592235    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:19.529601 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:19.529565    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afktfkf"]
Jul 27 19:33:19.551898 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:19.551873    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afktfkf"]
Jul 27 19:33:19.552027 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:19.551966    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afktfkf"
Jul 27 19:33:19.554636 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:19.554612    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-mzjlz\""
Jul 27 19:33:19.554752 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:19.554652    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 27 19:33:19.555405 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:19.555390    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 27 19:33:19.605513 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:19.605489    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4133fc41-332a-4355-b0e9-5cc0541c85e5-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afktfkf\" (UID: \"4133fc41-332a-4355-b0e9-5cc0541c85e5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afktfkf"
Jul 27 19:33:19.605614 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:19.605572    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4133fc41-332a-4355-b0e9-5cc0541c85e5-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afktfkf\" (UID: \"4133fc41-332a-4355-b0e9-5cc0541c85e5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afktfkf"
Jul 27 19:33:19.605614 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:19.605603    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhphm\" (UniqueName: \"kubernetes.io/projected/4133fc41-332a-4355-b0e9-5cc0541c85e5-kube-api-access-nhphm\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afktfkf\" (UID: \"4133fc41-332a-4355-b0e9-5cc0541c85e5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afktfkf"
Jul 27 19:33:19.664370 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:19.664341    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:19.706180 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:19.706151    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4133fc41-332a-4355-b0e9-5cc0541c85e5-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afktfkf\" (UID: \"4133fc41-332a-4355-b0e9-5cc0541c85e5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afktfkf"
Jul 27 19:33:19.706344 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:19.706195    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhphm\" (UniqueName: \"kubernetes.io/projected/4133fc41-332a-4355-b0e9-5cc0541c85e5-kube-api-access-nhphm\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afktfkf\" (UID: \"4133fc41-332a-4355-b0e9-5cc0541c85e5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afktfkf"
Jul 27 19:33:19.706344 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:19.706243    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4133fc41-332a-4355-b0e9-5cc0541c85e5-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afktfkf\" (UID: \"4133fc41-332a-4355-b0e9-5cc0541c85e5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afktfkf"
Jul 27 19:33:19.706622 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:19.706599    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4133fc41-332a-4355-b0e9-5cc0541c85e5-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afktfkf\" (UID: \"4133fc41-332a-4355-b0e9-5cc0541c85e5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afktfkf"
Jul 27 19:33:19.706693 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:19.706622    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4133fc41-332a-4355-b0e9-5cc0541c85e5-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afktfkf\" (UID: \"4133fc41-332a-4355-b0e9-5cc0541c85e5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afktfkf"
Jul 27 19:33:19.714681 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:19.714659    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhphm\" (UniqueName: \"kubernetes.io/projected/4133fc41-332a-4355-b0e9-5cc0541c85e5-kube-api-access-nhphm\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afktfkf\" (UID: \"4133fc41-332a-4355-b0e9-5cc0541c85e5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afktfkf"
Jul 27 19:33:19.860957 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:19.860888    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afktfkf"
Jul 27 19:33:20.009961 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:20.009937    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afktfkf"]
Jul 27 19:33:20.012150 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:33:20.012122    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4133fc41_332a_4355_b0e9_5cc0541c85e5.slice/crio-824f6280574e9e2e9413468cf6223660f22459d16e67757905b258af5f281795 WatchSource:0}: Error finding container 824f6280574e9e2e9413468cf6223660f22459d16e67757905b258af5f281795: Status 404 returned error can't find the container with id 824f6280574e9e2e9413468cf6223660f22459d16e67757905b258af5f281795
Jul 27 19:33:20.245456 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:20.245398    2558 generic.go:356] "Generic (PLEG): container finished" podID="4133fc41-332a-4355-b0e9-5cc0541c85e5" containerID="5aa4655b273283fd972dbe1da51f6e631acf11a51703d044b6125a4c259af0c4" exitCode=0
Jul 27 19:33:20.245619 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:20.245465    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afktfkf" event={"ID":"4133fc41-332a-4355-b0e9-5cc0541c85e5","Type":"ContainerDied","Data":"5aa4655b273283fd972dbe1da51f6e631acf11a51703d044b6125a4c259af0c4"}
Jul 27 19:33:20.245619 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:20.245500    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afktfkf" event={"ID":"4133fc41-332a-4355-b0e9-5cc0541c85e5","Type":"ContainerStarted","Data":"824f6280574e9e2e9413468cf6223660f22459d16e67757905b258af5f281795"}
Jul 27 19:33:22.258675 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:22.258634    2558 generic.go:356] "Generic (PLEG): container finished" podID="4133fc41-332a-4355-b0e9-5cc0541c85e5" containerID="2982e45e8f7cda2495fa3ddbb0271f3d03aa5baaa823c16771832f254372f50a" exitCode=0
Jul 27 19:33:22.259079 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:22.258701    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afktfkf" event={"ID":"4133fc41-332a-4355-b0e9-5cc0541c85e5","Type":"ContainerDied","Data":"2982e45e8f7cda2495fa3ddbb0271f3d03aa5baaa823c16771832f254372f50a"}
Jul 27 19:33:23.268138 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:23.268102    2558 generic.go:356] "Generic (PLEG): container finished" podID="4133fc41-332a-4355-b0e9-5cc0541c85e5" containerID="626aa40b1c5432ba9ca514e38462cae6e07fe92e6bc5af2c7a4882f25e9a0f49" exitCode=0
Jul 27 19:33:23.268549 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:23.268186    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afktfkf" event={"ID":"4133fc41-332a-4355-b0e9-5cc0541c85e5","Type":"ContainerDied","Data":"626aa40b1c5432ba9ca514e38462cae6e07fe92e6bc5af2c7a4882f25e9a0f49"}
Jul 27 19:33:24.391559 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:24.391534    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afktfkf"
Jul 27 19:33:24.446783 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:24.446753    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4133fc41-332a-4355-b0e9-5cc0541c85e5-util\") pod \"4133fc41-332a-4355-b0e9-5cc0541c85e5\" (UID: \"4133fc41-332a-4355-b0e9-5cc0541c85e5\") "
Jul 27 19:33:24.446978 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:24.446829    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4133fc41-332a-4355-b0e9-5cc0541c85e5-bundle\") pod \"4133fc41-332a-4355-b0e9-5cc0541c85e5\" (UID: \"4133fc41-332a-4355-b0e9-5cc0541c85e5\") "
Jul 27 19:33:24.446978 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:24.446856    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nhphm\" (UniqueName: \"kubernetes.io/projected/4133fc41-332a-4355-b0e9-5cc0541c85e5-kube-api-access-nhphm\") pod \"4133fc41-332a-4355-b0e9-5cc0541c85e5\" (UID: \"4133fc41-332a-4355-b0e9-5cc0541c85e5\") "
Jul 27 19:33:24.447549 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:24.447508    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4133fc41-332a-4355-b0e9-5cc0541c85e5-bundle" (OuterVolumeSpecName: "bundle") pod "4133fc41-332a-4355-b0e9-5cc0541c85e5" (UID: "4133fc41-332a-4355-b0e9-5cc0541c85e5"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:33:24.449128 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:24.449098    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4133fc41-332a-4355-b0e9-5cc0541c85e5-kube-api-access-nhphm" (OuterVolumeSpecName: "kube-api-access-nhphm") pod "4133fc41-332a-4355-b0e9-5cc0541c85e5" (UID: "4133fc41-332a-4355-b0e9-5cc0541c85e5"). InnerVolumeSpecName "kube-api-access-nhphm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:33:24.453117 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:24.453089    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4133fc41-332a-4355-b0e9-5cc0541c85e5-util" (OuterVolumeSpecName: "util") pod "4133fc41-332a-4355-b0e9-5cc0541c85e5" (UID: "4133fc41-332a-4355-b0e9-5cc0541c85e5"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:33:24.547670 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:24.547601    2558 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4133fc41-332a-4355-b0e9-5cc0541c85e5-bundle\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:33:24.547670 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:24.547625    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nhphm\" (UniqueName: \"kubernetes.io/projected/4133fc41-332a-4355-b0e9-5cc0541c85e5-kube-api-access-nhphm\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:33:24.547670 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:24.547638    2558 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4133fc41-332a-4355-b0e9-5cc0541c85e5-util\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:33:25.279632 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:25.279607    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afktfkf"
Jul 27 19:33:25.279860 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:25.279606    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afktfkf" event={"ID":"4133fc41-332a-4355-b0e9-5cc0541c85e5","Type":"ContainerDied","Data":"824f6280574e9e2e9413468cf6223660f22459d16e67757905b258af5f281795"}
Jul 27 19:33:25.279860 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:25.279726    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="824f6280574e9e2e9413468cf6223660f22459d16e67757905b258af5f281795"
Jul 27 19:33:30.575108 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:30.575078    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:33.997523 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:33.997485    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:34.057459 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:34.057423    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-86d698cc8c-g5xw9"]
Jul 27 19:33:34.057804 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:34.057782    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4133fc41-332a-4355-b0e9-5cc0541c85e5" containerName="util"
Jul 27 19:33:34.057804 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:34.057804    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="4133fc41-332a-4355-b0e9-5cc0541c85e5" containerName="util"
Jul 27 19:33:34.057970 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:34.057816    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4133fc41-332a-4355-b0e9-5cc0541c85e5" containerName="extract"
Jul 27 19:33:34.057970 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:34.057822    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="4133fc41-332a-4355-b0e9-5cc0541c85e5" containerName="extract"
Jul 27 19:33:34.057970 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:34.057841    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4133fc41-332a-4355-b0e9-5cc0541c85e5" containerName="pull"
Jul 27 19:33:34.057970 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:34.057846    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="4133fc41-332a-4355-b0e9-5cc0541c85e5" containerName="pull"
Jul 27 19:33:34.057970 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:34.057897    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="4133fc41-332a-4355-b0e9-5cc0541c85e5" containerName="extract"
Jul 27 19:33:34.082594 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:34.082565    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-86d698cc8c-g5xw9"]
Jul 27 19:33:34.082734 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:34.082575    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-86d698cc8c-g5xw9"
Jul 27 19:33:34.085373 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:34.085336    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 27 19:33:34.085522 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:34.085378    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 27 19:33:34.085522 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:34.085336    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 27 19:33:34.086635 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:34.086609    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:34.086760 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:34.086694    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 27 19:33:34.086760 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:34.086705    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-ckjgb\""
Jul 27 19:33:34.086760 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:34.086651    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 27 19:33:34.228875 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:34.228841    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/15e0f92b-511e-482d-a5ec-69408cf73193-manager-config\") pod \"lws-controller-manager-86d698cc8c-g5xw9\" (UID: \"15e0f92b-511e-482d-a5ec-69408cf73193\") " pod="openshift-lws-operator/lws-controller-manager-86d698cc8c-g5xw9"
Jul 27 19:33:34.229062 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:34.228884    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/15e0f92b-511e-482d-a5ec-69408cf73193-metrics-cert\") pod \"lws-controller-manager-86d698cc8c-g5xw9\" (UID: \"15e0f92b-511e-482d-a5ec-69408cf73193\") " pod="openshift-lws-operator/lws-controller-manager-86d698cc8c-g5xw9"
Jul 27 19:33:34.229062 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:34.228923    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/15e0f92b-511e-482d-a5ec-69408cf73193-cert\") pod \"lws-controller-manager-86d698cc8c-g5xw9\" (UID: \"15e0f92b-511e-482d-a5ec-69408cf73193\") " pod="openshift-lws-operator/lws-controller-manager-86d698cc8c-g5xw9"
Jul 27 19:33:34.229062 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:34.228952    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpcgd\" (UniqueName: \"kubernetes.io/projected/15e0f92b-511e-482d-a5ec-69408cf73193-kube-api-access-lpcgd\") pod \"lws-controller-manager-86d698cc8c-g5xw9\" (UID: \"15e0f92b-511e-482d-a5ec-69408cf73193\") " pod="openshift-lws-operator/lws-controller-manager-86d698cc8c-g5xw9"
Jul 27 19:33:34.330354 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:34.330273    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/15e0f92b-511e-482d-a5ec-69408cf73193-manager-config\") pod \"lws-controller-manager-86d698cc8c-g5xw9\" (UID: \"15e0f92b-511e-482d-a5ec-69408cf73193\") " pod="openshift-lws-operator/lws-controller-manager-86d698cc8c-g5xw9"
Jul 27 19:33:34.330354 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:34.330315    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/15e0f92b-511e-482d-a5ec-69408cf73193-metrics-cert\") pod \"lws-controller-manager-86d698cc8c-g5xw9\" (UID: \"15e0f92b-511e-482d-a5ec-69408cf73193\") " pod="openshift-lws-operator/lws-controller-manager-86d698cc8c-g5xw9"
Jul 27 19:33:34.330354 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:34.330347    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/15e0f92b-511e-482d-a5ec-69408cf73193-cert\") pod \"lws-controller-manager-86d698cc8c-g5xw9\" (UID: \"15e0f92b-511e-482d-a5ec-69408cf73193\") " pod="openshift-lws-operator/lws-controller-manager-86d698cc8c-g5xw9"
Jul 27 19:33:34.330612 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:34.330379    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpcgd\" (UniqueName: \"kubernetes.io/projected/15e0f92b-511e-482d-a5ec-69408cf73193-kube-api-access-lpcgd\") pod \"lws-controller-manager-86d698cc8c-g5xw9\" (UID: \"15e0f92b-511e-482d-a5ec-69408cf73193\") " pod="openshift-lws-operator/lws-controller-manager-86d698cc8c-g5xw9"
Jul 27 19:33:34.330962 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:34.330920    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/15e0f92b-511e-482d-a5ec-69408cf73193-manager-config\") pod \"lws-controller-manager-86d698cc8c-g5xw9\" (UID: \"15e0f92b-511e-482d-a5ec-69408cf73193\") " pod="openshift-lws-operator/lws-controller-manager-86d698cc8c-g5xw9"
Jul 27 19:33:34.333035 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:34.333014    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/15e0f92b-511e-482d-a5ec-69408cf73193-cert\") pod \"lws-controller-manager-86d698cc8c-g5xw9\" (UID: \"15e0f92b-511e-482d-a5ec-69408cf73193\") " pod="openshift-lws-operator/lws-controller-manager-86d698cc8c-g5xw9"
Jul 27 19:33:34.333112 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:34.333012    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/15e0f92b-511e-482d-a5ec-69408cf73193-metrics-cert\") pod \"lws-controller-manager-86d698cc8c-g5xw9\" (UID: \"15e0f92b-511e-482d-a5ec-69408cf73193\") " pod="openshift-lws-operator/lws-controller-manager-86d698cc8c-g5xw9"
Jul 27 19:33:34.340303 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:34.340276    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpcgd\" (UniqueName: \"kubernetes.io/projected/15e0f92b-511e-482d-a5ec-69408cf73193-kube-api-access-lpcgd\") pod \"lws-controller-manager-86d698cc8c-g5xw9\" (UID: \"15e0f92b-511e-482d-a5ec-69408cf73193\") " pod="openshift-lws-operator/lws-controller-manager-86d698cc8c-g5xw9"
Jul 27 19:33:34.393013 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:34.392983    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-86d698cc8c-g5xw9"
Jul 27 19:33:34.559733 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:34.559699    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-86d698cc8c-g5xw9"]
Jul 27 19:33:34.561962 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:33:34.561937    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15e0f92b_511e_482d_a5ec_69408cf73193.slice/crio-7cbf173e6a602a649d4693666d8f2a234a40b9e85507fc350d56dcc3b51d150d WatchSource:0}: Error finding container 7cbf173e6a602a649d4693666d8f2a234a40b9e85507fc350d56dcc3b51d150d: Status 404 returned error can't find the container with id 7cbf173e6a602a649d4693666d8f2a234a40b9e85507fc350d56dcc3b51d150d
Jul 27 19:33:35.327631 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:35.327587    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-86d698cc8c-g5xw9" event={"ID":"15e0f92b-511e-482d-a5ec-69408cf73193","Type":"ContainerStarted","Data":"7cbf173e6a602a649d4693666d8f2a234a40b9e85507fc350d56dcc3b51d150d"}
Jul 27 19:33:36.261326 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:36.261295    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:36.334072 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:36.333975    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-86d698cc8c-g5xw9" event={"ID":"15e0f92b-511e-482d-a5ec-69408cf73193","Type":"ContainerStarted","Data":"9276ef2cd0d50432e42df17288bec802f1e6f572f061507bfcdf7c0ad894fe1c"}
Jul 27 19:33:36.334455 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:36.334066    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-86d698cc8c-g5xw9"
Jul 27 19:33:36.348006 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:36.347976    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:36.352169 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:36.351838    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-86d698cc8c-g5xw9" podStartSLOduration=0.869824359 podStartE2EDuration="2.351819391s" totalImagesPullingTime="1.481995032s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:33:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:33:36.350465888 +0000 UTC m=+429.222657859" watchObservedRunningTime="2026-07-27 19:33:36.351819391 +0000 UTC m=+429.224011368"
Jul 27 19:33:36.874626 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:36.874585    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:37.130579 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:37.130492    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:37.219817 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:37.219784    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6c9f4fc9b4-zbrwt"]
Jul 27 19:33:37.222288 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:37.222268    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6c9f4fc9b4-zbrwt"
Jul 27 19:33:37.225032 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:37.225008    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 27 19:33:37.225158 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:37.225119    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-grzqz\""
Jul 27 19:33:37.225255 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:37.225239    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 27 19:33:37.225317 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:37.225257    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 27 19:33:37.225684 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:37.225669    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 27 19:33:37.242210 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:37.242185    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6c9f4fc9b4-zbrwt"]
Jul 27 19:33:37.277338 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:37.277309    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:37.355696 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:37.355666    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbn4g\" (UniqueName: \"kubernetes.io/projected/6bd0e839-62c0-48cc-a815-d1b944088d2e-kube-api-access-dbn4g\") pod \"opendatahub-operator-controller-manager-6c9f4fc9b4-zbrwt\" (UID: \"6bd0e839-62c0-48cc-a815-d1b944088d2e\") " pod="opendatahub/opendatahub-operator-controller-manager-6c9f4fc9b4-zbrwt"
Jul 27 19:33:37.356062 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:37.355705    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6bd0e839-62c0-48cc-a815-d1b944088d2e-manifests\") pod \"opendatahub-operator-controller-manager-6c9f4fc9b4-zbrwt\" (UID: \"6bd0e839-62c0-48cc-a815-d1b944088d2e\") " pod="opendatahub/opendatahub-operator-controller-manager-6c9f4fc9b4-zbrwt"
Jul 27 19:33:37.356062 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:37.355739    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6bd0e839-62c0-48cc-a815-d1b944088d2e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6c9f4fc9b4-zbrwt\" (UID: \"6bd0e839-62c0-48cc-a815-d1b944088d2e\") " pod="opendatahub/opendatahub-operator-controller-manager-6c9f4fc9b4-zbrwt"
Jul 27 19:33:37.356062 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:37.355793    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6bd0e839-62c0-48cc-a815-d1b944088d2e-tmp\") pod \"opendatahub-operator-controller-manager-6c9f4fc9b4-zbrwt\" (UID: \"6bd0e839-62c0-48cc-a815-d1b944088d2e\") " pod="opendatahub/opendatahub-operator-controller-manager-6c9f4fc9b4-zbrwt"
Jul 27 19:33:37.356062 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:37.355827    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6bd0e839-62c0-48cc-a815-d1b944088d2e-webhook-cert\") pod \"opendatahub-operator-controller-manager-6c9f4fc9b4-zbrwt\" (UID: \"6bd0e839-62c0-48cc-a815-d1b944088d2e\") " pod="opendatahub/opendatahub-operator-controller-manager-6c9f4fc9b4-zbrwt"
Jul 27 19:33:37.380289 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:37.380265    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:37.456843 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:37.456758    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6bd0e839-62c0-48cc-a815-d1b944088d2e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6c9f4fc9b4-zbrwt\" (UID: \"6bd0e839-62c0-48cc-a815-d1b944088d2e\") " pod="opendatahub/opendatahub-operator-controller-manager-6c9f4fc9b4-zbrwt"
Jul 27 19:33:37.456843 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:37.456802    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6bd0e839-62c0-48cc-a815-d1b944088d2e-tmp\") pod \"opendatahub-operator-controller-manager-6c9f4fc9b4-zbrwt\" (UID: \"6bd0e839-62c0-48cc-a815-d1b944088d2e\") " pod="opendatahub/opendatahub-operator-controller-manager-6c9f4fc9b4-zbrwt"
Jul 27 19:33:37.457256 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:37.456988    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6bd0e839-62c0-48cc-a815-d1b944088d2e-webhook-cert\") pod \"opendatahub-operator-controller-manager-6c9f4fc9b4-zbrwt\" (UID: \"6bd0e839-62c0-48cc-a815-d1b944088d2e\") " pod="opendatahub/opendatahub-operator-controller-manager-6c9f4fc9b4-zbrwt"
Jul 27 19:33:37.457256 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:37.457066    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbn4g\" (UniqueName: \"kubernetes.io/projected/6bd0e839-62c0-48cc-a815-d1b944088d2e-kube-api-access-dbn4g\") pod \"opendatahub-operator-controller-manager-6c9f4fc9b4-zbrwt\" (UID: \"6bd0e839-62c0-48cc-a815-d1b944088d2e\") " pod="opendatahub/opendatahub-operator-controller-manager-6c9f4fc9b4-zbrwt"
Jul 27 19:33:37.457256 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:37.457123    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6bd0e839-62c0-48cc-a815-d1b944088d2e-manifests\") pod \"opendatahub-operator-controller-manager-6c9f4fc9b4-zbrwt\" (UID: \"6bd0e839-62c0-48cc-a815-d1b944088d2e\") " pod="opendatahub/opendatahub-operator-controller-manager-6c9f4fc9b4-zbrwt"
Jul 27 19:33:37.457555 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:37.457484    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6bd0e839-62c0-48cc-a815-d1b944088d2e-manifests\") pod \"opendatahub-operator-controller-manager-6c9f4fc9b4-zbrwt\" (UID: \"6bd0e839-62c0-48cc-a815-d1b944088d2e\") " pod="opendatahub/opendatahub-operator-controller-manager-6c9f4fc9b4-zbrwt"
Jul 27 19:33:37.459389 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:37.459358    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6bd0e839-62c0-48cc-a815-d1b944088d2e-tmp\") pod \"opendatahub-operator-controller-manager-6c9f4fc9b4-zbrwt\" (UID: \"6bd0e839-62c0-48cc-a815-d1b944088d2e\") " pod="opendatahub/opendatahub-operator-controller-manager-6c9f4fc9b4-zbrwt"
Jul 27 19:33:37.459547 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:37.459529    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6bd0e839-62c0-48cc-a815-d1b944088d2e-webhook-cert\") pod \"opendatahub-operator-controller-manager-6c9f4fc9b4-zbrwt\" (UID: \"6bd0e839-62c0-48cc-a815-d1b944088d2e\") " pod="opendatahub/opendatahub-operator-controller-manager-6c9f4fc9b4-zbrwt"
Jul 27 19:33:37.459788 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:37.459770    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6bd0e839-62c0-48cc-a815-d1b944088d2e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6c9f4fc9b4-zbrwt\" (UID: \"6bd0e839-62c0-48cc-a815-d1b944088d2e\") " pod="opendatahub/opendatahub-operator-controller-manager-6c9f4fc9b4-zbrwt"
Jul 27 19:33:37.465199 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:37.465180    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbn4g\" (UniqueName: \"kubernetes.io/projected/6bd0e839-62c0-48cc-a815-d1b944088d2e-kube-api-access-dbn4g\") pod \"opendatahub-operator-controller-manager-6c9f4fc9b4-zbrwt\" (UID: \"6bd0e839-62c0-48cc-a815-d1b944088d2e\") " pod="opendatahub/opendatahub-operator-controller-manager-6c9f4fc9b4-zbrwt"
Jul 27 19:33:37.478912 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:37.478888    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:37.532898 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:37.532866    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6c9f4fc9b4-zbrwt"
Jul 27 19:33:37.605930 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:37.605904    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:37.713918 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:37.713847    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6c9f4fc9b4-zbrwt"]
Jul 27 19:33:37.717521 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:33:37.717492    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bd0e839_62c0_48cc_a815_d1b944088d2e.slice/crio-dbd3e77b484ab2f53be55fd83b31bf17498ddcc865345812106daa83e7b1e78e WatchSource:0}: Error finding container dbd3e77b484ab2f53be55fd83b31bf17498ddcc865345812106daa83e7b1e78e: Status 404 returned error can't find the container with id dbd3e77b484ab2f53be55fd83b31bf17498ddcc865345812106daa83e7b1e78e
Jul 27 19:33:37.718106 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:37.718081    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:37.805226 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:37.805192    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:37.876179 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:37.876132    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:38.346814 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:38.346776    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6c9f4fc9b4-zbrwt" event={"ID":"6bd0e839-62c0-48cc-a815-d1b944088d2e","Type":"ContainerStarted","Data":"dbd3e77b484ab2f53be55fd83b31bf17498ddcc865345812106daa83e7b1e78e"}
Jul 27 19:33:41.367028 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:41.367000    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6c9f4fc9b4-zbrwt" event={"ID":"6bd0e839-62c0-48cc-a815-d1b944088d2e","Type":"ContainerStarted","Data":"6c525e725d4823aa6e11b1fa5969e15ed4431c755e32734d165cbd180b6b41e0"}
Jul 27 19:33:42.372763 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:42.372731    2558 generic.go:356] "Generic (PLEG): container finished" podID="6bd0e839-62c0-48cc-a815-d1b944088d2e" containerID="6c525e725d4823aa6e11b1fa5969e15ed4431c755e32734d165cbd180b6b41e0" exitCode=0
Jul 27 19:33:42.373172 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:42.372777    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6c9f4fc9b4-zbrwt" event={"ID":"6bd0e839-62c0-48cc-a815-d1b944088d2e","Type":"ContainerDied","Data":"6c525e725d4823aa6e11b1fa5969e15ed4431c755e32734d165cbd180b6b41e0"}
Jul 27 19:33:43.379549 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:43.379514    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6c9f4fc9b4-zbrwt" event={"ID":"6bd0e839-62c0-48cc-a815-d1b944088d2e","Type":"ContainerStarted","Data":"2b7a26d89c07bf6ef0af15190d8b551592b51eb43507cf1201000a2c2c161783"}
Jul 27 19:33:43.379937 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:43.379681    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6c9f4fc9b4-zbrwt"
Jul 27 19:33:43.402610 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:43.402560    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6c9f4fc9b4-zbrwt" podStartSLOduration=2.264323897 podStartE2EDuration="6.402546299s" totalImagesPullingTime="3.134916547s" totalInitContainerRuntime="1.003305855s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:33:37 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:33:43.399803572 +0000 UTC m=+436.271995549" watchObservedRunningTime="2026-07-27 19:33:43.402546299 +0000 UTC m=+436.274738272"
Jul 27 19:33:47.340337 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:47.340300    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-86d698cc8c-g5xw9"
Jul 27 19:33:48.990234 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:48.990132    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:49.090212 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:49.090175    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:54.385766 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:54.385739    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6c9f4fc9b4-zbrwt"
Jul 27 19:33:56.067134 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:56.067103    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:56.250049 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:56.249994    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:59.169858 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:59.169826    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:59.367835 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:59.367791    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5b7cdcf968-xsw47"]
Jul 27 19:33:59.371394 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:59.371375    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5b7cdcf968-xsw47"
Jul 27 19:33:59.376251 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:59.376222    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 27 19:33:59.376613 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:59.376593    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 27 19:33:59.377256 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:59.377235    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-z6zwf\""
Jul 27 19:33:59.377367 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:59.377255    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 27 19:33:59.377367 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:59.377277    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 27 19:33:59.403479 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:59.403452    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5b7cdcf968-xsw47"]
Jul 27 19:33:59.448160 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:59.448087    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5cb52b2-b8b8-463c-b3cc-96e8e46f26cd-tls-certs\") pod \"kube-auth-proxy-5b7cdcf968-xsw47\" (UID: \"b5cb52b2-b8b8-463c-b3cc-96e8e46f26cd\") " pod="openshift-ingress/kube-auth-proxy-5b7cdcf968-xsw47"
Jul 27 19:33:59.448160 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:59.448129    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b5cb52b2-b8b8-463c-b3cc-96e8e46f26cd-tmp\") pod \"kube-auth-proxy-5b7cdcf968-xsw47\" (UID: \"b5cb52b2-b8b8-463c-b3cc-96e8e46f26cd\") " pod="openshift-ingress/kube-auth-proxy-5b7cdcf968-xsw47"
Jul 27 19:33:59.448312 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:59.448199    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kcrf2\" (UniqueName: \"kubernetes.io/projected/b5cb52b2-b8b8-463c-b3cc-96e8e46f26cd-kube-api-access-kcrf2\") pod \"kube-auth-proxy-5b7cdcf968-xsw47\" (UID: \"b5cb52b2-b8b8-463c-b3cc-96e8e46f26cd\") " pod="openshift-ingress/kube-auth-proxy-5b7cdcf968-xsw47"
Jul 27 19:33:59.476398 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:59.476364    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:59.549018 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:59.548990    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kcrf2\" (UniqueName: \"kubernetes.io/projected/b5cb52b2-b8b8-463c-b3cc-96e8e46f26cd-kube-api-access-kcrf2\") pod \"kube-auth-proxy-5b7cdcf968-xsw47\" (UID: \"b5cb52b2-b8b8-463c-b3cc-96e8e46f26cd\") " pod="openshift-ingress/kube-auth-proxy-5b7cdcf968-xsw47"
Jul 27 19:33:59.549166 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:59.549041    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5cb52b2-b8b8-463c-b3cc-96e8e46f26cd-tls-certs\") pod \"kube-auth-proxy-5b7cdcf968-xsw47\" (UID: \"b5cb52b2-b8b8-463c-b3cc-96e8e46f26cd\") " pod="openshift-ingress/kube-auth-proxy-5b7cdcf968-xsw47"
Jul 27 19:33:59.549166 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:59.549080    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b5cb52b2-b8b8-463c-b3cc-96e8e46f26cd-tmp\") pod \"kube-auth-proxy-5b7cdcf968-xsw47\" (UID: \"b5cb52b2-b8b8-463c-b3cc-96e8e46f26cd\") " pod="openshift-ingress/kube-auth-proxy-5b7cdcf968-xsw47"
Jul 27 19:33:59.551893 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:59.551867    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b5cb52b2-b8b8-463c-b3cc-96e8e46f26cd-tmp\") pod \"kube-auth-proxy-5b7cdcf968-xsw47\" (UID: \"b5cb52b2-b8b8-463c-b3cc-96e8e46f26cd\") " pod="openshift-ingress/kube-auth-proxy-5b7cdcf968-xsw47"
Jul 27 19:33:59.552039 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:59.551968    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5cb52b2-b8b8-463c-b3cc-96e8e46f26cd-tls-certs\") pod \"kube-auth-proxy-5b7cdcf968-xsw47\" (UID: \"b5cb52b2-b8b8-463c-b3cc-96e8e46f26cd\") " pod="openshift-ingress/kube-auth-proxy-5b7cdcf968-xsw47"
Jul 27 19:33:59.567340 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:59.567318    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kcrf2\" (UniqueName: \"kubernetes.io/projected/b5cb52b2-b8b8-463c-b3cc-96e8e46f26cd-kube-api-access-kcrf2\") pod \"kube-auth-proxy-5b7cdcf968-xsw47\" (UID: \"b5cb52b2-b8b8-463c-b3cc-96e8e46f26cd\") " pod="openshift-ingress/kube-auth-proxy-5b7cdcf968-xsw47"
Jul 27 19:33:59.567643 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:59.567576    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:59.642958 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:59.642902    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:59.683580 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:59.683539    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5b7cdcf968-xsw47"
Jul 27 19:33:59.743799 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:59.743773    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:59.845821 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:33:59.845792    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5b7cdcf968-xsw47"]
Jul 27 19:33:59.848996 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:33:59.848965    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5cb52b2_b8b8_463c_b3cc_96e8e46f26cd.slice/crio-d6cbe154e1a44d1f1f5ffdb10b6882557f9a5957c8b675b504218716379482b3 WatchSource:0}: Error finding container d6cbe154e1a44d1f1f5ffdb10b6882557f9a5957c8b675b504218716379482b3: Status 404 returned error can't find the container with id d6cbe154e1a44d1f1f5ffdb10b6882557f9a5957c8b675b504218716379482b3
Jul 27 19:34:00.209319 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:34:00.209288    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:34:00.458620 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:34:00.458566    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5b7cdcf968-xsw47" event={"ID":"b5cb52b2-b8b8-463c-b3cc-96e8e46f26cd","Type":"ContainerStarted","Data":"d6cbe154e1a44d1f1f5ffdb10b6882557f9a5957c8b675b504218716379482b3"}
Jul 27 19:34:00.516280 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:34:00.516192    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:34:00.668737 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:34:00.668703    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:34:00.773298 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:34:00.773205    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:34:02.180118 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:34:02.180078    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:34:02.470421 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:34:02.470323    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5b7cdcf968-xsw47" event={"ID":"b5cb52b2-b8b8-463c-b3cc-96e8e46f26cd","Type":"ContainerStarted","Data":"34e6d163738133a1d295843232dcc5b8e74bd5378a65148c7679b509d2a13860"}
Jul 27 19:34:05.688524 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:34:05.688489    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:34:05.800376 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:34:05.800339    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:34:06.104032 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:34:06.103950    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:34:06.378916 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:34:06.378880    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:34:06.480128 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:34:06.480094    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:34:10.750742 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:34:10.750711    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:34:10.856387 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:34:10.856354    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:34:10.933948 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:34:10.933917    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:34:11.160899 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:34:11.160871    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:34:11.397235 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:34:11.397204    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:34:12.093450 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:34:12.093394    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:34:12.189278 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:34:12.189245    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:34:57.685371 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:34:57.685337    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:34:57.759284 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:34:57.759255    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:34:57.903062 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:34:57.903032    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:35:09.142818 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:09.142784    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:35:09.243869 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:09.243838    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:35:13.226869 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:13.226801    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5b7cdcf968-xsw47" podStartSLOduration=72.216933793 podStartE2EDuration="1m14.226783321s" totalImagesPullingTime="2.009849528s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:33:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:34:02.513763546 +0000 UTC m=+455.385955533" watchObservedRunningTime="2026-07-27 19:35:13.226783321 +0000 UTC m=+526.098975314"
Jul 27 19:35:13.228296 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:13.228277    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qrh56"]
Jul 27 19:35:13.230969 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:13.230952    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qrh56"
Jul 27 19:35:13.233468 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:13.233446    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 27 19:35:13.233468 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:13.233462    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 27 19:35:13.234130 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:13.234113    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-d5plc\""
Jul 27 19:35:13.239424 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:13.239385    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qrh56"]
Jul 27 19:35:13.343720 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:13.343685    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2cf47ec3-ce9f-4a23-9a8a-b45415b64482-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qrh56\" (UID: \"2cf47ec3-ce9f-4a23-9a8a-b45415b64482\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qrh56"
Jul 27 19:35:13.343720 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:13.343725    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2cf47ec3-ce9f-4a23-9a8a-b45415b64482-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qrh56\" (UID: \"2cf47ec3-ce9f-4a23-9a8a-b45415b64482\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qrh56"
Jul 27 19:35:13.343944 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:13.343749    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4547r\" (UniqueName: \"kubernetes.io/projected/2cf47ec3-ce9f-4a23-9a8a-b45415b64482-kube-api-access-4547r\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qrh56\" (UID: \"2cf47ec3-ce9f-4a23-9a8a-b45415b64482\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qrh56"
Jul 27 19:35:13.444500 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:13.444463    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4547r\" (UniqueName: \"kubernetes.io/projected/2cf47ec3-ce9f-4a23-9a8a-b45415b64482-kube-api-access-4547r\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qrh56\" (UID: \"2cf47ec3-ce9f-4a23-9a8a-b45415b64482\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qrh56"
Jul 27 19:35:13.444668 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:13.444559    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2cf47ec3-ce9f-4a23-9a8a-b45415b64482-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qrh56\" (UID: \"2cf47ec3-ce9f-4a23-9a8a-b45415b64482\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qrh56"
Jul 27 19:35:13.444668 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:13.444580    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2cf47ec3-ce9f-4a23-9a8a-b45415b64482-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qrh56\" (UID: \"2cf47ec3-ce9f-4a23-9a8a-b45415b64482\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qrh56"
Jul 27 19:35:13.444892 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:13.444876    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2cf47ec3-ce9f-4a23-9a8a-b45415b64482-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qrh56\" (UID: \"2cf47ec3-ce9f-4a23-9a8a-b45415b64482\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qrh56"
Jul 27 19:35:13.444974 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:13.444911    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2cf47ec3-ce9f-4a23-9a8a-b45415b64482-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qrh56\" (UID: \"2cf47ec3-ce9f-4a23-9a8a-b45415b64482\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qrh56"
Jul 27 19:35:13.452852 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:13.452821    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4547r\" (UniqueName: \"kubernetes.io/projected/2cf47ec3-ce9f-4a23-9a8a-b45415b64482-kube-api-access-4547r\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qrh56\" (UID: \"2cf47ec3-ce9f-4a23-9a8a-b45415b64482\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qrh56"
Jul 27 19:35:13.540425 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:13.540343    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qrh56"
Jul 27 19:35:13.671985 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:35:13.671954    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cf47ec3_ce9f_4a23_9a8a_b45415b64482.slice/crio-1a95358934d6e90f6e416f5716b3790e86ab6b2fc3c7b3fcd0223a89dad93b5c WatchSource:0}: Error finding container 1a95358934d6e90f6e416f5716b3790e86ab6b2fc3c7b3fcd0223a89dad93b5c: Status 404 returned error can't find the container with id 1a95358934d6e90f6e416f5716b3790e86ab6b2fc3c7b3fcd0223a89dad93b5c
Jul 27 19:35:13.675126 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:13.675101    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qrh56"]
Jul 27 19:35:13.800530 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:13.800455    2558 generic.go:356] "Generic (PLEG): container finished" podID="2cf47ec3-ce9f-4a23-9a8a-b45415b64482" containerID="d79f3f7e89e6f4bcf8317ec0d4a03a3ea085d82a1973fe8a13311832156870fe" exitCode=0
Jul 27 19:35:13.800658 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:13.800556    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qrh56" event={"ID":"2cf47ec3-ce9f-4a23-9a8a-b45415b64482","Type":"ContainerDied","Data":"d79f3f7e89e6f4bcf8317ec0d4a03a3ea085d82a1973fe8a13311832156870fe"}
Jul 27 19:35:13.800658 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:13.800609    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qrh56" event={"ID":"2cf47ec3-ce9f-4a23-9a8a-b45415b64482","Type":"ContainerStarted","Data":"1a95358934d6e90f6e416f5716b3790e86ab6b2fc3c7b3fcd0223a89dad93b5c"}
Jul 27 19:35:14.021663 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:14.021629    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cvkh7"]
Jul 27 19:35:14.024464 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:14.024444    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cvkh7"
Jul 27 19:35:14.033342 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:14.033315    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cvkh7"]
Jul 27 19:35:14.150295 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:14.150258    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4a03b37a-e996-4c5f-bb3a-3201a0481e74-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cvkh7\" (UID: \"4a03b37a-e996-4c5f-bb3a-3201a0481e74\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cvkh7"
Jul 27 19:35:14.150295 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:14.150302    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s858s\" (UniqueName: \"kubernetes.io/projected/4a03b37a-e996-4c5f-bb3a-3201a0481e74-kube-api-access-s858s\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cvkh7\" (UID: \"4a03b37a-e996-4c5f-bb3a-3201a0481e74\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cvkh7"
Jul 27 19:35:14.150541 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:14.150348    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4a03b37a-e996-4c5f-bb3a-3201a0481e74-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cvkh7\" (UID: \"4a03b37a-e996-4c5f-bb3a-3201a0481e74\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cvkh7"
Jul 27 19:35:14.251061 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:14.251024    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4a03b37a-e996-4c5f-bb3a-3201a0481e74-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cvkh7\" (UID: \"4a03b37a-e996-4c5f-bb3a-3201a0481e74\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cvkh7"
Jul 27 19:35:14.251479 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:14.251102    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4a03b37a-e996-4c5f-bb3a-3201a0481e74-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cvkh7\" (UID: \"4a03b37a-e996-4c5f-bb3a-3201a0481e74\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cvkh7"
Jul 27 19:35:14.251479 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:14.251129    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s858s\" (UniqueName: \"kubernetes.io/projected/4a03b37a-e996-4c5f-bb3a-3201a0481e74-kube-api-access-s858s\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cvkh7\" (UID: \"4a03b37a-e996-4c5f-bb3a-3201a0481e74\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cvkh7"
Jul 27 19:35:14.251479 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:14.251375    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4a03b37a-e996-4c5f-bb3a-3201a0481e74-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cvkh7\" (UID: \"4a03b37a-e996-4c5f-bb3a-3201a0481e74\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cvkh7"
Jul 27 19:35:14.251479 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:14.251470    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4a03b37a-e996-4c5f-bb3a-3201a0481e74-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cvkh7\" (UID: \"4a03b37a-e996-4c5f-bb3a-3201a0481e74\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cvkh7"
Jul 27 19:35:14.259969 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:14.259948    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s858s\" (UniqueName: \"kubernetes.io/projected/4a03b37a-e996-4c5f-bb3a-3201a0481e74-kube-api-access-s858s\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cvkh7\" (UID: \"4a03b37a-e996-4c5f-bb3a-3201a0481e74\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cvkh7"
Jul 27 19:35:14.334460 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:14.334432    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cvkh7"
Jul 27 19:35:14.507767 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:14.507746    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cvkh7"]
Jul 27 19:35:14.551297 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:35:14.551266    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a03b37a_e996_4c5f_bb3a_3201a0481e74.slice/crio-2bd57e1dd0a704e8f8d45c1751d698ac23e50ab545fc688c56eff17c88bd9f6e WatchSource:0}: Error finding container 2bd57e1dd0a704e8f8d45c1751d698ac23e50ab545fc688c56eff17c88bd9f6e: Status 404 returned error can't find the container with id 2bd57e1dd0a704e8f8d45c1751d698ac23e50ab545fc688c56eff17c88bd9f6e
Jul 27 19:35:14.623959 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:14.623928    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kvgqm"]
Jul 27 19:35:14.626519 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:14.626498    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kvgqm"
Jul 27 19:35:14.635365 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:14.635337    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kvgqm"]
Jul 27 19:35:14.755241 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:14.755214    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/641fc784-7dcc-4cfc-9e4b-6f1b50f224dd-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kvgqm\" (UID: \"641fc784-7dcc-4cfc-9e4b-6f1b50f224dd\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kvgqm"
Jul 27 19:35:14.755814 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:14.755491    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/641fc784-7dcc-4cfc-9e4b-6f1b50f224dd-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kvgqm\" (UID: \"641fc784-7dcc-4cfc-9e4b-6f1b50f224dd\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kvgqm"
Jul 27 19:35:14.755957 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:14.755901    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qmh8\" (UniqueName: \"kubernetes.io/projected/641fc784-7dcc-4cfc-9e4b-6f1b50f224dd-kube-api-access-7qmh8\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kvgqm\" (UID: \"641fc784-7dcc-4cfc-9e4b-6f1b50f224dd\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kvgqm"
Jul 27 19:35:14.807233 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:14.807202    2558 generic.go:356] "Generic (PLEG): container finished" podID="2cf47ec3-ce9f-4a23-9a8a-b45415b64482" containerID="5bd69b3e4ccfedab4aa1fa34562cefec70ee1b37dccae8f6b4cb8b3a8c593dbb" exitCode=0
Jul 27 19:35:14.807356 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:14.807239    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qrh56" event={"ID":"2cf47ec3-ce9f-4a23-9a8a-b45415b64482","Type":"ContainerDied","Data":"5bd69b3e4ccfedab4aa1fa34562cefec70ee1b37dccae8f6b4cb8b3a8c593dbb"}
Jul 27 19:35:14.808940 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:14.808921    2558 generic.go:356] "Generic (PLEG): container finished" podID="4a03b37a-e996-4c5f-bb3a-3201a0481e74" containerID="3d125d0b6fbcc8a1f37313dd51c443b91400150e955bbdbfd3a5c00acb633180" exitCode=0
Jul 27 19:35:14.808998 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:14.808981    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cvkh7" event={"ID":"4a03b37a-e996-4c5f-bb3a-3201a0481e74","Type":"ContainerDied","Data":"3d125d0b6fbcc8a1f37313dd51c443b91400150e955bbdbfd3a5c00acb633180"}
Jul 27 19:35:14.809041 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:14.809015    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cvkh7" event={"ID":"4a03b37a-e996-4c5f-bb3a-3201a0481e74","Type":"ContainerStarted","Data":"2bd57e1dd0a704e8f8d45c1751d698ac23e50ab545fc688c56eff17c88bd9f6e"}
Jul 27 19:35:14.856431 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:14.856388    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/641fc784-7dcc-4cfc-9e4b-6f1b50f224dd-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kvgqm\" (UID: \"641fc784-7dcc-4cfc-9e4b-6f1b50f224dd\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kvgqm"
Jul 27 19:35:14.856571 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:14.856458    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qmh8\" (UniqueName: \"kubernetes.io/projected/641fc784-7dcc-4cfc-9e4b-6f1b50f224dd-kube-api-access-7qmh8\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kvgqm\" (UID: \"641fc784-7dcc-4cfc-9e4b-6f1b50f224dd\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kvgqm"
Jul 27 19:35:14.856571 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:14.856487    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/641fc784-7dcc-4cfc-9e4b-6f1b50f224dd-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kvgqm\" (UID: \"641fc784-7dcc-4cfc-9e4b-6f1b50f224dd\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kvgqm"
Jul 27 19:35:14.856849 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:14.856829    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/641fc784-7dcc-4cfc-9e4b-6f1b50f224dd-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kvgqm\" (UID: \"641fc784-7dcc-4cfc-9e4b-6f1b50f224dd\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kvgqm"
Jul 27 19:35:14.856919 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:14.856829    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/641fc784-7dcc-4cfc-9e4b-6f1b50f224dd-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kvgqm\" (UID: \"641fc784-7dcc-4cfc-9e4b-6f1b50f224dd\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kvgqm"
Jul 27 19:35:14.864274 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:14.864247    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qmh8\" (UniqueName: \"kubernetes.io/projected/641fc784-7dcc-4cfc-9e4b-6f1b50f224dd-kube-api-access-7qmh8\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kvgqm\" (UID: \"641fc784-7dcc-4cfc-9e4b-6f1b50f224dd\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kvgqm"
Jul 27 19:35:14.937436 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:14.937389    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kvgqm"
Jul 27 19:35:15.025310 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:15.025278    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732gpq7"]
Jul 27 19:35:15.028598 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:15.028576    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732gpq7"
Jul 27 19:35:15.038181 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:15.038116    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732gpq7"]
Jul 27 19:35:15.059120 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:15.059087    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5fa21c1c-c090-438b-a17a-eb72ab2c48a4-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732gpq7\" (UID: \"5fa21c1c-c090-438b-a17a-eb72ab2c48a4\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732gpq7"
Jul 27 19:35:15.059270 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:15.059170    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwgh6\" (UniqueName: \"kubernetes.io/projected/5fa21c1c-c090-438b-a17a-eb72ab2c48a4-kube-api-access-dwgh6\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732gpq7\" (UID: \"5fa21c1c-c090-438b-a17a-eb72ab2c48a4\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732gpq7"
Jul 27 19:35:15.059270 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:15.059190    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5fa21c1c-c090-438b-a17a-eb72ab2c48a4-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732gpq7\" (UID: \"5fa21c1c-c090-438b-a17a-eb72ab2c48a4\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732gpq7"
Jul 27 19:35:15.071215 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:15.071186    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kvgqm"]
Jul 27 19:35:15.074009 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:35:15.073984    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod641fc784_7dcc_4cfc_9e4b_6f1b50f224dd.slice/crio-523b6e04518821b378a538523ab94916701ab5f92fbc482d7eb1444443f847ff WatchSource:0}: Error finding container 523b6e04518821b378a538523ab94916701ab5f92fbc482d7eb1444443f847ff: Status 404 returned error can't find the container with id 523b6e04518821b378a538523ab94916701ab5f92fbc482d7eb1444443f847ff
Jul 27 19:35:15.160229 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:15.160140    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwgh6\" (UniqueName: \"kubernetes.io/projected/5fa21c1c-c090-438b-a17a-eb72ab2c48a4-kube-api-access-dwgh6\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732gpq7\" (UID: \"5fa21c1c-c090-438b-a17a-eb72ab2c48a4\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732gpq7"
Jul 27 19:35:15.160229 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:15.160174    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5fa21c1c-c090-438b-a17a-eb72ab2c48a4-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732gpq7\" (UID: \"5fa21c1c-c090-438b-a17a-eb72ab2c48a4\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732gpq7"
Jul 27 19:35:15.160371 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:15.160305    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5fa21c1c-c090-438b-a17a-eb72ab2c48a4-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732gpq7\" (UID: \"5fa21c1c-c090-438b-a17a-eb72ab2c48a4\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732gpq7"
Jul 27 19:35:15.160652 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:15.160601    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5fa21c1c-c090-438b-a17a-eb72ab2c48a4-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732gpq7\" (UID: \"5fa21c1c-c090-438b-a17a-eb72ab2c48a4\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732gpq7"
Jul 27 19:35:15.160798 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:15.160663    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5fa21c1c-c090-438b-a17a-eb72ab2c48a4-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732gpq7\" (UID: \"5fa21c1c-c090-438b-a17a-eb72ab2c48a4\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732gpq7"
Jul 27 19:35:15.168095 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:15.168063    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwgh6\" (UniqueName: \"kubernetes.io/projected/5fa21c1c-c090-438b-a17a-eb72ab2c48a4-kube-api-access-dwgh6\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732gpq7\" (UID: \"5fa21c1c-c090-438b-a17a-eb72ab2c48a4\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732gpq7"
Jul 27 19:35:15.339003 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:15.338978    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732gpq7"
Jul 27 19:35:15.470241 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:15.470183    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732gpq7"]
Jul 27 19:35:15.472365 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:35:15.472338    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fa21c1c_c090_438b_a17a_eb72ab2c48a4.slice/crio-a8fb102a8cfcbefe29359bf8003f80c425d4a9bd5c21126e91059ac3d98b3144 WatchSource:0}: Error finding container a8fb102a8cfcbefe29359bf8003f80c425d4a9bd5c21126e91059ac3d98b3144: Status 404 returned error can't find the container with id a8fb102a8cfcbefe29359bf8003f80c425d4a9bd5c21126e91059ac3d98b3144
Jul 27 19:35:15.815845 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:15.815730    2558 generic.go:356] "Generic (PLEG): container finished" podID="641fc784-7dcc-4cfc-9e4b-6f1b50f224dd" containerID="bb4cebd31b2ee44853a7508ddc400d158d2961097587ec573519dc2641d0ca14" exitCode=0
Jul 27 19:35:15.816006 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:15.815951    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kvgqm" event={"ID":"641fc784-7dcc-4cfc-9e4b-6f1b50f224dd","Type":"ContainerDied","Data":"bb4cebd31b2ee44853a7508ddc400d158d2961097587ec573519dc2641d0ca14"}
Jul 27 19:35:15.816006 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:15.816001    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kvgqm" event={"ID":"641fc784-7dcc-4cfc-9e4b-6f1b50f224dd","Type":"ContainerStarted","Data":"523b6e04518821b378a538523ab94916701ab5f92fbc482d7eb1444443f847ff"}
Jul 27 19:35:15.818239 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:15.818211    2558 generic.go:356] "Generic (PLEG): container finished" podID="4a03b37a-e996-4c5f-bb3a-3201a0481e74" containerID="25e25447c17f4bafd931b3d5545634b3298610ea1d28dcc50af8a3d1f8a105ce" exitCode=0
Jul 27 19:35:15.818365 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:15.818299    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cvkh7" event={"ID":"4a03b37a-e996-4c5f-bb3a-3201a0481e74","Type":"ContainerDied","Data":"25e25447c17f4bafd931b3d5545634b3298610ea1d28dcc50af8a3d1f8a105ce"}
Jul 27 19:35:15.820665 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:15.820630    2558 generic.go:356] "Generic (PLEG): container finished" podID="2cf47ec3-ce9f-4a23-9a8a-b45415b64482" containerID="bbac25f7223378b6f7ca59317ff564e6f3092b2c4aa99f283c6cbe0c8c0a6eae" exitCode=0
Jul 27 19:35:15.820759 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:15.820716    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qrh56" event={"ID":"2cf47ec3-ce9f-4a23-9a8a-b45415b64482","Type":"ContainerDied","Data":"bbac25f7223378b6f7ca59317ff564e6f3092b2c4aa99f283c6cbe0c8c0a6eae"}
Jul 27 19:35:15.822156 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:15.822092    2558 generic.go:356] "Generic (PLEG): container finished" podID="5fa21c1c-c090-438b-a17a-eb72ab2c48a4" containerID="81f0d3b9e808a7a4e2ade672179999a274895d8921ddfd8cb30f190861a81c35" exitCode=0
Jul 27 19:35:15.822156 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:15.822134    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732gpq7" event={"ID":"5fa21c1c-c090-438b-a17a-eb72ab2c48a4","Type":"ContainerDied","Data":"81f0d3b9e808a7a4e2ade672179999a274895d8921ddfd8cb30f190861a81c35"}
Jul 27 19:35:15.822287 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:15.822172    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732gpq7" event={"ID":"5fa21c1c-c090-438b-a17a-eb72ab2c48a4","Type":"ContainerStarted","Data":"a8fb102a8cfcbefe29359bf8003f80c425d4a9bd5c21126e91059ac3d98b3144"}
Jul 27 19:35:16.829285 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:16.829196    2558 generic.go:356] "Generic (PLEG): container finished" podID="4a03b37a-e996-4c5f-bb3a-3201a0481e74" containerID="ea13190b723302dab2065b3180ff3440e52d3989d9146adcf2bec8303520e159" exitCode=0
Jul 27 19:35:16.829727 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:16.829287    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cvkh7" event={"ID":"4a03b37a-e996-4c5f-bb3a-3201a0481e74","Type":"ContainerDied","Data":"ea13190b723302dab2065b3180ff3440e52d3989d9146adcf2bec8303520e159"}
Jul 27 19:35:16.836722 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:16.836695    2558 generic.go:356] "Generic (PLEG): container finished" podID="5fa21c1c-c090-438b-a17a-eb72ab2c48a4" containerID="4c9fdfc76554b08342a5e732510ecaa5fa16c95a9d37f89bbc0d2e80dfdc1dbe" exitCode=0
Jul 27 19:35:16.836877 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:16.836767    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732gpq7" event={"ID":"5fa21c1c-c090-438b-a17a-eb72ab2c48a4","Type":"ContainerDied","Data":"4c9fdfc76554b08342a5e732510ecaa5fa16c95a9d37f89bbc0d2e80dfdc1dbe"}
Jul 27 19:35:16.838854 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:16.838826    2558 generic.go:356] "Generic (PLEG): container finished" podID="641fc784-7dcc-4cfc-9e4b-6f1b50f224dd" containerID="97d8b8e73787d2f562942b5e881ca8978d3f46671c9fd61c421f2cfa75d37ae0" exitCode=0
Jul 27 19:35:16.838963 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:16.838903    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kvgqm" event={"ID":"641fc784-7dcc-4cfc-9e4b-6f1b50f224dd","Type":"ContainerDied","Data":"97d8b8e73787d2f562942b5e881ca8978d3f46671c9fd61c421f2cfa75d37ae0"}
Jul 27 19:35:16.970451 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:16.970385    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qrh56"
Jul 27 19:35:17.080168 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:17.080097    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4547r\" (UniqueName: \"kubernetes.io/projected/2cf47ec3-ce9f-4a23-9a8a-b45415b64482-kube-api-access-4547r\") pod \"2cf47ec3-ce9f-4a23-9a8a-b45415b64482\" (UID: \"2cf47ec3-ce9f-4a23-9a8a-b45415b64482\") "
Jul 27 19:35:17.080168 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:17.080142    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2cf47ec3-ce9f-4a23-9a8a-b45415b64482-bundle\") pod \"2cf47ec3-ce9f-4a23-9a8a-b45415b64482\" (UID: \"2cf47ec3-ce9f-4a23-9a8a-b45415b64482\") "
Jul 27 19:35:17.080331 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:17.080173    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2cf47ec3-ce9f-4a23-9a8a-b45415b64482-util\") pod \"2cf47ec3-ce9f-4a23-9a8a-b45415b64482\" (UID: \"2cf47ec3-ce9f-4a23-9a8a-b45415b64482\") "
Jul 27 19:35:17.080691 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:17.080666    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2cf47ec3-ce9f-4a23-9a8a-b45415b64482-bundle" (OuterVolumeSpecName: "bundle") pod "2cf47ec3-ce9f-4a23-9a8a-b45415b64482" (UID: "2cf47ec3-ce9f-4a23-9a8a-b45415b64482"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:35:17.082342 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:17.082319    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2cf47ec3-ce9f-4a23-9a8a-b45415b64482-kube-api-access-4547r" (OuterVolumeSpecName: "kube-api-access-4547r") pod "2cf47ec3-ce9f-4a23-9a8a-b45415b64482" (UID: "2cf47ec3-ce9f-4a23-9a8a-b45415b64482"). InnerVolumeSpecName "kube-api-access-4547r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:35:17.085562 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:17.085541    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2cf47ec3-ce9f-4a23-9a8a-b45415b64482-util" (OuterVolumeSpecName: "util") pod "2cf47ec3-ce9f-4a23-9a8a-b45415b64482" (UID: "2cf47ec3-ce9f-4a23-9a8a-b45415b64482"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:35:17.181383 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:17.181356    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4547r\" (UniqueName: \"kubernetes.io/projected/2cf47ec3-ce9f-4a23-9a8a-b45415b64482-kube-api-access-4547r\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:35:17.181383 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:17.181380    2558 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2cf47ec3-ce9f-4a23-9a8a-b45415b64482-bundle\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:35:17.181383 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:17.181389    2558 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2cf47ec3-ce9f-4a23-9a8a-b45415b64482-util\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:35:17.846830 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:17.846737    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qrh56" event={"ID":"2cf47ec3-ce9f-4a23-9a8a-b45415b64482","Type":"ContainerDied","Data":"1a95358934d6e90f6e416f5716b3790e86ab6b2fc3c7b3fcd0223a89dad93b5c"}
Jul 27 19:35:17.846830 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:17.846756    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qrh56"
Jul 27 19:35:17.846830 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:17.846789    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1a95358934d6e90f6e416f5716b3790e86ab6b2fc3c7b3fcd0223a89dad93b5c"
Jul 27 19:35:17.849131 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:17.849093    2558 generic.go:356] "Generic (PLEG): container finished" podID="5fa21c1c-c090-438b-a17a-eb72ab2c48a4" containerID="4fdad47a94c576e5872bcfc019317698878098e3f0adad501089178080b16760" exitCode=0
Jul 27 19:35:17.849249 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:17.849174    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732gpq7" event={"ID":"5fa21c1c-c090-438b-a17a-eb72ab2c48a4","Type":"ContainerDied","Data":"4fdad47a94c576e5872bcfc019317698878098e3f0adad501089178080b16760"}
Jul 27 19:35:17.851292 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:17.851271    2558 generic.go:356] "Generic (PLEG): container finished" podID="641fc784-7dcc-4cfc-9e4b-6f1b50f224dd" containerID="a1fe32d109ff80841bb9b15675e139015c54c8f30d26372ff729986f5a177aed" exitCode=0
Jul 27 19:35:17.851396 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:17.851352    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kvgqm" event={"ID":"641fc784-7dcc-4cfc-9e4b-6f1b50f224dd","Type":"ContainerDied","Data":"a1fe32d109ff80841bb9b15675e139015c54c8f30d26372ff729986f5a177aed"}
Jul 27 19:35:17.974987 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:17.974968    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cvkh7"
Jul 27 19:35:18.088981 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:18.088946    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4a03b37a-e996-4c5f-bb3a-3201a0481e74-util\") pod \"4a03b37a-e996-4c5f-bb3a-3201a0481e74\" (UID: \"4a03b37a-e996-4c5f-bb3a-3201a0481e74\") "
Jul 27 19:35:18.088981 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:18.088983    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4a03b37a-e996-4c5f-bb3a-3201a0481e74-bundle\") pod \"4a03b37a-e996-4c5f-bb3a-3201a0481e74\" (UID: \"4a03b37a-e996-4c5f-bb3a-3201a0481e74\") "
Jul 27 19:35:18.089211 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:18.089044    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s858s\" (UniqueName: \"kubernetes.io/projected/4a03b37a-e996-4c5f-bb3a-3201a0481e74-kube-api-access-s858s\") pod \"4a03b37a-e996-4c5f-bb3a-3201a0481e74\" (UID: \"4a03b37a-e996-4c5f-bb3a-3201a0481e74\") "
Jul 27 19:35:18.089513 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:18.089489    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a03b37a-e996-4c5f-bb3a-3201a0481e74-bundle" (OuterVolumeSpecName: "bundle") pod "4a03b37a-e996-4c5f-bb3a-3201a0481e74" (UID: "4a03b37a-e996-4c5f-bb3a-3201a0481e74"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:35:18.091207 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:18.091183    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a03b37a-e996-4c5f-bb3a-3201a0481e74-kube-api-access-s858s" (OuterVolumeSpecName: "kube-api-access-s858s") pod "4a03b37a-e996-4c5f-bb3a-3201a0481e74" (UID: "4a03b37a-e996-4c5f-bb3a-3201a0481e74"). InnerVolumeSpecName "kube-api-access-s858s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:35:18.094320 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:18.094284    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a03b37a-e996-4c5f-bb3a-3201a0481e74-util" (OuterVolumeSpecName: "util") pod "4a03b37a-e996-4c5f-bb3a-3201a0481e74" (UID: "4a03b37a-e996-4c5f-bb3a-3201a0481e74"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:35:18.189767 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:18.189737    2558 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4a03b37a-e996-4c5f-bb3a-3201a0481e74-bundle\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:35:18.189767 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:18.189764    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s858s\" (UniqueName: \"kubernetes.io/projected/4a03b37a-e996-4c5f-bb3a-3201a0481e74-kube-api-access-s858s\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:35:18.189927 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:18.189778    2558 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4a03b37a-e996-4c5f-bb3a-3201a0481e74-util\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:35:18.858974 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:18.858949    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cvkh7"
Jul 27 19:35:18.859403 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:18.858985    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cvkh7" event={"ID":"4a03b37a-e996-4c5f-bb3a-3201a0481e74","Type":"ContainerDied","Data":"2bd57e1dd0a704e8f8d45c1751d698ac23e50ab545fc688c56eff17c88bd9f6e"}
Jul 27 19:35:18.859403 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:18.859037    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2bd57e1dd0a704e8f8d45c1751d698ac23e50ab545fc688c56eff17c88bd9f6e"
Jul 27 19:35:18.992274 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:18.992251    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732gpq7"
Jul 27 19:35:19.030942 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:19.030917    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kvgqm"
Jul 27 19:35:19.097168 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:19.097142    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/641fc784-7dcc-4cfc-9e4b-6f1b50f224dd-bundle\") pod \"641fc784-7dcc-4cfc-9e4b-6f1b50f224dd\" (UID: \"641fc784-7dcc-4cfc-9e4b-6f1b50f224dd\") "
Jul 27 19:35:19.097168 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:19.097171    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5fa21c1c-c090-438b-a17a-eb72ab2c48a4-util\") pod \"5fa21c1c-c090-438b-a17a-eb72ab2c48a4\" (UID: \"5fa21c1c-c090-438b-a17a-eb72ab2c48a4\") "
Jul 27 19:35:19.097334 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:19.097225    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7qmh8\" (UniqueName: \"kubernetes.io/projected/641fc784-7dcc-4cfc-9e4b-6f1b50f224dd-kube-api-access-7qmh8\") pod \"641fc784-7dcc-4cfc-9e4b-6f1b50f224dd\" (UID: \"641fc784-7dcc-4cfc-9e4b-6f1b50f224dd\") "
Jul 27 19:35:19.097334 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:19.097246    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/641fc784-7dcc-4cfc-9e4b-6f1b50f224dd-util\") pod \"641fc784-7dcc-4cfc-9e4b-6f1b50f224dd\" (UID: \"641fc784-7dcc-4cfc-9e4b-6f1b50f224dd\") "
Jul 27 19:35:19.097334 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:19.097260    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dwgh6\" (UniqueName: \"kubernetes.io/projected/5fa21c1c-c090-438b-a17a-eb72ab2c48a4-kube-api-access-dwgh6\") pod \"5fa21c1c-c090-438b-a17a-eb72ab2c48a4\" (UID: \"5fa21c1c-c090-438b-a17a-eb72ab2c48a4\") "
Jul 27 19:35:19.097334 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:19.097278    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5fa21c1c-c090-438b-a17a-eb72ab2c48a4-bundle\") pod \"5fa21c1c-c090-438b-a17a-eb72ab2c48a4\" (UID: \"5fa21c1c-c090-438b-a17a-eb72ab2c48a4\") "
Jul 27 19:35:19.097959 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:19.097747    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/641fc784-7dcc-4cfc-9e4b-6f1b50f224dd-bundle" (OuterVolumeSpecName: "bundle") pod "641fc784-7dcc-4cfc-9e4b-6f1b50f224dd" (UID: "641fc784-7dcc-4cfc-9e4b-6f1b50f224dd"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:35:19.098084 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:19.098043    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fa21c1c-c090-438b-a17a-eb72ab2c48a4-bundle" (OuterVolumeSpecName: "bundle") pod "5fa21c1c-c090-438b-a17a-eb72ab2c48a4" (UID: "5fa21c1c-c090-438b-a17a-eb72ab2c48a4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:35:19.099636 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:19.099609    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5fa21c1c-c090-438b-a17a-eb72ab2c48a4-kube-api-access-dwgh6" (OuterVolumeSpecName: "kube-api-access-dwgh6") pod "5fa21c1c-c090-438b-a17a-eb72ab2c48a4" (UID: "5fa21c1c-c090-438b-a17a-eb72ab2c48a4"). InnerVolumeSpecName "kube-api-access-dwgh6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:35:19.099807 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:19.099787    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/641fc784-7dcc-4cfc-9e4b-6f1b50f224dd-kube-api-access-7qmh8" (OuterVolumeSpecName: "kube-api-access-7qmh8") pod "641fc784-7dcc-4cfc-9e4b-6f1b50f224dd" (UID: "641fc784-7dcc-4cfc-9e4b-6f1b50f224dd"). InnerVolumeSpecName "kube-api-access-7qmh8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:35:19.103110 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:19.103082    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fa21c1c-c090-438b-a17a-eb72ab2c48a4-util" (OuterVolumeSpecName: "util") pod "5fa21c1c-c090-438b-a17a-eb72ab2c48a4" (UID: "5fa21c1c-c090-438b-a17a-eb72ab2c48a4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:35:19.103424 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:19.103389    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/641fc784-7dcc-4cfc-9e4b-6f1b50f224dd-util" (OuterVolumeSpecName: "util") pod "641fc784-7dcc-4cfc-9e4b-6f1b50f224dd" (UID: "641fc784-7dcc-4cfc-9e4b-6f1b50f224dd"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:35:19.198662 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:19.198627    2558 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/641fc784-7dcc-4cfc-9e4b-6f1b50f224dd-bundle\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:35:19.198662 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:19.198660    2558 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5fa21c1c-c090-438b-a17a-eb72ab2c48a4-util\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:35:19.198662 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:19.198670    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7qmh8\" (UniqueName: \"kubernetes.io/projected/641fc784-7dcc-4cfc-9e4b-6f1b50f224dd-kube-api-access-7qmh8\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:35:19.198860 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:19.198678    2558 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/641fc784-7dcc-4cfc-9e4b-6f1b50f224dd-util\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:35:19.198860 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:19.198687    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dwgh6\" (UniqueName: \"kubernetes.io/projected/5fa21c1c-c090-438b-a17a-eb72ab2c48a4-kube-api-access-dwgh6\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:35:19.198860 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:19.198695    2558 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5fa21c1c-c090-438b-a17a-eb72ab2c48a4-bundle\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:35:19.865488 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:19.865463    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732gpq7"
Jul 27 19:35:19.865899 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:19.865456    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732gpq7" event={"ID":"5fa21c1c-c090-438b-a17a-eb72ab2c48a4","Type":"ContainerDied","Data":"a8fb102a8cfcbefe29359bf8003f80c425d4a9bd5c21126e91059ac3d98b3144"}
Jul 27 19:35:19.865899 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:19.865580    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a8fb102a8cfcbefe29359bf8003f80c425d4a9bd5c21126e91059ac3d98b3144"
Jul 27 19:35:19.867489 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:19.867464    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kvgqm"
Jul 27 19:35:19.867656 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:19.867462    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kvgqm" event={"ID":"641fc784-7dcc-4cfc-9e4b-6f1b50f224dd","Type":"ContainerDied","Data":"523b6e04518821b378a538523ab94916701ab5f92fbc482d7eb1444443f847ff"}
Jul 27 19:35:19.867769 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:19.867674    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="523b6e04518821b378a538523ab94916701ab5f92fbc482d7eb1444443f847ff"
Jul 27 19:35:20.687351 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:20.687303    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:35:23.492773 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:23.492738    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:35:28.695540 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:28.695509    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:35:29.312560 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:29.312519    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:35:29.753685 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:29.753656    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:35:32.894947 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:32.894916    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:35:33.924931 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:33.924897    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:35:34.034197 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:34.034162    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:35:34.946791 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:34.946763    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-5ql7p"]
Jul 27 19:35:34.947185 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:34.947130    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a03b37a-e996-4c5f-bb3a-3201a0481e74" containerName="util"
Jul 27 19:35:34.947185 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:34.947143    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a03b37a-e996-4c5f-bb3a-3201a0481e74" containerName="util"
Jul 27 19:35:34.947185 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:34.947157    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2cf47ec3-ce9f-4a23-9a8a-b45415b64482" containerName="extract"
Jul 27 19:35:34.947185 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:34.947163    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="2cf47ec3-ce9f-4a23-9a8a-b45415b64482" containerName="extract"
Jul 27 19:35:34.947185 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:34.947170    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5fa21c1c-c090-438b-a17a-eb72ab2c48a4" containerName="pull"
Jul 27 19:35:34.947185 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:34.947175    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="5fa21c1c-c090-438b-a17a-eb72ab2c48a4" containerName="pull"
Jul 27 19:35:34.947185 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:34.947182    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5fa21c1c-c090-438b-a17a-eb72ab2c48a4" containerName="util"
Jul 27 19:35:34.947185 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:34.947188    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="5fa21c1c-c090-438b-a17a-eb72ab2c48a4" containerName="util"
Jul 27 19:35:34.947581 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:34.947200    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2cf47ec3-ce9f-4a23-9a8a-b45415b64482" containerName="util"
Jul 27 19:35:34.947581 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:34.947208    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="2cf47ec3-ce9f-4a23-9a8a-b45415b64482" containerName="util"
Jul 27 19:35:34.947581 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:34.947216    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a03b37a-e996-4c5f-bb3a-3201a0481e74" containerName="pull"
Jul 27 19:35:34.947581 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:34.947222    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a03b37a-e996-4c5f-bb3a-3201a0481e74" containerName="pull"
Jul 27 19:35:34.947581 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:34.947232    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5fa21c1c-c090-438b-a17a-eb72ab2c48a4" containerName="extract"
Jul 27 19:35:34.947581 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:34.947237    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="5fa21c1c-c090-438b-a17a-eb72ab2c48a4" containerName="extract"
Jul 27 19:35:34.947581 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:34.947243    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="641fc784-7dcc-4cfc-9e4b-6f1b50f224dd" containerName="extract"
Jul 27 19:35:34.947581 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:34.947248    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="641fc784-7dcc-4cfc-9e4b-6f1b50f224dd" containerName="extract"
Jul 27 19:35:34.947581 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:34.947253    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2cf47ec3-ce9f-4a23-9a8a-b45415b64482" containerName="pull"
Jul 27 19:35:34.947581 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:34.947258    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="2cf47ec3-ce9f-4a23-9a8a-b45415b64482" containerName="pull"
Jul 27 19:35:34.947581 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:34.947266    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="641fc784-7dcc-4cfc-9e4b-6f1b50f224dd" containerName="pull"
Jul 27 19:35:34.947581 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:34.947272    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="641fc784-7dcc-4cfc-9e4b-6f1b50f224dd" containerName="pull"
Jul 27 19:35:34.947581 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:34.947283    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="641fc784-7dcc-4cfc-9e4b-6f1b50f224dd" containerName="util"
Jul 27 19:35:34.947581 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:34.947291    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="641fc784-7dcc-4cfc-9e4b-6f1b50f224dd" containerName="util"
Jul 27 19:35:34.947581 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:34.947299    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a03b37a-e996-4c5f-bb3a-3201a0481e74" containerName="extract"
Jul 27 19:35:34.947581 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:34.947304    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a03b37a-e996-4c5f-bb3a-3201a0481e74" containerName="extract"
Jul 27 19:35:34.947581 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:34.947358    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="641fc784-7dcc-4cfc-9e4b-6f1b50f224dd" containerName="extract"
Jul 27 19:35:34.947581 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:34.947368    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="2cf47ec3-ce9f-4a23-9a8a-b45415b64482" containerName="extract"
Jul 27 19:35:34.947581 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:34.947379    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a03b37a-e996-4c5f-bb3a-3201a0481e74" containerName="extract"
Jul 27 19:35:34.947581 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:34.947390    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="5fa21c1c-c090-438b-a17a-eb72ab2c48a4" containerName="extract"
Jul 27 19:35:34.950230 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:34.950215    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5ql7p"
Jul 27 19:35:34.952776 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:34.952756    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 27 19:35:34.953583 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:34.953555    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-x866b\""
Jul 27 19:35:34.953583 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:34.953565    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 27 19:35:34.953737 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:34.953560    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 27 19:35:34.960366 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:34.960346    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-5ql7p"]
Jul 27 19:35:35.032467 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:35.032433    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54hth\" (UniqueName: \"kubernetes.io/projected/bbda13de-368c-468d-ab0d-bee187a9f1e6-kube-api-access-54hth\") pod \"dns-operator-controller-manager-864dd78c4c-5ql7p\" (UID: \"bbda13de-368c-468d-ab0d-bee187a9f1e6\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5ql7p"
Jul 27 19:35:35.133534 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:35.133498    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54hth\" (UniqueName: \"kubernetes.io/projected/bbda13de-368c-468d-ab0d-bee187a9f1e6-kube-api-access-54hth\") pod \"dns-operator-controller-manager-864dd78c4c-5ql7p\" (UID: \"bbda13de-368c-468d-ab0d-bee187a9f1e6\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5ql7p"
Jul 27 19:35:35.148984 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:35.148947    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-54hth\" (UniqueName: \"kubernetes.io/projected/bbda13de-368c-468d-ab0d-bee187a9f1e6-kube-api-access-54hth\") pod \"dns-operator-controller-manager-864dd78c4c-5ql7p\" (UID: \"bbda13de-368c-468d-ab0d-bee187a9f1e6\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5ql7p"
Jul 27 19:35:35.152825 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:35.152799    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-h55r8"]
Jul 27 19:35:35.155682 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:35.155663    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-h55r8"
Jul 27 19:35:35.158465 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:35.158444    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-d5plc\""
Jul 27 19:35:35.159382 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:35.159363    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 27 19:35:35.159595 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:35.159581    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 27 19:35:35.168207 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:35.168188    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-h55r8"]
Jul 27 19:35:35.181603 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:35.181581    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:35:35.261432 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:35.261328    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5ql7p"
Jul 27 19:35:35.335586 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:35.335554    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zq54n\" (UniqueName: \"kubernetes.io/projected/b036279b-e3e1-48fb-ab19-c6d96d41a423-kube-api-access-zq54n\") pod \"kuadrant-console-plugin-66b5774985-h55r8\" (UID: \"b036279b-e3e1-48fb-ab19-c6d96d41a423\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-h55r8"
Jul 27 19:35:35.335756 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:35.335592    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b036279b-e3e1-48fb-ab19-c6d96d41a423-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-h55r8\" (UID: \"b036279b-e3e1-48fb-ab19-c6d96d41a423\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-h55r8"
Jul 27 19:35:35.335756 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:35.335635    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/b036279b-e3e1-48fb-ab19-c6d96d41a423-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-h55r8\" (UID: \"b036279b-e3e1-48fb-ab19-c6d96d41a423\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-h55r8"
Jul 27 19:35:35.398187 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:35.398163    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-5ql7p"]
Jul 27 19:35:35.400473 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:35:35.400399    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbda13de_368c_468d_ab0d_bee187a9f1e6.slice/crio-3b3bf4e1e04092ee90255465bbca3b24c1d3470c863be09d52e2c3051fa71baa WatchSource:0}: Error finding container 3b3bf4e1e04092ee90255465bbca3b24c1d3470c863be09d52e2c3051fa71baa: Status 404 returned error can't find the container with id 3b3bf4e1e04092ee90255465bbca3b24c1d3470c863be09d52e2c3051fa71baa
Jul 27 19:35:35.436767 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:35.436736    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zq54n\" (UniqueName: \"kubernetes.io/projected/b036279b-e3e1-48fb-ab19-c6d96d41a423-kube-api-access-zq54n\") pod \"kuadrant-console-plugin-66b5774985-h55r8\" (UID: \"b036279b-e3e1-48fb-ab19-c6d96d41a423\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-h55r8"
Jul 27 19:35:35.436895 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:35.436771    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b036279b-e3e1-48fb-ab19-c6d96d41a423-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-h55r8\" (UID: \"b036279b-e3e1-48fb-ab19-c6d96d41a423\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-h55r8"
Jul 27 19:35:35.436895 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:35.436814    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/b036279b-e3e1-48fb-ab19-c6d96d41a423-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-h55r8\" (UID: \"b036279b-e3e1-48fb-ab19-c6d96d41a423\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-h55r8"
Jul 27 19:35:35.436982 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:35:35.436945    2558 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Jul 27 19:35:35.437022 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:35:35.437007    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b036279b-e3e1-48fb-ab19-c6d96d41a423-plugin-serving-cert podName:b036279b-e3e1-48fb-ab19-c6d96d41a423 nodeName:}" failed. No retries permitted until 2026-07-27 19:35:35.936987593 +0000 UTC m=+548.809179560 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/b036279b-e3e1-48fb-ab19-c6d96d41a423-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-h55r8" (UID: "b036279b-e3e1-48fb-ab19-c6d96d41a423") : secret "plugin-serving-cert" not found
Jul 27 19:35:35.437380 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:35.437363    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b036279b-e3e1-48fb-ab19-c6d96d41a423-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-h55r8\" (UID: \"b036279b-e3e1-48fb-ab19-c6d96d41a423\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-h55r8"
Jul 27 19:35:35.444965 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:35.444946    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zq54n\" (UniqueName: \"kubernetes.io/projected/b036279b-e3e1-48fb-ab19-c6d96d41a423-kube-api-access-zq54n\") pod \"kuadrant-console-plugin-66b5774985-h55r8\" (UID: \"b036279b-e3e1-48fb-ab19-c6d96d41a423\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-h55r8"
Jul 27 19:35:35.521892 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:35.521820    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:35:35.759345 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:35.759314    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:35:35.942781 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:35.942742    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/b036279b-e3e1-48fb-ab19-c6d96d41a423-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-h55r8\" (UID: \"b036279b-e3e1-48fb-ab19-c6d96d41a423\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-h55r8"
Jul 27 19:35:35.945802 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:35.945778    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/b036279b-e3e1-48fb-ab19-c6d96d41a423-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-h55r8\" (UID: \"b036279b-e3e1-48fb-ab19-c6d96d41a423\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-h55r8"
Jul 27 19:35:35.950998 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:35.950971    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5ql7p" event={"ID":"bbda13de-368c-468d-ab0d-bee187a9f1e6","Type":"ContainerStarted","Data":"3b3bf4e1e04092ee90255465bbca3b24c1d3470c863be09d52e2c3051fa71baa"}
Jul 27 19:35:36.067779 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:36.067730    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-h55r8"
Jul 27 19:35:36.215492 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:36.215468    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-h55r8"]
Jul 27 19:35:36.217605 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:35:36.217575    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb036279b_e3e1_48fb_ab19_c6d96d41a423.slice/crio-727ce7481b5af6e7918b329afca2fa107a474537dc117c2b95d696878cc0800e WatchSource:0}: Error finding container 727ce7481b5af6e7918b329afca2fa107a474537dc117c2b95d696878cc0800e: Status 404 returned error can't find the container with id 727ce7481b5af6e7918b329afca2fa107a474537dc117c2b95d696878cc0800e
Jul 27 19:35:36.959330 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:36.959292    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-h55r8" event={"ID":"b036279b-e3e1-48fb-ab19-c6d96d41a423","Type":"ContainerStarted","Data":"727ce7481b5af6e7918b329afca2fa107a474537dc117c2b95d696878cc0800e"}
Jul 27 19:35:37.969366 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:37.969326    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5ql7p" event={"ID":"bbda13de-368c-468d-ab0d-bee187a9f1e6","Type":"ContainerStarted","Data":"ec3ad1e3eea0d6aeae8cd71f53a544c05977314275b644150b5dcb48f52b3941"}
Jul 27 19:35:37.969829 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:37.969472    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5ql7p"
Jul 27 19:35:37.993936 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:37.993869    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5ql7p" podStartSLOduration=1.812073733 podStartE2EDuration="3.99384703s" totalImagesPullingTime="2.181773297s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:35:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:35:37.99239975 +0000 UTC m=+550.864591736" watchObservedRunningTime="2026-07-27 19:35:37.99384703 +0000 UTC m=+550.866039005"
Jul 27 19:35:38.509854 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:38.509811    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:35:40.701868 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:40.701832    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:35:40.802587 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:40.802554    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:35:47.105470 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:47.105432    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kg2ph"]
Jul 27 19:35:47.154859 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:47.154825    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kg2ph"]
Jul 27 19:35:47.155057 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:47.154959    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kg2ph"
Jul 27 19:35:47.157617 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:47.157593    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-htjbz\""
Jul 27 19:35:47.246615 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:47.246581    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/09106494-3ae8-4b87-a583-ec42a938d603-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-kg2ph\" (UID: \"09106494-3ae8-4b87-a583-ec42a938d603\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kg2ph"
Jul 27 19:35:47.246789 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:47.246655    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wk4l\" (UniqueName: \"kubernetes.io/projected/09106494-3ae8-4b87-a583-ec42a938d603-kube-api-access-2wk4l\") pod \"kuadrant-operator-controller-manager-86878c769f-kg2ph\" (UID: \"09106494-3ae8-4b87-a583-ec42a938d603\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kg2ph"
Jul 27 19:35:47.347353 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:47.347311    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/09106494-3ae8-4b87-a583-ec42a938d603-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-kg2ph\" (UID: \"09106494-3ae8-4b87-a583-ec42a938d603\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kg2ph"
Jul 27 19:35:47.347551 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:47.347395    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wk4l\" (UniqueName: \"kubernetes.io/projected/09106494-3ae8-4b87-a583-ec42a938d603-kube-api-access-2wk4l\") pod \"kuadrant-operator-controller-manager-86878c769f-kg2ph\" (UID: \"09106494-3ae8-4b87-a583-ec42a938d603\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kg2ph"
Jul 27 19:35:47.347703 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:47.347680    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/09106494-3ae8-4b87-a583-ec42a938d603-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-kg2ph\" (UID: \"09106494-3ae8-4b87-a583-ec42a938d603\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kg2ph"
Jul 27 19:35:47.359429 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:47.359348    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wk4l\" (UniqueName: \"kubernetes.io/projected/09106494-3ae8-4b87-a583-ec42a938d603-kube-api-access-2wk4l\") pod \"kuadrant-operator-controller-manager-86878c769f-kg2ph\" (UID: \"09106494-3ae8-4b87-a583-ec42a938d603\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kg2ph"
Jul 27 19:35:47.466550 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:47.466499    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kg2ph"
Jul 27 19:35:48.979255 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:48.979217    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5ql7p"
Jul 27 19:35:50.921817 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:50.921779    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:35:50.995204 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:50.995172    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:35:53.384814 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:53.384783    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:35:53.444088 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:53.444052    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:35:56.763657 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:56.763635    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kg2ph"]
Jul 27 19:35:56.767603 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:35:56.767575    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09106494_3ae8_4b87_a583_ec42a938d603.slice/crio-34f2acab7d223469b2e07f9dd8c488cd67e24b1b8690979457e6a9494a082761 WatchSource:0}: Error finding container 34f2acab7d223469b2e07f9dd8c488cd67e24b1b8690979457e6a9494a082761: Status 404 returned error can't find the container with id 34f2acab7d223469b2e07f9dd8c488cd67e24b1b8690979457e6a9494a082761
Jul 27 19:35:57.083141 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:57.083038    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-h55r8" event={"ID":"b036279b-e3e1-48fb-ab19-c6d96d41a423","Type":"ContainerStarted","Data":"86c4925b58b859b268b711834293f0f392241122fb03c5311fa879870c98d78f"}
Jul 27 19:35:57.084384 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:57.084358    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kg2ph" event={"ID":"09106494-3ae8-4b87-a583-ec42a938d603","Type":"ContainerStarted","Data":"34f2acab7d223469b2e07f9dd8c488cd67e24b1b8690979457e6a9494a082761"}
Jul 27 19:35:57.100194 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:57.100152    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-h55r8" podStartSLOduration=1.60294707 podStartE2EDuration="22.10013705s" totalImagesPullingTime="20.49718998s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:35:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:35:57.099390192 +0000 UTC m=+569.971582167" watchObservedRunningTime="2026-07-27 19:35:57.10013705 +0000 UTC m=+569.972329024"
Jul 27 19:35:57.712202 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:57.712164    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:35:57.826290 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:57.826255    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:35:58.110096 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:58.110011    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:35:58.205998 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:58.205959    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:35:59.921989 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:59.921936    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:35:59.988769 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:35:59.988734    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:36:01.113352 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:36:01.113321    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kg2ph" event={"ID":"09106494-3ae8-4b87-a583-ec42a938d603","Type":"ContainerStarted","Data":"d1bf1bfac8e3b5738089d2671860a181466323ce418d5bdbb1c616f7ea78f77d"}
Jul 27 19:36:01.113780 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:36:01.113441    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kg2ph"
Jul 27 19:36:01.132776 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:36:01.132709    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kg2ph" podStartSLOduration=10.195471383 podStartE2EDuration="14.13269266s" totalImagesPullingTime="3.937221277s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:35:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:36:01.131217579 +0000 UTC m=+574.003409554" watchObservedRunningTime="2026-07-27 19:36:01.13269266 +0000 UTC m=+574.004884631"
Jul 27 19:36:08.692406 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:36:08.692369    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:36:08.793198 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:36:08.793166    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:36:11.197588 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:36:11.197555    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:36:11.299376 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:36:11.299289    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:36:12.121404 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:36:12.121374    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kg2ph"
Jul 27 19:36:15.490713 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:36:15.490673    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:36:15.900645 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:36:15.900613    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:36:19.090778 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:36:19.090748    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:36:19.192025 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:36:19.191989    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:36:27.535304 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:36:27.535272    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5dfc2_103bab23-bca3-472a-974b-7731f83c5bfb/ovn-acl-logging/0.log"
Jul 27 19:36:27.536073 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:36:27.536030    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5dfc2_103bab23-bca3-472a-974b-7731f83c5bfb/ovn-acl-logging/0.log"
Jul 27 19:36:28.837529 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:36:28.837488    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:36:28.938432 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:36:28.938372    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:36:29.315120 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:36:29.315081    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:36:29.445667 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:36:29.445634    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:36:31.753016 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:36:31.752967    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:36:31.853123 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:36:31.853092    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:36:33.253362 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:36:33.253332    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:36:33.382975 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:36:33.382937    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:36:33.484902 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:36:33.484866    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:36:34.047428 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:36:34.047382    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-f7zbw"]
Jul 27 19:36:34.080595 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:36:34.080518    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-f7zbw"]
Jul 27 19:36:34.080595 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:36:34.080595    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-f7zbw"
Jul 27 19:36:34.083005 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:36:34.082983    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-rjpxs\""
Jul 27 19:36:34.086060 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:36:34.086025    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:36:34.161049 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:36:34.161005    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrxls\" (UniqueName: \"kubernetes.io/projected/be6a58a7-abc5-4d35-badb-ec697dc54d8b-kube-api-access-jrxls\") pod \"authorino-58cd955675-f7zbw\" (UID: \"be6a58a7-abc5-4d35-badb-ec697dc54d8b\") " pod="kuadrant-system/authorino-58cd955675-f7zbw"
Jul 27 19:36:34.186871 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:36:34.186834    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:36:34.261622 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:36:34.261583    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrxls\" (UniqueName: \"kubernetes.io/projected/be6a58a7-abc5-4d35-badb-ec697dc54d8b-kube-api-access-jrxls\") pod \"authorino-58cd955675-f7zbw\" (UID: \"be6a58a7-abc5-4d35-badb-ec697dc54d8b\") " pod="kuadrant-system/authorino-58cd955675-f7zbw"
Jul 27 19:36:34.270452 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:36:34.270394    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrxls\" (UniqueName: \"kubernetes.io/projected/be6a58a7-abc5-4d35-badb-ec697dc54d8b-kube-api-access-jrxls\") pod \"authorino-58cd955675-f7zbw\" (UID: \"be6a58a7-abc5-4d35-badb-ec697dc54d8b\") " pod="kuadrant-system/authorino-58cd955675-f7zbw"
Jul 27 19:36:34.390760 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:36:34.390718    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-f7zbw"
Jul 27 19:36:34.534963 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:36:34.534938    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-f7zbw"]
Jul 27 19:36:34.536978 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:36:34.536950    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbe6a58a7_abc5_4d35_badb_ec697dc54d8b.slice/crio-584d9d2e7e09a6afbe21b2b8d60d043d34989dee2cb179cc0e27464c61f1fd7c WatchSource:0}: Error finding container 584d9d2e7e09a6afbe21b2b8d60d043d34989dee2cb179cc0e27464c61f1fd7c: Status 404 returned error can't find the container with id 584d9d2e7e09a6afbe21b2b8d60d043d34989dee2cb179cc0e27464c61f1fd7c
Jul 27 19:36:35.306693 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:36:35.306650    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-f7zbw" event={"ID":"be6a58a7-abc5-4d35-badb-ec697dc54d8b","Type":"ContainerStarted","Data":"584d9d2e7e09a6afbe21b2b8d60d043d34989dee2cb179cc0e27464c61f1fd7c"}
Jul 27 19:36:38.325561 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:36:38.325518    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-f7zbw" event={"ID":"be6a58a7-abc5-4d35-badb-ec697dc54d8b","Type":"ContainerStarted","Data":"31453a3adf9c243f73b3d9d33e3b71208b250b52ec64868ba0635cc9d9067b23"}
Jul 27 19:36:38.341470 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:36:38.341401    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-f7zbw" podStartSLOduration=1.200016414 podStartE2EDuration="4.341382169s" totalImagesPullingTime="3.141365755s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:36:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:36:38.339334813 +0000 UTC m=+611.211526803" watchObservedRunningTime="2026-07-27 19:36:38.341382169 +0000 UTC m=+611.213574143"
Jul 27 19:36:39.351143 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:36:39.351105    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-f7zbw"]
Jul 27 19:36:40.336623 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:36:40.336580    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-f7zbw" podUID="be6a58a7-abc5-4d35-badb-ec697dc54d8b" containerName="authorino" containerID="cri-o://31453a3adf9c243f73b3d9d33e3b71208b250b52ec64868ba0635cc9d9067b23" gracePeriod=30
Jul 27 19:36:40.595747 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:36:40.595679    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-f7zbw"
Jul 27 19:36:40.609125 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:36:40.609093    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jrxls\" (UniqueName: \"kubernetes.io/projected/be6a58a7-abc5-4d35-badb-ec697dc54d8b-kube-api-access-jrxls\") pod \"be6a58a7-abc5-4d35-badb-ec697dc54d8b\" (UID: \"be6a58a7-abc5-4d35-badb-ec697dc54d8b\") "
Jul 27 19:36:40.611561 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:36:40.611506    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be6a58a7-abc5-4d35-badb-ec697dc54d8b-kube-api-access-jrxls" (OuterVolumeSpecName: "kube-api-access-jrxls") pod "be6a58a7-abc5-4d35-badb-ec697dc54d8b" (UID: "be6a58a7-abc5-4d35-badb-ec697dc54d8b"). InnerVolumeSpecName "kube-api-access-jrxls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:36:40.710044 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:36:40.710010    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jrxls\" (UniqueName: \"kubernetes.io/projected/be6a58a7-abc5-4d35-badb-ec697dc54d8b-kube-api-access-jrxls\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:36:41.343161 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:36:41.343125    2558 generic.go:356] "Generic (PLEG): container finished" podID="be6a58a7-abc5-4d35-badb-ec697dc54d8b" containerID="31453a3adf9c243f73b3d9d33e3b71208b250b52ec64868ba0635cc9d9067b23" exitCode=0
Jul 27 19:36:41.343348 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:36:41.343175    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-f7zbw"
Jul 27 19:36:41.343348 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:36:41.343215    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-f7zbw" event={"ID":"be6a58a7-abc5-4d35-badb-ec697dc54d8b","Type":"ContainerDied","Data":"31453a3adf9c243f73b3d9d33e3b71208b250b52ec64868ba0635cc9d9067b23"}
Jul 27 19:36:41.343348 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:36:41.343265    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-f7zbw" event={"ID":"be6a58a7-abc5-4d35-badb-ec697dc54d8b","Type":"ContainerDied","Data":"584d9d2e7e09a6afbe21b2b8d60d043d34989dee2cb179cc0e27464c61f1fd7c"}
Jul 27 19:36:41.343348 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:36:41.343287    2558 scope.go:117] "RemoveContainer" containerID="31453a3adf9c243f73b3d9d33e3b71208b250b52ec64868ba0635cc9d9067b23"
Jul 27 19:36:41.353020 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:36:41.352998    2558 scope.go:117] "RemoveContainer" containerID="31453a3adf9c243f73b3d9d33e3b71208b250b52ec64868ba0635cc9d9067b23"
Jul 27 19:36:41.353318 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:36:41.353296    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31453a3adf9c243f73b3d9d33e3b71208b250b52ec64868ba0635cc9d9067b23\": container with ID starting with 31453a3adf9c243f73b3d9d33e3b71208b250b52ec64868ba0635cc9d9067b23 not found: ID does not exist" containerID="31453a3adf9c243f73b3d9d33e3b71208b250b52ec64868ba0635cc9d9067b23"
Jul 27 19:36:41.353376 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:36:41.353328    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31453a3adf9c243f73b3d9d33e3b71208b250b52ec64868ba0635cc9d9067b23"} err="failed to get container status \"31453a3adf9c243f73b3d9d33e3b71208b250b52ec64868ba0635cc9d9067b23\": rpc error: code = NotFound desc = could not find container \"31453a3adf9c243f73b3d9d33e3b71208b250b52ec64868ba0635cc9d9067b23\": container with ID starting with 31453a3adf9c243f73b3d9d33e3b71208b250b52ec64868ba0635cc9d9067b23 not found: ID does not exist"
Jul 27 19:36:41.363938 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:36:41.363900    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-f7zbw"]
Jul 27 19:36:41.367483 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:36:41.367453    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-f7zbw"]
Jul 27 19:36:41.674519 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:36:41.674481    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="be6a58a7-abc5-4d35-badb-ec697dc54d8b" path="/var/lib/kubelet/pods/be6a58a7-abc5-4d35-badb-ec697dc54d8b/volumes"
Jul 27 19:36:44.669679 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:36:44.669646    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:36:54.579791 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:36:54.579697    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:36:54.684360 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:36:54.684328    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:36:55.079828 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:36:55.079799    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:37:07.757945 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:07.757909    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:37:07.971041 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:07.971010    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-8xf6l"]
Jul 27 19:37:07.971356 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:07.971345    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be6a58a7-abc5-4d35-badb-ec697dc54d8b" containerName="authorino"
Jul 27 19:37:07.971401 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:07.971358    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="be6a58a7-abc5-4d35-badb-ec697dc54d8b" containerName="authorino"
Jul 27 19:37:07.971455 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:07.971438    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="be6a58a7-abc5-4d35-badb-ec697dc54d8b" containerName="authorino"
Jul 27 19:37:07.973752 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:07.973732    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-8xf6l"
Jul 27 19:37:07.976053 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:07.976032    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-rjpxs\""
Jul 27 19:37:07.982956 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:07.982938    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-8xf6l"]
Jul 27 19:37:08.043515 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:08.043427    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtsbs\" (UniqueName: \"kubernetes.io/projected/527463d4-b05e-447d-9ef6-9c6a04e8fd1a-kube-api-access-rtsbs\") pod \"authorino-5d4b67c446-8xf6l\" (UID: \"527463d4-b05e-447d-9ef6-9c6a04e8fd1a\") " pod="kuadrant-system/authorino-5d4b67c446-8xf6l"
Jul 27 19:37:08.109820 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:08.109789    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:37:08.144191 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:08.144153    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtsbs\" (UniqueName: \"kubernetes.io/projected/527463d4-b05e-447d-9ef6-9c6a04e8fd1a-kube-api-access-rtsbs\") pod \"authorino-5d4b67c446-8xf6l\" (UID: \"527463d4-b05e-447d-9ef6-9c6a04e8fd1a\") " pod="kuadrant-system/authorino-5d4b67c446-8xf6l"
Jul 27 19:37:08.152123 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:08.152102    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtsbs\" (UniqueName: \"kubernetes.io/projected/527463d4-b05e-447d-9ef6-9c6a04e8fd1a-kube-api-access-rtsbs\") pod \"authorino-5d4b67c446-8xf6l\" (UID: \"527463d4-b05e-447d-9ef6-9c6a04e8fd1a\") " pod="kuadrant-system/authorino-5d4b67c446-8xf6l"
Jul 27 19:37:08.193912 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:08.193886    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-8xf6l"]
Jul 27 19:37:08.194141 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:08.194130    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-8xf6l"
Jul 27 19:37:08.326241 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:08.326215    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-8xf6l"]
Jul 27 19:37:08.328169 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:37:08.328142    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod527463d4_b05e_447d_9ef6_9c6a04e8fd1a.slice/crio-73e04900672d5768e86311def6767bb6b815460e86652622823e1d445ea9d109 WatchSource:0}: Error finding container 73e04900672d5768e86311def6767bb6b815460e86652622823e1d445ea9d109: Status 404 returned error can't find the container with id 73e04900672d5768e86311def6767bb6b815460e86652622823e1d445ea9d109
Jul 27 19:37:08.329429 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:08.329391    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 19:37:08.411266 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:08.411237    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6bf8cc55c-7k2lf"]
Jul 27 19:37:08.414509 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:08.414488    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6bf8cc55c-7k2lf"
Jul 27 19:37:08.417363 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:08.417343    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 27 19:37:08.424153 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:08.424129    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6bf8cc55c-7k2lf"]
Jul 27 19:37:08.482753 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:08.482711    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-8xf6l" event={"ID":"527463d4-b05e-447d-9ef6-9c6a04e8fd1a","Type":"ContainerStarted","Data":"73e04900672d5768e86311def6767bb6b815460e86652622823e1d445ea9d109"}
Jul 27 19:37:08.548084 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:08.548051    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fgn6\" (UniqueName: \"kubernetes.io/projected/d32b3fe7-2366-4120-b666-7b95f2d82b0f-kube-api-access-6fgn6\") pod \"authorino-6bf8cc55c-7k2lf\" (UID: \"d32b3fe7-2366-4120-b666-7b95f2d82b0f\") " pod="kuadrant-system/authorino-6bf8cc55c-7k2lf"
Jul 27 19:37:08.548254 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:08.548128    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d32b3fe7-2366-4120-b666-7b95f2d82b0f-tls-cert\") pod \"authorino-6bf8cc55c-7k2lf\" (UID: \"d32b3fe7-2366-4120-b666-7b95f2d82b0f\") " pod="kuadrant-system/authorino-6bf8cc55c-7k2lf"
Jul 27 19:37:08.649169 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:08.649139    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fgn6\" (UniqueName: \"kubernetes.io/projected/d32b3fe7-2366-4120-b666-7b95f2d82b0f-kube-api-access-6fgn6\") pod \"authorino-6bf8cc55c-7k2lf\" (UID: \"d32b3fe7-2366-4120-b666-7b95f2d82b0f\") " pod="kuadrant-system/authorino-6bf8cc55c-7k2lf"
Jul 27 19:37:08.649325 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:08.649203    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d32b3fe7-2366-4120-b666-7b95f2d82b0f-tls-cert\") pod \"authorino-6bf8cc55c-7k2lf\" (UID: \"d32b3fe7-2366-4120-b666-7b95f2d82b0f\") " pod="kuadrant-system/authorino-6bf8cc55c-7k2lf"
Jul 27 19:37:08.651758 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:08.651739    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d32b3fe7-2366-4120-b666-7b95f2d82b0f-tls-cert\") pod \"authorino-6bf8cc55c-7k2lf\" (UID: \"d32b3fe7-2366-4120-b666-7b95f2d82b0f\") " pod="kuadrant-system/authorino-6bf8cc55c-7k2lf"
Jul 27 19:37:08.657257 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:08.657229    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fgn6\" (UniqueName: \"kubernetes.io/projected/d32b3fe7-2366-4120-b666-7b95f2d82b0f-kube-api-access-6fgn6\") pod \"authorino-6bf8cc55c-7k2lf\" (UID: \"d32b3fe7-2366-4120-b666-7b95f2d82b0f\") " pod="kuadrant-system/authorino-6bf8cc55c-7k2lf"
Jul 27 19:37:08.724441 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:08.724399    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6bf8cc55c-7k2lf"
Jul 27 19:37:08.865642 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:08.865496    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6bf8cc55c-7k2lf"]
Jul 27 19:37:08.867670 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:37:08.867645    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd32b3fe7_2366_4120_b666_7b95f2d82b0f.slice/crio-0831b8d6c47369457e4916539e12d00f36eba8636eb9f8af8f647a0ea1464cf9 WatchSource:0}: Error finding container 0831b8d6c47369457e4916539e12d00f36eba8636eb9f8af8f647a0ea1464cf9: Status 404 returned error can't find the container with id 0831b8d6c47369457e4916539e12d00f36eba8636eb9f8af8f647a0ea1464cf9
Jul 27 19:37:09.489894 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:09.489795    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-8xf6l" event={"ID":"527463d4-b05e-447d-9ef6-9c6a04e8fd1a","Type":"ContainerStarted","Data":"e79a528bb4ab46c011e8b472bd8bfee6d1be42626faaba1bda035feb92b124ac"}
Jul 27 19:37:09.489894 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:09.489814    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-8xf6l" podUID="527463d4-b05e-447d-9ef6-9c6a04e8fd1a" containerName="authorino" containerID="cri-o://e79a528bb4ab46c011e8b472bd8bfee6d1be42626faaba1bda035feb92b124ac" gracePeriod=30
Jul 27 19:37:09.491517 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:09.491487    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6bf8cc55c-7k2lf" event={"ID":"d32b3fe7-2366-4120-b666-7b95f2d82b0f","Type":"ContainerStarted","Data":"4f67bc26c4d9fbfc8ed49b8b0bd6fda02a55103978c8b4e8d70fde39cafb489f"}
Jul 27 19:37:09.491661 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:09.491527    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6bf8cc55c-7k2lf" event={"ID":"d32b3fe7-2366-4120-b666-7b95f2d82b0f","Type":"ContainerStarted","Data":"0831b8d6c47369457e4916539e12d00f36eba8636eb9f8af8f647a0ea1464cf9"}
Jul 27 19:37:09.506284 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:09.506243    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-8xf6l" podStartSLOduration=2.126940296 podStartE2EDuration="2.506228814s" totalImagesPullingTime="379.288518ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:37:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:37:09.504556641 +0000 UTC m=+642.376748652" watchObservedRunningTime="2026-07-27 19:37:09.506228814 +0000 UTC m=+642.378420838"
Jul 27 19:37:09.522853 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:09.522794    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6bf8cc55c-7k2lf" podStartSLOduration=1.22168789 podStartE2EDuration="1.522778274s" totalImagesPullingTime="301.090384ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:37:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:37:09.522083089 +0000 UTC m=+642.394275066" watchObservedRunningTime="2026-07-27 19:37:09.522778274 +0000 UTC m=+642.394970248"
Jul 27 19:37:09.726894 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:09.726872    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-8xf6l"
Jul 27 19:37:09.860207 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:09.860128    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rtsbs\" (UniqueName: \"kubernetes.io/projected/527463d4-b05e-447d-9ef6-9c6a04e8fd1a-kube-api-access-rtsbs\") pod \"527463d4-b05e-447d-9ef6-9c6a04e8fd1a\" (UID: \"527463d4-b05e-447d-9ef6-9c6a04e8fd1a\") "
Jul 27 19:37:09.862311 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:09.862287    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/527463d4-b05e-447d-9ef6-9c6a04e8fd1a-kube-api-access-rtsbs" (OuterVolumeSpecName: "kube-api-access-rtsbs") pod "527463d4-b05e-447d-9ef6-9c6a04e8fd1a" (UID: "527463d4-b05e-447d-9ef6-9c6a04e8fd1a"). InnerVolumeSpecName "kube-api-access-rtsbs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:37:09.961429 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:09.961393    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rtsbs\" (UniqueName: \"kubernetes.io/projected/527463d4-b05e-447d-9ef6-9c6a04e8fd1a-kube-api-access-rtsbs\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:37:10.498349 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:10.498274    2558 generic.go:356] "Generic (PLEG): container finished" podID="527463d4-b05e-447d-9ef6-9c6a04e8fd1a" containerID="e79a528bb4ab46c011e8b472bd8bfee6d1be42626faaba1bda035feb92b124ac" exitCode=0
Jul 27 19:37:10.498349 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:10.498315    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-8xf6l"
Jul 27 19:37:10.498551 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:10.498364    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-8xf6l" event={"ID":"527463d4-b05e-447d-9ef6-9c6a04e8fd1a","Type":"ContainerDied","Data":"e79a528bb4ab46c011e8b472bd8bfee6d1be42626faaba1bda035feb92b124ac"}
Jul 27 19:37:10.498551 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:10.498405    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-8xf6l" event={"ID":"527463d4-b05e-447d-9ef6-9c6a04e8fd1a","Type":"ContainerDied","Data":"73e04900672d5768e86311def6767bb6b815460e86652622823e1d445ea9d109"}
Jul 27 19:37:10.498551 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:10.498435    2558 scope.go:117] "RemoveContainer" containerID="e79a528bb4ab46c011e8b472bd8bfee6d1be42626faaba1bda035feb92b124ac"
Jul 27 19:37:10.507379 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:10.507365    2558 scope.go:117] "RemoveContainer" containerID="e79a528bb4ab46c011e8b472bd8bfee6d1be42626faaba1bda035feb92b124ac"
Jul 27 19:37:10.507672 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:37:10.507653    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e79a528bb4ab46c011e8b472bd8bfee6d1be42626faaba1bda035feb92b124ac\": container with ID starting with e79a528bb4ab46c011e8b472bd8bfee6d1be42626faaba1bda035feb92b124ac not found: ID does not exist" containerID="e79a528bb4ab46c011e8b472bd8bfee6d1be42626faaba1bda035feb92b124ac"
Jul 27 19:37:10.507744 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:10.507689    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e79a528bb4ab46c011e8b472bd8bfee6d1be42626faaba1bda035feb92b124ac"} err="failed to get container status \"e79a528bb4ab46c011e8b472bd8bfee6d1be42626faaba1bda035feb92b124ac\": rpc error: code = NotFound desc = could not find container \"e79a528bb4ab46c011e8b472bd8bfee6d1be42626faaba1bda035feb92b124ac\": container with ID starting with e79a528bb4ab46c011e8b472bd8bfee6d1be42626faaba1bda035feb92b124ac not found: ID does not exist"
Jul 27 19:37:10.518331 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:10.518312    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-8xf6l"]
Jul 27 19:37:10.520319 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:10.520302    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-8xf6l"]
Jul 27 19:37:11.674314 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:11.674280    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="527463d4-b05e-447d-9ef6-9c6a04e8fd1a" path="/var/lib/kubelet/pods/527463d4-b05e-447d-9ef6-9c6a04e8fd1a/volumes"
Jul 27 19:37:14.010578 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:14.010546    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:37:14.124860 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:14.124831    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:37:14.193298 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:14.193275    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:37:15.072916 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:15.072883    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:37:18.267003 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:18.266948    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:37:18.366799 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:18.366761    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:37:27.545126 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:27.545097    2558 scope.go:117] "RemoveContainer" containerID="ccfd8d4bfa9aa9c2ad0b6fa9bea4e877e4ec0ba814a2b093ff255d1420d2be62"
Jul 27 19:37:27.553292 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:27.553269    2558 scope.go:117] "RemoveContainer" containerID="739d6c59010607ca4447e341f0241ccaab34b8cfd2977b36e0786e2ef2d742e4"
Jul 27 19:37:27.560596 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:27.560576    2558 scope.go:117] "RemoveContainer" containerID="e667c829ad34c078d62cb2ba95216c8b19ef24878a4c73cb892adf2797b92cb2"
Jul 27 19:37:27.567767 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:27.567746    2558 scope.go:117] "RemoveContainer" containerID="61e6492e60e09c1d082eb2d1d947538c85b634fe0e269717d94dd93f425ab4b7"
Jul 27 19:37:27.575115 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:27.575096    2558 scope.go:117] "RemoveContainer" containerID="87adc48d29aebe74aec150cf7f525fb04b39d152c761c051debc531d2cc8920f"
Jul 27 19:37:27.582195 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:27.582175    2558 scope.go:117] "RemoveContainer" containerID="e8fd0f725d10315f19df2e17e79be0653712edaf457e5eb9adf8b8588080744d"
Jul 27 19:37:27.593127 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:27.593105    2558 scope.go:117] "RemoveContainer" containerID="8855e85fdc59ba86ab264527b046d6f96cebe18f8684aaf3d7f38f928903dcde"
Jul 27 19:37:50.897285 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:50.897246    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:37:50.996817 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:50.996789    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:37:51.560627 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:51.560588    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:37:51.655132 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:51.655100    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:37:52.049379 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:52.049352    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:37:52.137479 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:37:52.137453    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:38:03.661972 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:38:03.661941    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:38:03.759426 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:38:03.759382    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:38:30.354873 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:38:30.354789    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:38:38.352225 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:38:38.352179    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:38:38.720281 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:38:38.720247    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:38:38.783005 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:38:38.782971    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:38:46.461388 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:38:46.461353    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:38:53.815154 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:38:53.815115    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmmh5r"]
Jul 27 19:38:53.815593 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:38:53.815550    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="527463d4-b05e-447d-9ef6-9c6a04e8fd1a" containerName="authorino"
Jul 27 19:38:53.815593 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:38:53.815564    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="527463d4-b05e-447d-9ef6-9c6a04e8fd1a" containerName="authorino"
Jul 27 19:38:53.815710 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:38:53.815625    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="527463d4-b05e-447d-9ef6-9c6a04e8fd1a" containerName="authorino"
Jul 27 19:38:53.817776 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:38:53.817761    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmmh5r"
Jul 27 19:38:53.820262 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:38:53.820240    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 27 19:38:53.821259 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:38:53.821234    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-zc62b\""
Jul 27 19:38:53.821378 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:38:53.821263    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 27 19:38:53.821378 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:38:53.821302    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 27 19:38:53.827956 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:38:53.827932    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmmh5r"]
Jul 27 19:38:53.929073 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:38:53.929040    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:38:53.946275 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:38:53.946237    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8bf5e0d0-50bf-421d-9032-bef3ae29a6ba-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmmh5r\" (UID: \"8bf5e0d0-50bf-421d-9032-bef3ae29a6ba\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmmh5r"
Jul 27 19:38:53.946389 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:38:53.946281    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8bf5e0d0-50bf-421d-9032-bef3ae29a6ba-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmmh5r\" (UID: \"8bf5e0d0-50bf-421d-9032-bef3ae29a6ba\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmmh5r"
Jul 27 19:38:53.946389 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:38:53.946312    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8bf5e0d0-50bf-421d-9032-bef3ae29a6ba-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmmh5r\" (UID: \"8bf5e0d0-50bf-421d-9032-bef3ae29a6ba\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmmh5r"
Jul 27 19:38:53.946516 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:38:53.946384    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4sm85\" (UniqueName: \"kubernetes.io/projected/8bf5e0d0-50bf-421d-9032-bef3ae29a6ba-kube-api-access-4sm85\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmmh5r\" (UID: \"8bf5e0d0-50bf-421d-9032-bef3ae29a6ba\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmmh5r"
Jul 27 19:38:53.946516 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:38:53.946433    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8bf5e0d0-50bf-421d-9032-bef3ae29a6ba-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmmh5r\" (UID: \"8bf5e0d0-50bf-421d-9032-bef3ae29a6ba\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmmh5r"
Jul 27 19:38:53.946516 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:38:53.946495    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8bf5e0d0-50bf-421d-9032-bef3ae29a6ba-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmmh5r\" (UID: \"8bf5e0d0-50bf-421d-9032-bef3ae29a6ba\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmmh5r"
Jul 27 19:38:54.047891 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:38:54.047861    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4sm85\" (UniqueName: \"kubernetes.io/projected/8bf5e0d0-50bf-421d-9032-bef3ae29a6ba-kube-api-access-4sm85\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmmh5r\" (UID: \"8bf5e0d0-50bf-421d-9032-bef3ae29a6ba\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmmh5r"
Jul 27 19:38:54.048084 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:38:54.047902    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8bf5e0d0-50bf-421d-9032-bef3ae29a6ba-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmmh5r\" (UID: \"8bf5e0d0-50bf-421d-9032-bef3ae29a6ba\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmmh5r"
Jul 27 19:38:54.048084 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:38:54.047928    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8bf5e0d0-50bf-421d-9032-bef3ae29a6ba-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmmh5r\" (UID: \"8bf5e0d0-50bf-421d-9032-bef3ae29a6ba\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmmh5r"
Jul 27 19:38:54.048084 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:38:54.047965    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8bf5e0d0-50bf-421d-9032-bef3ae29a6ba-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmmh5r\" (UID: \"8bf5e0d0-50bf-421d-9032-bef3ae29a6ba\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmmh5r"
Jul 27 19:38:54.048084 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:38:54.047981    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8bf5e0d0-50bf-421d-9032-bef3ae29a6ba-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmmh5r\" (UID: \"8bf5e0d0-50bf-421d-9032-bef3ae29a6ba\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmmh5r"
Jul 27 19:38:54.048084 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:38:54.048007    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8bf5e0d0-50bf-421d-9032-bef3ae29a6ba-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmmh5r\" (UID: \"8bf5e0d0-50bf-421d-9032-bef3ae29a6ba\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmmh5r"
Jul 27 19:38:54.048347 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:38:54.048325    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8bf5e0d0-50bf-421d-9032-bef3ae29a6ba-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmmh5r\" (UID: \"8bf5e0d0-50bf-421d-9032-bef3ae29a6ba\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmmh5r"
Jul 27 19:38:54.048392 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:38:54.048335    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8bf5e0d0-50bf-421d-9032-bef3ae29a6ba-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmmh5r\" (UID: \"8bf5e0d0-50bf-421d-9032-bef3ae29a6ba\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmmh5r"
Jul 27 19:38:54.048491 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:38:54.048448    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8bf5e0d0-50bf-421d-9032-bef3ae29a6ba-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmmh5r\" (UID: \"8bf5e0d0-50bf-421d-9032-bef3ae29a6ba\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmmh5r"
Jul 27 19:38:54.050490 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:38:54.050461    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8bf5e0d0-50bf-421d-9032-bef3ae29a6ba-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmmh5r\" (UID: \"8bf5e0d0-50bf-421d-9032-bef3ae29a6ba\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmmh5r"
Jul 27 19:38:54.050607 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:38:54.050567    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8bf5e0d0-50bf-421d-9032-bef3ae29a6ba-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmmh5r\" (UID: \"8bf5e0d0-50bf-421d-9032-bef3ae29a6ba\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmmh5r"
Jul 27 19:38:54.056187 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:38:54.056170    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4sm85\" (UniqueName: \"kubernetes.io/projected/8bf5e0d0-50bf-421d-9032-bef3ae29a6ba-kube-api-access-4sm85\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmmh5r\" (UID: \"8bf5e0d0-50bf-421d-9032-bef3ae29a6ba\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmmh5r"
Jul 27 19:38:54.127949 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:38:54.127914    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmmh5r"
Jul 27 19:38:54.259795 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:38:54.259767    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmmh5r"]
Jul 27 19:38:54.261436 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:38:54.261394    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bf5e0d0_50bf_421d_9032_bef3ae29a6ba.slice/crio-6c0333ef14ae45f127d97cd5eb69d8a0ba56683ece0dc605e70099b7b4a4423b WatchSource:0}: Error finding container 6c0333ef14ae45f127d97cd5eb69d8a0ba56683ece0dc605e70099b7b4a4423b: Status 404 returned error can't find the container with id 6c0333ef14ae45f127d97cd5eb69d8a0ba56683ece0dc605e70099b7b4a4423b
Jul 27 19:38:55.048738 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:38:55.048701    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmmh5r" event={"ID":"8bf5e0d0-50bf-421d-9032-bef3ae29a6ba","Type":"ContainerStarted","Data":"6c0333ef14ae45f127d97cd5eb69d8a0ba56683ece0dc605e70099b7b4a4423b"}
Jul 27 19:38:56.058078 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:38:56.058038    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmmh5r" event={"ID":"8bf5e0d0-50bf-421d-9032-bef3ae29a6ba","Type":"ContainerStarted","Data":"860ebadb6ed74e544ddfc662c34d3f5af35b6989bd6a12a6d40c8c1b5b1dd755"}
Jul 27 19:38:56.077055 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:38:56.076991    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmmh5r" podStartSLOduration=2.104824393 podStartE2EDuration="3.076971459s" totalImagesPullingTime="972.147066ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:38:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:38:56.075469447 +0000 UTC m=+748.947661425" watchObservedRunningTime="2026-07-27 19:38:56.076971459 +0000 UTC m=+748.949163434"
Jul 27 19:39:01.862903 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:39:01.862868    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:39:02.229866 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:39:02.229794    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-4wgfz"]
Jul 27 19:39:02.232455 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:39:02.232437    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4wgfz"
Jul 27 19:39:02.234856 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:39:02.234834    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 27 19:39:02.241971 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:39:02.241950    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-4wgfz"]
Jul 27 19:39:02.292940 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:39:02.292910    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:39:02.327457 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:39:02.324622    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a498b8bd-3065-48e6-bf8a-34ab47aaae24-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4wgfz\" (UID: \"a498b8bd-3065-48e6-bf8a-34ab47aaae24\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4wgfz"
Jul 27 19:39:02.327457 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:39:02.324772    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a498b8bd-3065-48e6-bf8a-34ab47aaae24-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4wgfz\" (UID: \"a498b8bd-3065-48e6-bf8a-34ab47aaae24\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4wgfz"
Jul 27 19:39:02.327457 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:39:02.324806    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a498b8bd-3065-48e6-bf8a-34ab47aaae24-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4wgfz\" (UID: \"a498b8bd-3065-48e6-bf8a-34ab47aaae24\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4wgfz"
Jul 27 19:39:02.327457 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:39:02.324836    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a498b8bd-3065-48e6-bf8a-34ab47aaae24-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4wgfz\" (UID: \"a498b8bd-3065-48e6-bf8a-34ab47aaae24\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4wgfz"
Jul 27 19:39:02.327457 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:39:02.324884    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbv4s\" (UniqueName: \"kubernetes.io/projected/a498b8bd-3065-48e6-bf8a-34ab47aaae24-kube-api-access-lbv4s\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4wgfz\" (UID: \"a498b8bd-3065-48e6-bf8a-34ab47aaae24\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4wgfz"
Jul 27 19:39:02.327457 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:39:02.324941    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a498b8bd-3065-48e6-bf8a-34ab47aaae24-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4wgfz\" (UID: \"a498b8bd-3065-48e6-bf8a-34ab47aaae24\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4wgfz"
Jul 27 19:39:02.426243 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:39:02.426199    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a498b8bd-3065-48e6-bf8a-34ab47aaae24-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4wgfz\" (UID: \"a498b8bd-3065-48e6-bf8a-34ab47aaae24\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4wgfz"
Jul 27 19:39:02.426444 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:39:02.426308    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a498b8bd-3065-48e6-bf8a-34ab47aaae24-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4wgfz\" (UID: \"a498b8bd-3065-48e6-bf8a-34ab47aaae24\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4wgfz"
Jul 27 19:39:02.426444 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:39:02.426335    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a498b8bd-3065-48e6-bf8a-34ab47aaae24-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4wgfz\" (UID: \"a498b8bd-3065-48e6-bf8a-34ab47aaae24\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4wgfz"
Jul 27 19:39:02.426444 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:39:02.426362    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a498b8bd-3065-48e6-bf8a-34ab47aaae24-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4wgfz\" (UID: \"a498b8bd-3065-48e6-bf8a-34ab47aaae24\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4wgfz"
Jul 27 19:39:02.426444 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:39:02.426399    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbv4s\" (UniqueName: \"kubernetes.io/projected/a498b8bd-3065-48e6-bf8a-34ab47aaae24-kube-api-access-lbv4s\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4wgfz\" (UID: \"a498b8bd-3065-48e6-bf8a-34ab47aaae24\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4wgfz"
Jul 27 19:39:02.426682 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:39:02.426466    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a498b8bd-3065-48e6-bf8a-34ab47aaae24-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4wgfz\" (UID: \"a498b8bd-3065-48e6-bf8a-34ab47aaae24\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4wgfz"
Jul 27 19:39:02.426741 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:39:02.426701    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a498b8bd-3065-48e6-bf8a-34ab47aaae24-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4wgfz\" (UID: \"a498b8bd-3065-48e6-bf8a-34ab47aaae24\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4wgfz"
Jul 27 19:39:02.426853 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:39:02.426764    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a498b8bd-3065-48e6-bf8a-34ab47aaae24-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4wgfz\" (UID: \"a498b8bd-3065-48e6-bf8a-34ab47aaae24\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4wgfz"
Jul 27 19:39:02.426897 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:39:02.426851    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a498b8bd-3065-48e6-bf8a-34ab47aaae24-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4wgfz\" (UID: \"a498b8bd-3065-48e6-bf8a-34ab47aaae24\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4wgfz"
Jul 27 19:39:02.428907 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:39:02.428888    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a498b8bd-3065-48e6-bf8a-34ab47aaae24-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4wgfz\" (UID: \"a498b8bd-3065-48e6-bf8a-34ab47aaae24\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4wgfz"
Jul 27 19:39:02.429192 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:39:02.429171    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a498b8bd-3065-48e6-bf8a-34ab47aaae24-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4wgfz\" (UID: \"a498b8bd-3065-48e6-bf8a-34ab47aaae24\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4wgfz"
Jul 27 19:39:02.439556 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:39:02.439535    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbv4s\" (UniqueName: \"kubernetes.io/projected/a498b8bd-3065-48e6-bf8a-34ab47aaae24-kube-api-access-lbv4s\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4wgfz\" (UID: \"a498b8bd-3065-48e6-bf8a-34ab47aaae24\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4wgfz"
Jul 27 19:39:02.543854 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:39:02.543785    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4wgfz"
Jul 27 19:39:02.682964 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:39:02.682940    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-4wgfz"]
Jul 27 19:39:02.684830 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:39:02.684806    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda498b8bd_3065_48e6_bf8a_34ab47aaae24.slice/crio-d3b4ef478d3e07626816ee5c9d00da02f50bdcd6017e4e942273208adb6b02c9 WatchSource:0}: Error finding container d3b4ef478d3e07626816ee5c9d00da02f50bdcd6017e4e942273208adb6b02c9: Status 404 returned error can't find the container with id d3b4ef478d3e07626816ee5c9d00da02f50bdcd6017e4e942273208adb6b02c9
Jul 27 19:39:03.096953 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:39:03.096922    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4wgfz" event={"ID":"a498b8bd-3065-48e6-bf8a-34ab47aaae24","Type":"ContainerStarted","Data":"5da258f4a3ef9a8f00a8d6ed713af7dcd6b0e891bfa607ff8136b2b78b627c94"}
Jul 27 19:39:03.097315 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:39:03.096972    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4wgfz" event={"ID":"a498b8bd-3065-48e6-bf8a-34ab47aaae24","Type":"ContainerStarted","Data":"d3b4ef478d3e07626816ee5c9d00da02f50bdcd6017e4e942273208adb6b02c9"}
Jul 27 19:39:03.116114 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:39:03.116062    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4wgfz" podStartSLOduration=0.919876311 podStartE2EDuration="1.116048227s" totalImagesPullingTime="196.171916ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:39:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:39:03.113387984 +0000 UTC m=+755.985579960" watchObservedRunningTime="2026-07-27 19:39:03.116048227 +0000 UTC m=+755.988240233"
Jul 27 19:39:04.128916 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:39:04.128876    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmmh5r"
Jul 27 19:39:04.129376 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:39:04.128959    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmmh5r"
Jul 27 19:39:04.141240 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:39:04.141213    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmmh5r"
Jul 27 19:39:05.123319 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:39:05.123283    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmmh5r"
Jul 27 19:39:07.264700 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:39:07.264669    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:39:12.521154 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:39:12.521116    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:39:12.544987 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:39:12.544950    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4wgfz"
Jul 27 19:39:12.545152 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:39:12.545015    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4wgfz"
Jul 27 19:39:12.557764 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:39:12.557738    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4wgfz"
Jul 27 19:39:13.169023 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:39:13.168986    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4wgfz"
Jul 27 19:39:22.580518 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:39:22.580484    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:39:22.646318 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:39:22.646289    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:39:31.972031 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:39:31.972001    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:40:04.367524 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:04.367452    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-d99cd47d4-bgbtm"]
Jul 27 19:40:04.369898 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:04.369884    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-d99cd47d4-bgbtm"
Jul 27 19:40:04.374959 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:04.374934    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztjm5\" (UniqueName: \"kubernetes.io/projected/c552df59-560f-4daa-8a45-9b2135dbe53e-kube-api-access-ztjm5\") pod \"authorino-d99cd47d4-bgbtm\" (UID: \"c552df59-560f-4daa-8a45-9b2135dbe53e\") " pod="kuadrant-system/authorino-d99cd47d4-bgbtm"
Jul 27 19:40:04.375074 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:04.374984    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c552df59-560f-4daa-8a45-9b2135dbe53e-tls-cert\") pod \"authorino-d99cd47d4-bgbtm\" (UID: \"c552df59-560f-4daa-8a45-9b2135dbe53e\") " pod="kuadrant-system/authorino-d99cd47d4-bgbtm"
Jul 27 19:40:04.380626 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:04.380603    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-d99cd47d4-bgbtm"]
Jul 27 19:40:04.475875 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:04.475845    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztjm5\" (UniqueName: \"kubernetes.io/projected/c552df59-560f-4daa-8a45-9b2135dbe53e-kube-api-access-ztjm5\") pod \"authorino-d99cd47d4-bgbtm\" (UID: \"c552df59-560f-4daa-8a45-9b2135dbe53e\") " pod="kuadrant-system/authorino-d99cd47d4-bgbtm"
Jul 27 19:40:04.476033 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:04.475895    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c552df59-560f-4daa-8a45-9b2135dbe53e-tls-cert\") pod \"authorino-d99cd47d4-bgbtm\" (UID: \"c552df59-560f-4daa-8a45-9b2135dbe53e\") " pod="kuadrant-system/authorino-d99cd47d4-bgbtm"
Jul 27 19:40:04.478498 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:04.478470    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c552df59-560f-4daa-8a45-9b2135dbe53e-tls-cert\") pod \"authorino-d99cd47d4-bgbtm\" (UID: \"c552df59-560f-4daa-8a45-9b2135dbe53e\") " pod="kuadrant-system/authorino-d99cd47d4-bgbtm"
Jul 27 19:40:04.483853 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:04.483833    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztjm5\" (UniqueName: \"kubernetes.io/projected/c552df59-560f-4daa-8a45-9b2135dbe53e-kube-api-access-ztjm5\") pod \"authorino-d99cd47d4-bgbtm\" (UID: \"c552df59-560f-4daa-8a45-9b2135dbe53e\") " pod="kuadrant-system/authorino-d99cd47d4-bgbtm"
Jul 27 19:40:04.679362 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:04.679285    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-d99cd47d4-bgbtm"
Jul 27 19:40:04.719524 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:04.719490    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:40:04.818501 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:04.818462    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:40:04.818641 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:04.818623    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-d99cd47d4-bgbtm"]
Jul 27 19:40:04.820430 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:40:04.820392    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc552df59_560f_4daa_8a45_9b2135dbe53e.slice/crio-21ea92816c060cccbd6e9d6473158aa641f304ef21516ef0e1dec0a7ab6ae97a WatchSource:0}: Error finding container 21ea92816c060cccbd6e9d6473158aa641f304ef21516ef0e1dec0a7ab6ae97a: Status 404 returned error can't find the container with id 21ea92816c060cccbd6e9d6473158aa641f304ef21516ef0e1dec0a7ab6ae97a
Jul 27 19:40:05.434889 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:05.434850    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-d99cd47d4-bgbtm" event={"ID":"c552df59-560f-4daa-8a45-9b2135dbe53e","Type":"ContainerStarted","Data":"2b8935ab570021bb8d29177e9ba44a1bb4099f30a8c26005a7ba69907d2ab925"}
Jul 27 19:40:05.435304 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:05.434902    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-d99cd47d4-bgbtm" event={"ID":"c552df59-560f-4daa-8a45-9b2135dbe53e","Type":"ContainerStarted","Data":"21ea92816c060cccbd6e9d6473158aa641f304ef21516ef0e1dec0a7ab6ae97a"}
Jul 27 19:40:05.457626 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:05.457569    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-d99cd47d4-bgbtm" podStartSLOduration=0.981513293 podStartE2EDuration="1.457552359s" totalImagesPullingTime="476.039066ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:40:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:40:05.455107984 +0000 UTC m=+818.327299960" watchObservedRunningTime="2026-07-27 19:40:05.457552359 +0000 UTC m=+818.329744333"
Jul 27 19:40:05.505437 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:05.505392    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6bf8cc55c-7k2lf"]
Jul 27 19:40:05.505608 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:05.505572    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-6bf8cc55c-7k2lf" podUID="d32b3fe7-2366-4120-b666-7b95f2d82b0f" containerName="authorino" containerID="cri-o://4f67bc26c4d9fbfc8ed49b8b0bd6fda02a55103978c8b4e8d70fde39cafb489f" gracePeriod=30
Jul 27 19:40:05.763365 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:05.763338    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:40:05.780618 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:05.780588    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6bf8cc55c-7k2lf"
Jul 27 19:40:05.785855 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:05.785818    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6fgn6\" (UniqueName: \"kubernetes.io/projected/d32b3fe7-2366-4120-b666-7b95f2d82b0f-kube-api-access-6fgn6\") pod \"d32b3fe7-2366-4120-b666-7b95f2d82b0f\" (UID: \"d32b3fe7-2366-4120-b666-7b95f2d82b0f\") "
Jul 27 19:40:05.786044 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:05.785889    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d32b3fe7-2366-4120-b666-7b95f2d82b0f-tls-cert\") pod \"d32b3fe7-2366-4120-b666-7b95f2d82b0f\" (UID: \"d32b3fe7-2366-4120-b666-7b95f2d82b0f\") "
Jul 27 19:40:05.788463 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:05.788437    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d32b3fe7-2366-4120-b666-7b95f2d82b0f-kube-api-access-6fgn6" (OuterVolumeSpecName: "kube-api-access-6fgn6") pod "d32b3fe7-2366-4120-b666-7b95f2d82b0f" (UID: "d32b3fe7-2366-4120-b666-7b95f2d82b0f"). InnerVolumeSpecName "kube-api-access-6fgn6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:40:05.797761 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:05.797719    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d32b3fe7-2366-4120-b666-7b95f2d82b0f-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "d32b3fe7-2366-4120-b666-7b95f2d82b0f" (UID: "d32b3fe7-2366-4120-b666-7b95f2d82b0f"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:40:05.865017 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:05.864981    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:40:05.887031 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:05.886953    2558 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d32b3fe7-2366-4120-b666-7b95f2d82b0f-tls-cert\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:40:05.887031 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:05.886979    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6fgn6\" (UniqueName: \"kubernetes.io/projected/d32b3fe7-2366-4120-b666-7b95f2d82b0f-kube-api-access-6fgn6\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:40:06.442107 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:06.442070    2558 generic.go:356] "Generic (PLEG): container finished" podID="d32b3fe7-2366-4120-b666-7b95f2d82b0f" containerID="4f67bc26c4d9fbfc8ed49b8b0bd6fda02a55103978c8b4e8d70fde39cafb489f" exitCode=0
Jul 27 19:40:06.442564 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:06.442132    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6bf8cc55c-7k2lf"
Jul 27 19:40:06.442564 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:06.442164    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6bf8cc55c-7k2lf" event={"ID":"d32b3fe7-2366-4120-b666-7b95f2d82b0f","Type":"ContainerDied","Data":"4f67bc26c4d9fbfc8ed49b8b0bd6fda02a55103978c8b4e8d70fde39cafb489f"}
Jul 27 19:40:06.442564 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:06.442204    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6bf8cc55c-7k2lf" event={"ID":"d32b3fe7-2366-4120-b666-7b95f2d82b0f","Type":"ContainerDied","Data":"0831b8d6c47369457e4916539e12d00f36eba8636eb9f8af8f647a0ea1464cf9"}
Jul 27 19:40:06.442564 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:06.442221    2558 scope.go:117] "RemoveContainer" containerID="4f67bc26c4d9fbfc8ed49b8b0bd6fda02a55103978c8b4e8d70fde39cafb489f"
Jul 27 19:40:06.451132 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:06.451116    2558 scope.go:117] "RemoveContainer" containerID="4f67bc26c4d9fbfc8ed49b8b0bd6fda02a55103978c8b4e8d70fde39cafb489f"
Jul 27 19:40:06.451474 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:40:06.451406    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f67bc26c4d9fbfc8ed49b8b0bd6fda02a55103978c8b4e8d70fde39cafb489f\": container with ID starting with 4f67bc26c4d9fbfc8ed49b8b0bd6fda02a55103978c8b4e8d70fde39cafb489f not found: ID does not exist" containerID="4f67bc26c4d9fbfc8ed49b8b0bd6fda02a55103978c8b4e8d70fde39cafb489f"
Jul 27 19:40:06.451580 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:06.451485    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f67bc26c4d9fbfc8ed49b8b0bd6fda02a55103978c8b4e8d70fde39cafb489f"} err="failed to get container status \"4f67bc26c4d9fbfc8ed49b8b0bd6fda02a55103978c8b4e8d70fde39cafb489f\": rpc error: code = NotFound desc = could not find container \"4f67bc26c4d9fbfc8ed49b8b0bd6fda02a55103978c8b4e8d70fde39cafb489f\": container with ID starting with 4f67bc26c4d9fbfc8ed49b8b0bd6fda02a55103978c8b4e8d70fde39cafb489f not found: ID does not exist"
Jul 27 19:40:06.468276 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:06.468244    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6bf8cc55c-7k2lf"]
Jul 27 19:40:06.471833 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:06.471808    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6bf8cc55c-7k2lf"]
Jul 27 19:40:07.674204 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:07.674171    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d32b3fe7-2366-4120-b666-7b95f2d82b0f" path="/var/lib/kubelet/pods/d32b3fe7-2366-4120-b666-7b95f2d82b0f/volumes"
Jul 27 19:40:35.613876 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:35.613847    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:40:35.717160 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:35.717130    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:40:58.870536 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:58.870492    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7bfd7b9bdd-h69g8"]
Jul 27 19:40:58.871050 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:58.871029    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d32b3fe7-2366-4120-b666-7b95f2d82b0f" containerName="authorino"
Jul 27 19:40:58.871112 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:58.871054    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="d32b3fe7-2366-4120-b666-7b95f2d82b0f" containerName="authorino"
Jul 27 19:40:58.871147 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:58.871131    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="d32b3fe7-2366-4120-b666-7b95f2d82b0f" containerName="authorino"
Jul 27 19:40:58.874101 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:58.874080    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bfd7b9bdd-h69g8"
Jul 27 19:40:58.885905 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:58.885877    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7bfd7b9bdd-h69g8"]
Jul 27 19:40:58.926575 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:58.926542    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:40:58.958824 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:58.958776    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxwts\" (UniqueName: \"kubernetes.io/projected/8fb60a14-bcf1-491a-a882-ab2bbfbc02dc-kube-api-access-jxwts\") pod \"console-7bfd7b9bdd-h69g8\" (UID: \"8fb60a14-bcf1-491a-a882-ab2bbfbc02dc\") " pod="openshift-console/console-7bfd7b9bdd-h69g8"
Jul 27 19:40:58.958976 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:58.958844    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8fb60a14-bcf1-491a-a882-ab2bbfbc02dc-console-config\") pod \"console-7bfd7b9bdd-h69g8\" (UID: \"8fb60a14-bcf1-491a-a882-ab2bbfbc02dc\") " pod="openshift-console/console-7bfd7b9bdd-h69g8"
Jul 27 19:40:58.958976 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:58.958921    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8fb60a14-bcf1-491a-a882-ab2bbfbc02dc-console-serving-cert\") pod \"console-7bfd7b9bdd-h69g8\" (UID: \"8fb60a14-bcf1-491a-a882-ab2bbfbc02dc\") " pod="openshift-console/console-7bfd7b9bdd-h69g8"
Jul 27 19:40:58.958976 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:58.958950    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8fb60a14-bcf1-491a-a882-ab2bbfbc02dc-console-oauth-config\") pod \"console-7bfd7b9bdd-h69g8\" (UID: \"8fb60a14-bcf1-491a-a882-ab2bbfbc02dc\") " pod="openshift-console/console-7bfd7b9bdd-h69g8"
Jul 27 19:40:58.959078 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:58.959036    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fb60a14-bcf1-491a-a882-ab2bbfbc02dc-trusted-ca-bundle\") pod \"console-7bfd7b9bdd-h69g8\" (UID: \"8fb60a14-bcf1-491a-a882-ab2bbfbc02dc\") " pod="openshift-console/console-7bfd7b9bdd-h69g8"
Jul 27 19:40:58.959112 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:58.959079    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8fb60a14-bcf1-491a-a882-ab2bbfbc02dc-service-ca\") pod \"console-7bfd7b9bdd-h69g8\" (UID: \"8fb60a14-bcf1-491a-a882-ab2bbfbc02dc\") " pod="openshift-console/console-7bfd7b9bdd-h69g8"
Jul 27 19:40:58.959112 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:58.959097    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8fb60a14-bcf1-491a-a882-ab2bbfbc02dc-oauth-serving-cert\") pod \"console-7bfd7b9bdd-h69g8\" (UID: \"8fb60a14-bcf1-491a-a882-ab2bbfbc02dc\") " pod="openshift-console/console-7bfd7b9bdd-h69g8"
Jul 27 19:40:58.990623 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:58.990590    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:40:59.059603 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:59.059562    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fb60a14-bcf1-491a-a882-ab2bbfbc02dc-trusted-ca-bundle\") pod \"console-7bfd7b9bdd-h69g8\" (UID: \"8fb60a14-bcf1-491a-a882-ab2bbfbc02dc\") " pod="openshift-console/console-7bfd7b9bdd-h69g8"
Jul 27 19:40:59.059603 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:59.059607    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8fb60a14-bcf1-491a-a882-ab2bbfbc02dc-service-ca\") pod \"console-7bfd7b9bdd-h69g8\" (UID: \"8fb60a14-bcf1-491a-a882-ab2bbfbc02dc\") " pod="openshift-console/console-7bfd7b9bdd-h69g8"
Jul 27 19:40:59.059852 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:59.059624    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8fb60a14-bcf1-491a-a882-ab2bbfbc02dc-oauth-serving-cert\") pod \"console-7bfd7b9bdd-h69g8\" (UID: \"8fb60a14-bcf1-491a-a882-ab2bbfbc02dc\") " pod="openshift-console/console-7bfd7b9bdd-h69g8"
Jul 27 19:40:59.059852 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:59.059646    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxwts\" (UniqueName: \"kubernetes.io/projected/8fb60a14-bcf1-491a-a882-ab2bbfbc02dc-kube-api-access-jxwts\") pod \"console-7bfd7b9bdd-h69g8\" (UID: \"8fb60a14-bcf1-491a-a882-ab2bbfbc02dc\") " pod="openshift-console/console-7bfd7b9bdd-h69g8"
Jul 27 19:40:59.059852 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:59.059679    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8fb60a14-bcf1-491a-a882-ab2bbfbc02dc-console-config\") pod \"console-7bfd7b9bdd-h69g8\" (UID: \"8fb60a14-bcf1-491a-a882-ab2bbfbc02dc\") " pod="openshift-console/console-7bfd7b9bdd-h69g8"
Jul 27 19:40:59.059852 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:59.059702    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8fb60a14-bcf1-491a-a882-ab2bbfbc02dc-console-serving-cert\") pod \"console-7bfd7b9bdd-h69g8\" (UID: \"8fb60a14-bcf1-491a-a882-ab2bbfbc02dc\") " pod="openshift-console/console-7bfd7b9bdd-h69g8"
Jul 27 19:40:59.059852 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:59.059718    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8fb60a14-bcf1-491a-a882-ab2bbfbc02dc-console-oauth-config\") pod \"console-7bfd7b9bdd-h69g8\" (UID: \"8fb60a14-bcf1-491a-a882-ab2bbfbc02dc\") " pod="openshift-console/console-7bfd7b9bdd-h69g8"
Jul 27 19:40:59.060460 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:59.060406    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8fb60a14-bcf1-491a-a882-ab2bbfbc02dc-oauth-serving-cert\") pod \"console-7bfd7b9bdd-h69g8\" (UID: \"8fb60a14-bcf1-491a-a882-ab2bbfbc02dc\") " pod="openshift-console/console-7bfd7b9bdd-h69g8"
Jul 27 19:40:59.060620 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:59.060602    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fb60a14-bcf1-491a-a882-ab2bbfbc02dc-trusted-ca-bundle\") pod \"console-7bfd7b9bdd-h69g8\" (UID: \"8fb60a14-bcf1-491a-a882-ab2bbfbc02dc\") " pod="openshift-console/console-7bfd7b9bdd-h69g8"
Jul 27 19:40:59.060701 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:59.060553    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8fb60a14-bcf1-491a-a882-ab2bbfbc02dc-service-ca\") pod \"console-7bfd7b9bdd-h69g8\" (UID: \"8fb60a14-bcf1-491a-a882-ab2bbfbc02dc\") " pod="openshift-console/console-7bfd7b9bdd-h69g8"
Jul 27 19:40:59.060701 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:59.060600    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8fb60a14-bcf1-491a-a882-ab2bbfbc02dc-console-config\") pod \"console-7bfd7b9bdd-h69g8\" (UID: \"8fb60a14-bcf1-491a-a882-ab2bbfbc02dc\") " pod="openshift-console/console-7bfd7b9bdd-h69g8"
Jul 27 19:40:59.062884 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:59.062865    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8fb60a14-bcf1-491a-a882-ab2bbfbc02dc-console-oauth-config\") pod \"console-7bfd7b9bdd-h69g8\" (UID: \"8fb60a14-bcf1-491a-a882-ab2bbfbc02dc\") " pod="openshift-console/console-7bfd7b9bdd-h69g8"
Jul 27 19:40:59.063087 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:59.063066    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8fb60a14-bcf1-491a-a882-ab2bbfbc02dc-console-serving-cert\") pod \"console-7bfd7b9bdd-h69g8\" (UID: \"8fb60a14-bcf1-491a-a882-ab2bbfbc02dc\") " pod="openshift-console/console-7bfd7b9bdd-h69g8"
Jul 27 19:40:59.070043 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:59.070016    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxwts\" (UniqueName: \"kubernetes.io/projected/8fb60a14-bcf1-491a-a882-ab2bbfbc02dc-kube-api-access-jxwts\") pod \"console-7bfd7b9bdd-h69g8\" (UID: \"8fb60a14-bcf1-491a-a882-ab2bbfbc02dc\") " pod="openshift-console/console-7bfd7b9bdd-h69g8"
Jul 27 19:40:59.094919 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:59.094889    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:40:59.184145 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:59.184066    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bfd7b9bdd-h69g8"
Jul 27 19:40:59.192571 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:59.192547    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:40:59.322238 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:59.322212    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7bfd7b9bdd-h69g8"]
Jul 27 19:40:59.324759 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:40:59.324721    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fb60a14_bcf1_491a_a882_ab2bbfbc02dc.slice/crio-467dc487a6dcdcb37e1bac0c185a26bf61666ce236c9d0c869883b1a59154bfb WatchSource:0}: Error finding container 467dc487a6dcdcb37e1bac0c185a26bf61666ce236c9d0c869883b1a59154bfb: Status 404 returned error can't find the container with id 467dc487a6dcdcb37e1bac0c185a26bf61666ce236c9d0c869883b1a59154bfb
Jul 27 19:40:59.675202 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:59.675169    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:40:59.729504 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:59.729451    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bfd7b9bdd-h69g8" event={"ID":"8fb60a14-bcf1-491a-a882-ab2bbfbc02dc","Type":"ContainerStarted","Data":"b556f5b562030726760a8dd8641dd78cdf70639a2c1c189cb8831460b1c27b91"}
Jul 27 19:40:59.729712 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:59.729550    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bfd7b9bdd-h69g8" event={"ID":"8fb60a14-bcf1-491a-a882-ab2bbfbc02dc","Type":"ContainerStarted","Data":"467dc487a6dcdcb37e1bac0c185a26bf61666ce236c9d0c869883b1a59154bfb"}
Jul 27 19:40:59.740889 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:59.740852    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:40:59.749619 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:40:59.749566    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7bfd7b9bdd-h69g8" podStartSLOduration=1.7495483109999999 podStartE2EDuration="1.749548311s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:40:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:40:59.748715196 +0000 UTC m=+872.620907171" watchObservedRunningTime="2026-07-27 19:40:59.749548311 +0000 UTC m=+872.621740285"
Jul 27 19:41:09.184704 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:41:09.184660    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7bfd7b9bdd-h69g8"
Jul 27 19:41:09.185221 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:41:09.184739    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7bfd7b9bdd-h69g8"
Jul 27 19:41:09.189580 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:41:09.189548    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7bfd7b9bdd-h69g8"
Jul 27 19:41:09.788918 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:41:09.788886    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7bfd7b9bdd-h69g8"
Jul 27 19:41:09.837814 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:41:09.837775    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56ff6c45bc-kcmv8"]
Jul 27 19:41:09.939575 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:41:09.939543    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:41:10.039055 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:41:10.038972    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:41:10.099173 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:41:10.099143    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:41:27.573392 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:41:27.573306    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5dfc2_103bab23-bca3-472a-974b-7731f83c5bfb/ovn-acl-logging/0.log"
Jul 27 19:41:27.575640 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:41:27.575612    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5dfc2_103bab23-bca3-472a-974b-7731f83c5bfb/ovn-acl-logging/0.log"
Jul 27 19:41:34.859023 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:41:34.858961    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-56ff6c45bc-kcmv8" podUID="7643b91f-b8d8-4e7a-aefb-8fb2531b7029" containerName="console" containerID="cri-o://1589a28bf9c203cf3d0ebef5e8cf0141feb7a0f4162af7d97580d879578ed93a" gracePeriod=15
Jul 27 19:41:35.108645 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:41:35.108617    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56ff6c45bc-kcmv8"
Jul 27 19:41:35.297864 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:41:35.297821    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7643b91f-b8d8-4e7a-aefb-8fb2531b7029-service-ca\") pod \"7643b91f-b8d8-4e7a-aefb-8fb2531b7029\" (UID: \"7643b91f-b8d8-4e7a-aefb-8fb2531b7029\") "
Jul 27 19:41:35.297864 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:41:35.297871    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7643b91f-b8d8-4e7a-aefb-8fb2531b7029-trusted-ca-bundle\") pod \"7643b91f-b8d8-4e7a-aefb-8fb2531b7029\" (UID: \"7643b91f-b8d8-4e7a-aefb-8fb2531b7029\") "
Jul 27 19:41:35.298141 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:41:35.297931    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7643b91f-b8d8-4e7a-aefb-8fb2531b7029-console-oauth-config\") pod \"7643b91f-b8d8-4e7a-aefb-8fb2531b7029\" (UID: \"7643b91f-b8d8-4e7a-aefb-8fb2531b7029\") "
Jul 27 19:41:35.298141 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:41:35.297963    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r2pvn\" (UniqueName: \"kubernetes.io/projected/7643b91f-b8d8-4e7a-aefb-8fb2531b7029-kube-api-access-r2pvn\") pod \"7643b91f-b8d8-4e7a-aefb-8fb2531b7029\" (UID: \"7643b91f-b8d8-4e7a-aefb-8fb2531b7029\") "
Jul 27 19:41:35.298141 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:41:35.298008    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7643b91f-b8d8-4e7a-aefb-8fb2531b7029-console-config\") pod \"7643b91f-b8d8-4e7a-aefb-8fb2531b7029\" (UID: \"7643b91f-b8d8-4e7a-aefb-8fb2531b7029\") "
Jul 27 19:41:35.298141 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:41:35.298028    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7643b91f-b8d8-4e7a-aefb-8fb2531b7029-oauth-serving-cert\") pod \"7643b91f-b8d8-4e7a-aefb-8fb2531b7029\" (UID: \"7643b91f-b8d8-4e7a-aefb-8fb2531b7029\") "
Jul 27 19:41:35.298141 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:41:35.298102    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7643b91f-b8d8-4e7a-aefb-8fb2531b7029-console-serving-cert\") pod \"7643b91f-b8d8-4e7a-aefb-8fb2531b7029\" (UID: \"7643b91f-b8d8-4e7a-aefb-8fb2531b7029\") "
Jul 27 19:41:35.298372 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:41:35.298287    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7643b91f-b8d8-4e7a-aefb-8fb2531b7029-service-ca" (OuterVolumeSpecName: "service-ca") pod "7643b91f-b8d8-4e7a-aefb-8fb2531b7029" (UID: "7643b91f-b8d8-4e7a-aefb-8fb2531b7029"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:41:35.298509 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:41:35.298477    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7643b91f-b8d8-4e7a-aefb-8fb2531b7029-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "7643b91f-b8d8-4e7a-aefb-8fb2531b7029" (UID: "7643b91f-b8d8-4e7a-aefb-8fb2531b7029"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:41:35.298509 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:41:35.298488    2558 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7643b91f-b8d8-4e7a-aefb-8fb2531b7029-service-ca\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:41:35.298681 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:41:35.298512    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7643b91f-b8d8-4e7a-aefb-8fb2531b7029-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "7643b91f-b8d8-4e7a-aefb-8fb2531b7029" (UID: "7643b91f-b8d8-4e7a-aefb-8fb2531b7029"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:41:35.298681 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:41:35.298563    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7643b91f-b8d8-4e7a-aefb-8fb2531b7029-console-config" (OuterVolumeSpecName: "console-config") pod "7643b91f-b8d8-4e7a-aefb-8fb2531b7029" (UID: "7643b91f-b8d8-4e7a-aefb-8fb2531b7029"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:41:35.300403 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:41:35.300371    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7643b91f-b8d8-4e7a-aefb-8fb2531b7029-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "7643b91f-b8d8-4e7a-aefb-8fb2531b7029" (UID: "7643b91f-b8d8-4e7a-aefb-8fb2531b7029"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:41:35.300403 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:41:35.300385    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7643b91f-b8d8-4e7a-aefb-8fb2531b7029-kube-api-access-r2pvn" (OuterVolumeSpecName: "kube-api-access-r2pvn") pod "7643b91f-b8d8-4e7a-aefb-8fb2531b7029" (UID: "7643b91f-b8d8-4e7a-aefb-8fb2531b7029"). InnerVolumeSpecName "kube-api-access-r2pvn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:41:35.300403 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:41:35.300386    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7643b91f-b8d8-4e7a-aefb-8fb2531b7029-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "7643b91f-b8d8-4e7a-aefb-8fb2531b7029" (UID: "7643b91f-b8d8-4e7a-aefb-8fb2531b7029"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:41:35.399312 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:41:35.399266    2558 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7643b91f-b8d8-4e7a-aefb-8fb2531b7029-console-serving-cert\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:41:35.399312 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:41:35.399304    2558 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7643b91f-b8d8-4e7a-aefb-8fb2531b7029-trusted-ca-bundle\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:41:35.399312 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:41:35.399317    2558 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7643b91f-b8d8-4e7a-aefb-8fb2531b7029-console-oauth-config\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:41:35.399615 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:41:35.399329    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r2pvn\" (UniqueName: \"kubernetes.io/projected/7643b91f-b8d8-4e7a-aefb-8fb2531b7029-kube-api-access-r2pvn\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:41:35.399615 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:41:35.399342    2558 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7643b91f-b8d8-4e7a-aefb-8fb2531b7029-console-config\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:41:35.399615 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:41:35.399353    2558 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7643b91f-b8d8-4e7a-aefb-8fb2531b7029-oauth-serving-cert\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:41:35.944228 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:41:35.944127    2558 generic.go:356] "Generic (PLEG): container finished" podID="7643b91f-b8d8-4e7a-aefb-8fb2531b7029" containerID="1589a28bf9c203cf3d0ebef5e8cf0141feb7a0f4162af7d97580d879578ed93a" exitCode=0
Jul 27 19:41:35.944228 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:41:35.944189    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56ff6c45bc-kcmv8"
Jul 27 19:41:35.944228 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:41:35.944217    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56ff6c45bc-kcmv8" event={"ID":"7643b91f-b8d8-4e7a-aefb-8fb2531b7029","Type":"ContainerDied","Data":"1589a28bf9c203cf3d0ebef5e8cf0141feb7a0f4162af7d97580d879578ed93a"}
Jul 27 19:41:35.944845 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:41:35.944297    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56ff6c45bc-kcmv8" event={"ID":"7643b91f-b8d8-4e7a-aefb-8fb2531b7029","Type":"ContainerDied","Data":"4418609dedf616ad6dd0e0ef9889cb55a5388fcb8b325f334fef69667eea2f3c"}
Jul 27 19:41:35.944845 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:41:35.944319    2558 scope.go:117] "RemoveContainer" containerID="1589a28bf9c203cf3d0ebef5e8cf0141feb7a0f4162af7d97580d879578ed93a"
Jul 27 19:41:35.957264 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:41:35.957244    2558 scope.go:117] "RemoveContainer" containerID="1589a28bf9c203cf3d0ebef5e8cf0141feb7a0f4162af7d97580d879578ed93a"
Jul 27 19:41:35.957578 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:41:35.957560    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1589a28bf9c203cf3d0ebef5e8cf0141feb7a0f4162af7d97580d879578ed93a\": container with ID starting with 1589a28bf9c203cf3d0ebef5e8cf0141feb7a0f4162af7d97580d879578ed93a not found: ID does not exist" containerID="1589a28bf9c203cf3d0ebef5e8cf0141feb7a0f4162af7d97580d879578ed93a"
Jul 27 19:41:35.957635 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:41:35.957586    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1589a28bf9c203cf3d0ebef5e8cf0141feb7a0f4162af7d97580d879578ed93a"} err="failed to get container status \"1589a28bf9c203cf3d0ebef5e8cf0141feb7a0f4162af7d97580d879578ed93a\": rpc error: code = NotFound desc = could not find container \"1589a28bf9c203cf3d0ebef5e8cf0141feb7a0f4162af7d97580d879578ed93a\": container with ID starting with 1589a28bf9c203cf3d0ebef5e8cf0141feb7a0f4162af7d97580d879578ed93a not found: ID does not exist"
Jul 27 19:41:35.961397 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:41:35.961365    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56ff6c45bc-kcmv8"]
Jul 27 19:41:35.964578 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:41:35.964551    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-56ff6c45bc-kcmv8"]
Jul 27 19:41:37.676325 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:41:37.676291    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7643b91f-b8d8-4e7a-aefb-8fb2531b7029" path="/var/lib/kubelet/pods/7643b91f-b8d8-4e7a-aefb-8fb2531b7029/volumes"
Jul 27 19:42:09.469490 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:42:09.469453    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:42:11.502325 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:42:11.502292    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:42:51.054654 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:42:51.054552    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:42:51.159448 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:42:51.159391    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:42:51.425651 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:42:51.425618    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:42:57.372258 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:42:57.372223    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:42:57.430633 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:42:57.430603    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:42:57.491911 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:42:57.491866    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:42:57.601511 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:42:57.601475    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:42:57.733336 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:42:57.733225    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:43:03.543985 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:43:03.543950    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:43:03.644647 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:43:03.644611    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:43:53.119119 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:43:53.119081    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:46:27.608922 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:46:27.608893    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5dfc2_103bab23-bca3-472a-974b-7731f83c5bfb/ovn-acl-logging/0.log"
Jul 27 19:46:27.611921 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:46:27.611895    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5dfc2_103bab23-bca3-472a-974b-7731f83c5bfb/ovn-acl-logging/0.log"
Jul 27 19:46:27.974111 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:46:27.974074    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:46:51.304996 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:46:51.304953    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kg2ph"]
Jul 27 19:46:51.307368 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:46:51.305188    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kg2ph" podUID="09106494-3ae8-4b87-a583-ec42a938d603" containerName="manager" containerID="cri-o://d1bf1bfac8e3b5738089d2671860a181466323ce418d5bdbb1c616f7ea78f77d" gracePeriod=10
Jul 27 19:46:53.253468 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:46:53.253447    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kg2ph"
Jul 27 19:46:53.382083 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:46:53.382052    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2wk4l\" (UniqueName: \"kubernetes.io/projected/09106494-3ae8-4b87-a583-ec42a938d603-kube-api-access-2wk4l\") pod \"09106494-3ae8-4b87-a583-ec42a938d603\" (UID: \"09106494-3ae8-4b87-a583-ec42a938d603\") "
Jul 27 19:46:53.382247 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:46:53.382147    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/09106494-3ae8-4b87-a583-ec42a938d603-extensions-socket-volume\") pod \"09106494-3ae8-4b87-a583-ec42a938d603\" (UID: \"09106494-3ae8-4b87-a583-ec42a938d603\") "
Jul 27 19:46:53.382530 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:46:53.382504    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09106494-3ae8-4b87-a583-ec42a938d603-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "09106494-3ae8-4b87-a583-ec42a938d603" (UID: "09106494-3ae8-4b87-a583-ec42a938d603"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:46:53.384331 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:46:53.384307    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09106494-3ae8-4b87-a583-ec42a938d603-kube-api-access-2wk4l" (OuterVolumeSpecName: "kube-api-access-2wk4l") pod "09106494-3ae8-4b87-a583-ec42a938d603" (UID: "09106494-3ae8-4b87-a583-ec42a938d603"). InnerVolumeSpecName "kube-api-access-2wk4l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:46:53.483756 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:46:53.483726    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2wk4l\" (UniqueName: \"kubernetes.io/projected/09106494-3ae8-4b87-a583-ec42a938d603-kube-api-access-2wk4l\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:46:53.483756 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:46:53.483754    2558 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/09106494-3ae8-4b87-a583-ec42a938d603-extensions-socket-volume\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:46:53.654547 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:46:53.654449    2558 generic.go:356] "Generic (PLEG): container finished" podID="09106494-3ae8-4b87-a583-ec42a938d603" containerID="d1bf1bfac8e3b5738089d2671860a181466323ce418d5bdbb1c616f7ea78f77d" exitCode=0
Jul 27 19:46:53.654547 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:46:53.654511    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kg2ph"
Jul 27 19:46:53.654547 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:46:53.654536    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kg2ph" event={"ID":"09106494-3ae8-4b87-a583-ec42a938d603","Type":"ContainerDied","Data":"d1bf1bfac8e3b5738089d2671860a181466323ce418d5bdbb1c616f7ea78f77d"}
Jul 27 19:46:53.654817 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:46:53.654626    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kg2ph" event={"ID":"09106494-3ae8-4b87-a583-ec42a938d603","Type":"ContainerDied","Data":"34f2acab7d223469b2e07f9dd8c488cd67e24b1b8690979457e6a9494a082761"}
Jul 27 19:46:53.654817 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:46:53.654651    2558 scope.go:117] "RemoveContainer" containerID="d1bf1bfac8e3b5738089d2671860a181466323ce418d5bdbb1c616f7ea78f77d"
Jul 27 19:46:53.664975 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:46:53.664957    2558 scope.go:117] "RemoveContainer" containerID="d1bf1bfac8e3b5738089d2671860a181466323ce418d5bdbb1c616f7ea78f77d"
Jul 27 19:46:53.665232 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:46:53.665211    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1bf1bfac8e3b5738089d2671860a181466323ce418d5bdbb1c616f7ea78f77d\": container with ID starting with d1bf1bfac8e3b5738089d2671860a181466323ce418d5bdbb1c616f7ea78f77d not found: ID does not exist" containerID="d1bf1bfac8e3b5738089d2671860a181466323ce418d5bdbb1c616f7ea78f77d"
Jul 27 19:46:53.665334 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:46:53.665241    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1bf1bfac8e3b5738089d2671860a181466323ce418d5bdbb1c616f7ea78f77d"} err="failed to get container status \"d1bf1bfac8e3b5738089d2671860a181466323ce418d5bdbb1c616f7ea78f77d\": rpc error: code = NotFound desc = could not find container \"d1bf1bfac8e3b5738089d2671860a181466323ce418d5bdbb1c616f7ea78f77d\": container with ID starting with d1bf1bfac8e3b5738089d2671860a181466323ce418d5bdbb1c616f7ea78f77d not found: ID does not exist"
Jul 27 19:46:53.676250 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:46:53.676224    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kg2ph"]
Jul 27 19:46:53.680015 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:46:53.679994    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kg2ph"]
Jul 27 19:46:55.674814 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:46:55.674780    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09106494-3ae8-4b87-a583-ec42a938d603" path="/var/lib/kubelet/pods/09106494-3ae8-4b87-a583-ec42a938d603/volumes"
Jul 27 19:47:15.804183 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:47:15.804144    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:47:52.412605 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:47:52.412567    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dbbw8"]
Jul 27 19:47:52.413107 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:47:52.412912    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7643b91f-b8d8-4e7a-aefb-8fb2531b7029" containerName="console"
Jul 27 19:47:52.413107 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:47:52.412924    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="7643b91f-b8d8-4e7a-aefb-8fb2531b7029" containerName="console"
Jul 27 19:47:52.413107 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:47:52.412931    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09106494-3ae8-4b87-a583-ec42a938d603" containerName="manager"
Jul 27 19:47:52.413107 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:47:52.412936    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="09106494-3ae8-4b87-a583-ec42a938d603" containerName="manager"
Jul 27 19:47:52.413107 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:47:52.413004    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="09106494-3ae8-4b87-a583-ec42a938d603" containerName="manager"
Jul 27 19:47:52.413107 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:47:52.413011    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="7643b91f-b8d8-4e7a-aefb-8fb2531b7029" containerName="console"
Jul 27 19:47:52.417403 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:47:52.417384    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dbbw8"
Jul 27 19:47:52.419719 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:47:52.419699    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-htjbz\""
Jul 27 19:47:52.429102 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:47:52.429081    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dbbw8"]
Jul 27 19:47:52.597194 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:47:52.597165    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d794dacf-50aa-4a2c-91aa-d1fae69da1ae-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-dbbw8\" (UID: \"d794dacf-50aa-4a2c-91aa-d1fae69da1ae\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dbbw8"
Jul 27 19:47:52.597372 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:47:52.597219    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2xf8\" (UniqueName: \"kubernetes.io/projected/d794dacf-50aa-4a2c-91aa-d1fae69da1ae-kube-api-access-q2xf8\") pod \"kuadrant-operator-controller-manager-86878c769f-dbbw8\" (UID: \"d794dacf-50aa-4a2c-91aa-d1fae69da1ae\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dbbw8"
Jul 27 19:47:52.698117 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:47:52.698044    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2xf8\" (UniqueName: \"kubernetes.io/projected/d794dacf-50aa-4a2c-91aa-d1fae69da1ae-kube-api-access-q2xf8\") pod \"kuadrant-operator-controller-manager-86878c769f-dbbw8\" (UID: \"d794dacf-50aa-4a2c-91aa-d1fae69da1ae\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dbbw8"
Jul 27 19:47:52.698232 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:47:52.698121    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d794dacf-50aa-4a2c-91aa-d1fae69da1ae-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-dbbw8\" (UID: \"d794dacf-50aa-4a2c-91aa-d1fae69da1ae\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dbbw8"
Jul 27 19:47:52.698470 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:47:52.698451    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d794dacf-50aa-4a2c-91aa-d1fae69da1ae-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-dbbw8\" (UID: \"d794dacf-50aa-4a2c-91aa-d1fae69da1ae\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dbbw8"
Jul 27 19:47:52.712150 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:47:52.712129    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2xf8\" (UniqueName: \"kubernetes.io/projected/d794dacf-50aa-4a2c-91aa-d1fae69da1ae-kube-api-access-q2xf8\") pod \"kuadrant-operator-controller-manager-86878c769f-dbbw8\" (UID: \"d794dacf-50aa-4a2c-91aa-d1fae69da1ae\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dbbw8"
Jul 27 19:47:52.728006 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:47:52.727986    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dbbw8"
Jul 27 19:47:52.868837 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:47:52.868809    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dbbw8"]
Jul 27 19:47:52.870810 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:47:52.870788    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd794dacf_50aa_4a2c_91aa_d1fae69da1ae.slice/crio-e336b31a9bb218973cf731962b4d44e8935de3e6d35c7fb7cf74c394711e0742 WatchSource:0}: Error finding container e336b31a9bb218973cf731962b4d44e8935de3e6d35c7fb7cf74c394711e0742: Status 404 returned error can't find the container with id e336b31a9bb218973cf731962b4d44e8935de3e6d35c7fb7cf74c394711e0742
Jul 27 19:47:52.955459 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:47:52.955382    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dbbw8" event={"ID":"d794dacf-50aa-4a2c-91aa-d1fae69da1ae","Type":"ContainerStarted","Data":"7ecd9f32f1d11d2c3b5f7b42b9e6f7fd647ad8fcc5ca3742fb97ef4e86ee4665"}
Jul 27 19:47:52.955459 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:47:52.955455    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dbbw8" event={"ID":"d794dacf-50aa-4a2c-91aa-d1fae69da1ae","Type":"ContainerStarted","Data":"e336b31a9bb218973cf731962b4d44e8935de3e6d35c7fb7cf74c394711e0742"}
Jul 27 19:47:52.955626 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:47:52.955517    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dbbw8"
Jul 27 19:47:52.974773 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:47:52.974721    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dbbw8" podStartSLOduration=0.974705808 podStartE2EDuration="974.705808ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:47:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:47:52.973175053 +0000 UTC m=+1285.845367056" watchObservedRunningTime="2026-07-27 19:47:52.974705808 +0000 UTC m=+1285.846897782"
Jul 27 19:48:03.963131 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:48:03.963102    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dbbw8"
Jul 27 19:48:32.211403 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:48:32.211359    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:48:32.273284 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:48:32.273243    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:50:50.624661 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:50:50.624582    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-3ab3c697-kserve-68d4dc58bc-hmp46"]
Jul 27 19:50:50.627999 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:50:50.627983    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-3ab3c697-kserve-68d4dc58bc-hmp46"
Jul 27 19:50:50.630315 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:50:50.630284    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-3ab3c697-kserve-self-signed-certs\""
Jul 27 19:50:50.641773 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:50:50.641745    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-3ab3c697-kserve-68d4dc58bc-hmp46"]
Jul 27 19:50:50.682020 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:50:50.681990    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7d5cafe-1b6b-401b-958f-fd77224686c2-tmp-dir\") pod \"e2e-same-id-a-3ab3c697-kserve-68d4dc58bc-hmp46\" (UID: \"d7d5cafe-1b6b-401b-958f-fd77224686c2\") " pod="llm/e2e-same-id-a-3ab3c697-kserve-68d4dc58bc-hmp46"
Jul 27 19:50:50.682166 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:50:50.682039    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7d5cafe-1b6b-401b-958f-fd77224686c2-tls-certs\") pod \"e2e-same-id-a-3ab3c697-kserve-68d4dc58bc-hmp46\" (UID: \"d7d5cafe-1b6b-401b-958f-fd77224686c2\") " pod="llm/e2e-same-id-a-3ab3c697-kserve-68d4dc58bc-hmp46"
Jul 27 19:50:50.682166 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:50:50.682064    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d7d5cafe-1b6b-401b-958f-fd77224686c2-home\") pod \"e2e-same-id-a-3ab3c697-kserve-68d4dc58bc-hmp46\" (UID: \"d7d5cafe-1b6b-401b-958f-fd77224686c2\") " pod="llm/e2e-same-id-a-3ab3c697-kserve-68d4dc58bc-hmp46"
Jul 27 19:50:50.682166 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:50:50.682082    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qctxp\" (UniqueName: \"kubernetes.io/projected/d7d5cafe-1b6b-401b-958f-fd77224686c2-kube-api-access-qctxp\") pod \"e2e-same-id-a-3ab3c697-kserve-68d4dc58bc-hmp46\" (UID: \"d7d5cafe-1b6b-401b-958f-fd77224686c2\") " pod="llm/e2e-same-id-a-3ab3c697-kserve-68d4dc58bc-hmp46"
Jul 27 19:50:50.682166 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:50:50.682114    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d7d5cafe-1b6b-401b-958f-fd77224686c2-model-cache\") pod \"e2e-same-id-a-3ab3c697-kserve-68d4dc58bc-hmp46\" (UID: \"d7d5cafe-1b6b-401b-958f-fd77224686c2\") " pod="llm/e2e-same-id-a-3ab3c697-kserve-68d4dc58bc-hmp46"
Jul 27 19:50:50.682166 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:50:50.682142    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d7d5cafe-1b6b-401b-958f-fd77224686c2-dshm\") pod \"e2e-same-id-a-3ab3c697-kserve-68d4dc58bc-hmp46\" (UID: \"d7d5cafe-1b6b-401b-958f-fd77224686c2\") " pod="llm/e2e-same-id-a-3ab3c697-kserve-68d4dc58bc-hmp46"
Jul 27 19:50:50.762651 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:50:50.762624    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:50:50.782842 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:50:50.782813    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7d5cafe-1b6b-401b-958f-fd77224686c2-tmp-dir\") pod \"e2e-same-id-a-3ab3c697-kserve-68d4dc58bc-hmp46\" (UID: \"d7d5cafe-1b6b-401b-958f-fd77224686c2\") " pod="llm/e2e-same-id-a-3ab3c697-kserve-68d4dc58bc-hmp46"
Jul 27 19:50:50.782986 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:50:50.782867    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7d5cafe-1b6b-401b-958f-fd77224686c2-tls-certs\") pod \"e2e-same-id-a-3ab3c697-kserve-68d4dc58bc-hmp46\" (UID: \"d7d5cafe-1b6b-401b-958f-fd77224686c2\") " pod="llm/e2e-same-id-a-3ab3c697-kserve-68d4dc58bc-hmp46"
Jul 27 19:50:50.782986 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:50:50.782886    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d7d5cafe-1b6b-401b-958f-fd77224686c2-home\") pod \"e2e-same-id-a-3ab3c697-kserve-68d4dc58bc-hmp46\" (UID: \"d7d5cafe-1b6b-401b-958f-fd77224686c2\") " pod="llm/e2e-same-id-a-3ab3c697-kserve-68d4dc58bc-hmp46"
Jul 27 19:50:50.783093 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:50:50.782996    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qctxp\" (UniqueName: \"kubernetes.io/projected/d7d5cafe-1b6b-401b-958f-fd77224686c2-kube-api-access-qctxp\") pod \"e2e-same-id-a-3ab3c697-kserve-68d4dc58bc-hmp46\" (UID: \"d7d5cafe-1b6b-401b-958f-fd77224686c2\") " pod="llm/e2e-same-id-a-3ab3c697-kserve-68d4dc58bc-hmp46"
Jul 27 19:50:50.783093 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:50:50.783056    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d7d5cafe-1b6b-401b-958f-fd77224686c2-model-cache\") pod \"e2e-same-id-a-3ab3c697-kserve-68d4dc58bc-hmp46\" (UID: \"d7d5cafe-1b6b-401b-958f-fd77224686c2\") " pod="llm/e2e-same-id-a-3ab3c697-kserve-68d4dc58bc-hmp46"
Jul 27 19:50:50.783093 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:50:50.783089    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d7d5cafe-1b6b-401b-958f-fd77224686c2-dshm\") pod \"e2e-same-id-a-3ab3c697-kserve-68d4dc58bc-hmp46\" (UID: \"d7d5cafe-1b6b-401b-958f-fd77224686c2\") " pod="llm/e2e-same-id-a-3ab3c697-kserve-68d4dc58bc-hmp46"
Jul 27 19:50:50.783257 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:50:50.783243    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d7d5cafe-1b6b-401b-958f-fd77224686c2-home\") pod \"e2e-same-id-a-3ab3c697-kserve-68d4dc58bc-hmp46\" (UID: \"d7d5cafe-1b6b-401b-958f-fd77224686c2\") " pod="llm/e2e-same-id-a-3ab3c697-kserve-68d4dc58bc-hmp46"
Jul 27 19:50:50.783335 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:50:50.783291    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7d5cafe-1b6b-401b-958f-fd77224686c2-tmp-dir\") pod \"e2e-same-id-a-3ab3c697-kserve-68d4dc58bc-hmp46\" (UID: \"d7d5cafe-1b6b-401b-958f-fd77224686c2\") " pod="llm/e2e-same-id-a-3ab3c697-kserve-68d4dc58bc-hmp46"
Jul 27 19:50:50.783475 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:50:50.783431    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d7d5cafe-1b6b-401b-958f-fd77224686c2-model-cache\") pod \"e2e-same-id-a-3ab3c697-kserve-68d4dc58bc-hmp46\" (UID: \"d7d5cafe-1b6b-401b-958f-fd77224686c2\") " pod="llm/e2e-same-id-a-3ab3c697-kserve-68d4dc58bc-hmp46"
Jul 27 19:50:50.785331 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:50:50.785303    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d7d5cafe-1b6b-401b-958f-fd77224686c2-dshm\") pod \"e2e-same-id-a-3ab3c697-kserve-68d4dc58bc-hmp46\" (UID: \"d7d5cafe-1b6b-401b-958f-fd77224686c2\") " pod="llm/e2e-same-id-a-3ab3c697-kserve-68d4dc58bc-hmp46"
Jul 27 19:50:50.785535 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:50:50.785518    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7d5cafe-1b6b-401b-958f-fd77224686c2-tls-certs\") pod \"e2e-same-id-a-3ab3c697-kserve-68d4dc58bc-hmp46\" (UID: \"d7d5cafe-1b6b-401b-958f-fd77224686c2\") " pod="llm/e2e-same-id-a-3ab3c697-kserve-68d4dc58bc-hmp46"
Jul 27 19:50:50.791578 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:50:50.791556    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qctxp\" (UniqueName: \"kubernetes.io/projected/d7d5cafe-1b6b-401b-958f-fd77224686c2-kube-api-access-qctxp\") pod \"e2e-same-id-a-3ab3c697-kserve-68d4dc58bc-hmp46\" (UID: \"d7d5cafe-1b6b-401b-958f-fd77224686c2\") " pod="llm/e2e-same-id-a-3ab3c697-kserve-68d4dc58bc-hmp46"
Jul 27 19:50:50.937210 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:50:50.937121    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-3ab3c697-kserve-68d4dc58bc-hmp46"
Jul 27 19:50:51.088404 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:50:51.088379    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-3ab3c697-kserve-68d4dc58bc-hmp46"]
Jul 27 19:50:51.090011 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:50:51.089981    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd7d5cafe_1b6b_401b_958f_fd77224686c2.slice/crio-f6dc7a5d69880cf488c989fbe92c9b617a1b5734dbc6c1bdb0515e683f383a16 WatchSource:0}: Error finding container f6dc7a5d69880cf488c989fbe92c9b617a1b5734dbc6c1bdb0515e683f383a16: Status 404 returned error can't find the container with id f6dc7a5d69880cf488c989fbe92c9b617a1b5734dbc6c1bdb0515e683f383a16
Jul 27 19:50:51.911017 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:50:51.910980    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-3ab3c697-kserve-68d4dc58bc-hmp46" event={"ID":"d7d5cafe-1b6b-401b-958f-fd77224686c2","Type":"ContainerStarted","Data":"ce1ddb820ae216a025c8f4bedb7c356576858683832895096187bee6351f4f8d"}
Jul 27 19:50:51.911397 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:50:51.911027    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-3ab3c697-kserve-68d4dc58bc-hmp46" event={"ID":"d7d5cafe-1b6b-401b-958f-fd77224686c2","Type":"ContainerStarted","Data":"f6dc7a5d69880cf488c989fbe92c9b617a1b5734dbc6c1bdb0515e683f383a16"}
Jul 27 19:50:51.930350 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:50:51.930303    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-3ab3c697-kserve-68d4dc58bc-hmp46" podStartSLOduration=1.9302894739999998 podStartE2EDuration="1.930289474s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:50:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:50:51.928018259 +0000 UTC m=+1464.800210228" watchObservedRunningTime="2026-07-27 19:50:51.930289474 +0000 UTC m=+1464.802481448"
Jul 27 19:51:00.938291 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:00.938194    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-3ab3c697-kserve-68d4dc58bc-hmp46"
Jul 27 19:51:00.938291 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:00.938263    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-3ab3c697-kserve-68d4dc58bc-hmp46"
Jul 27 19:51:00.950938 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:00.950910    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-3ab3c697-kserve-68d4dc58bc-hmp46"
Jul 27 19:51:00.972894 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:00.972865    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-3ab3c697-kserve-68d4dc58bc-hmp46"
Jul 27 19:51:09.822521 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:09.822486    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-3ab3c697-kserve-5999486c47-xqdf4"]
Jul 27 19:51:09.825973 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:09.825952    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-3ab3c697-kserve-5999486c47-xqdf4"
Jul 27 19:51:09.828587 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:09.828560    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-3ab3c697-kserve-self-signed-certs\""
Jul 27 19:51:09.834590 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:09.834553    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-3ab3c697-kserve-5999486c47-xqdf4"]
Jul 27 19:51:09.861150 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:09.861096    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:51:09.959285 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:09.959252    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ada3592b-a1a9-42b2-a854-6cc7d68b6135-tmp-dir\") pod \"e2e-same-id-b-3ab3c697-kserve-5999486c47-xqdf4\" (UID: \"ada3592b-a1a9-42b2-a854-6cc7d68b6135\") " pod="llm/e2e-same-id-b-3ab3c697-kserve-5999486c47-xqdf4"
Jul 27 19:51:09.959539 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:09.959329    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ada3592b-a1a9-42b2-a854-6cc7d68b6135-home\") pod \"e2e-same-id-b-3ab3c697-kserve-5999486c47-xqdf4\" (UID: \"ada3592b-a1a9-42b2-a854-6cc7d68b6135\") " pod="llm/e2e-same-id-b-3ab3c697-kserve-5999486c47-xqdf4"
Jul 27 19:51:09.959539 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:09.959355    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ada3592b-a1a9-42b2-a854-6cc7d68b6135-tls-certs\") pod \"e2e-same-id-b-3ab3c697-kserve-5999486c47-xqdf4\" (UID: \"ada3592b-a1a9-42b2-a854-6cc7d68b6135\") " pod="llm/e2e-same-id-b-3ab3c697-kserve-5999486c47-xqdf4"
Jul 27 19:51:09.959539 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:09.959377    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ada3592b-a1a9-42b2-a854-6cc7d68b6135-model-cache\") pod \"e2e-same-id-b-3ab3c697-kserve-5999486c47-xqdf4\" (UID: \"ada3592b-a1a9-42b2-a854-6cc7d68b6135\") " pod="llm/e2e-same-id-b-3ab3c697-kserve-5999486c47-xqdf4"
Jul 27 19:51:09.959539 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:09.959461    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ada3592b-a1a9-42b2-a854-6cc7d68b6135-dshm\") pod \"e2e-same-id-b-3ab3c697-kserve-5999486c47-xqdf4\" (UID: \"ada3592b-a1a9-42b2-a854-6cc7d68b6135\") " pod="llm/e2e-same-id-b-3ab3c697-kserve-5999486c47-xqdf4"
Jul 27 19:51:09.959539 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:09.959514    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzccx\" (UniqueName: \"kubernetes.io/projected/ada3592b-a1a9-42b2-a854-6cc7d68b6135-kube-api-access-tzccx\") pod \"e2e-same-id-b-3ab3c697-kserve-5999486c47-xqdf4\" (UID: \"ada3592b-a1a9-42b2-a854-6cc7d68b6135\") " pod="llm/e2e-same-id-b-3ab3c697-kserve-5999486c47-xqdf4"
Jul 27 19:51:10.060775 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:10.060728    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ada3592b-a1a9-42b2-a854-6cc7d68b6135-tmp-dir\") pod \"e2e-same-id-b-3ab3c697-kserve-5999486c47-xqdf4\" (UID: \"ada3592b-a1a9-42b2-a854-6cc7d68b6135\") " pod="llm/e2e-same-id-b-3ab3c697-kserve-5999486c47-xqdf4"
Jul 27 19:51:10.060993 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:10.060807    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ada3592b-a1a9-42b2-a854-6cc7d68b6135-home\") pod \"e2e-same-id-b-3ab3c697-kserve-5999486c47-xqdf4\" (UID: \"ada3592b-a1a9-42b2-a854-6cc7d68b6135\") " pod="llm/e2e-same-id-b-3ab3c697-kserve-5999486c47-xqdf4"
Jul 27 19:51:10.060993 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:10.060831    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ada3592b-a1a9-42b2-a854-6cc7d68b6135-tls-certs\") pod \"e2e-same-id-b-3ab3c697-kserve-5999486c47-xqdf4\" (UID: \"ada3592b-a1a9-42b2-a854-6cc7d68b6135\") " pod="llm/e2e-same-id-b-3ab3c697-kserve-5999486c47-xqdf4"
Jul 27 19:51:10.060993 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:10.060854    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ada3592b-a1a9-42b2-a854-6cc7d68b6135-model-cache\") pod \"e2e-same-id-b-3ab3c697-kserve-5999486c47-xqdf4\" (UID: \"ada3592b-a1a9-42b2-a854-6cc7d68b6135\") " pod="llm/e2e-same-id-b-3ab3c697-kserve-5999486c47-xqdf4"
Jul 27 19:51:10.060993 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:10.060924    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ada3592b-a1a9-42b2-a854-6cc7d68b6135-dshm\") pod \"e2e-same-id-b-3ab3c697-kserve-5999486c47-xqdf4\" (UID: \"ada3592b-a1a9-42b2-a854-6cc7d68b6135\") " pod="llm/e2e-same-id-b-3ab3c697-kserve-5999486c47-xqdf4"
Jul 27 19:51:10.061295 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:10.061012    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzccx\" (UniqueName: \"kubernetes.io/projected/ada3592b-a1a9-42b2-a854-6cc7d68b6135-kube-api-access-tzccx\") pod \"e2e-same-id-b-3ab3c697-kserve-5999486c47-xqdf4\" (UID: \"ada3592b-a1a9-42b2-a854-6cc7d68b6135\") " pod="llm/e2e-same-id-b-3ab3c697-kserve-5999486c47-xqdf4"
Jul 27 19:51:10.061295 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:10.061203    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ada3592b-a1a9-42b2-a854-6cc7d68b6135-tmp-dir\") pod \"e2e-same-id-b-3ab3c697-kserve-5999486c47-xqdf4\" (UID: \"ada3592b-a1a9-42b2-a854-6cc7d68b6135\") " pod="llm/e2e-same-id-b-3ab3c697-kserve-5999486c47-xqdf4"
Jul 27 19:51:10.061295 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:10.061257    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ada3592b-a1a9-42b2-a854-6cc7d68b6135-model-cache\") pod \"e2e-same-id-b-3ab3c697-kserve-5999486c47-xqdf4\" (UID: \"ada3592b-a1a9-42b2-a854-6cc7d68b6135\") " pod="llm/e2e-same-id-b-3ab3c697-kserve-5999486c47-xqdf4"
Jul 27 19:51:10.061429 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:10.061294    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ada3592b-a1a9-42b2-a854-6cc7d68b6135-home\") pod \"e2e-same-id-b-3ab3c697-kserve-5999486c47-xqdf4\" (UID: \"ada3592b-a1a9-42b2-a854-6cc7d68b6135\") " pod="llm/e2e-same-id-b-3ab3c697-kserve-5999486c47-xqdf4"
Jul 27 19:51:10.063358 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:10.063327    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ada3592b-a1a9-42b2-a854-6cc7d68b6135-dshm\") pod \"e2e-same-id-b-3ab3c697-kserve-5999486c47-xqdf4\" (UID: \"ada3592b-a1a9-42b2-a854-6cc7d68b6135\") " pod="llm/e2e-same-id-b-3ab3c697-kserve-5999486c47-xqdf4"
Jul 27 19:51:10.063646 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:10.063630    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ada3592b-a1a9-42b2-a854-6cc7d68b6135-tls-certs\") pod \"e2e-same-id-b-3ab3c697-kserve-5999486c47-xqdf4\" (UID: \"ada3592b-a1a9-42b2-a854-6cc7d68b6135\") " pod="llm/e2e-same-id-b-3ab3c697-kserve-5999486c47-xqdf4"
Jul 27 19:51:10.069285 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:10.069263    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzccx\" (UniqueName: \"kubernetes.io/projected/ada3592b-a1a9-42b2-a854-6cc7d68b6135-kube-api-access-tzccx\") pod \"e2e-same-id-b-3ab3c697-kserve-5999486c47-xqdf4\" (UID: \"ada3592b-a1a9-42b2-a854-6cc7d68b6135\") " pod="llm/e2e-same-id-b-3ab3c697-kserve-5999486c47-xqdf4"
Jul 27 19:51:10.138564 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:10.138518    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-3ab3c697-kserve-5999486c47-xqdf4"
Jul 27 19:51:10.280495 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:10.280469    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-3ab3c697-kserve-5999486c47-xqdf4"]
Jul 27 19:51:10.282203 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:51:10.282177    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podada3592b_a1a9_42b2_a854_6cc7d68b6135.slice/crio-cc1d47bf5c2f5de541a46a6a05de389817e379055659dd8a832994df0cbc57ee WatchSource:0}: Error finding container cc1d47bf5c2f5de541a46a6a05de389817e379055659dd8a832994df0cbc57ee: Status 404 returned error can't find the container with id cc1d47bf5c2f5de541a46a6a05de389817e379055659dd8a832994df0cbc57ee
Jul 27 19:51:11.019472 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:11.019429    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-3ab3c697-kserve-5999486c47-xqdf4" event={"ID":"ada3592b-a1a9-42b2-a854-6cc7d68b6135","Type":"ContainerStarted","Data":"6355df9a84c7fef1ee7c4390a2e12d54804d4e0e8b256ce9599a46bfc0a2a08a"}
Jul 27 19:51:11.019472 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:11.019479    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-3ab3c697-kserve-5999486c47-xqdf4" event={"ID":"ada3592b-a1a9-42b2-a854-6cc7d68b6135","Type":"ContainerStarted","Data":"cc1d47bf5c2f5de541a46a6a05de389817e379055659dd8a832994df0cbc57ee"}
Jul 27 19:51:11.042811 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:11.042748    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-3ab3c697-kserve-5999486c47-xqdf4" podStartSLOduration=2.04273089 podStartE2EDuration="2.04273089s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:51:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:51:11.040206676 +0000 UTC m=+1483.912398650" watchObservedRunningTime="2026-07-27 19:51:11.04273089 +0000 UTC m=+1483.914922864"
Jul 27 19:51:20.139249 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:20.139198    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-3ab3c697-kserve-5999486c47-xqdf4"
Jul 27 19:51:20.139811 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:20.139269    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-3ab3c697-kserve-5999486c47-xqdf4"
Jul 27 19:51:20.152115 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:20.152083    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-3ab3c697-kserve-5999486c47-xqdf4"
Jul 27 19:51:21.087617 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:21.087582    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-3ab3c697-kserve-5999486c47-xqdf4"
Jul 27 19:51:27.650754 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:27.650724    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5dfc2_103bab23-bca3-472a-974b-7731f83c5bfb/ovn-acl-logging/0.log"
Jul 27 19:51:27.655668 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:27.655631    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5dfc2_103bab23-bca3-472a-974b-7731f83c5bfb/ovn-acl-logging/0.log"
Jul 27 19:51:39.875601 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:39.875555    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-3ab3c697-kserve-68d4dc58bc-hmp46"]
Jul 27 19:51:40.751496 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:40.751463    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:51:40.819199 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:40.819159    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-3ab3c697-kserve-5999486c47-xqdf4"]
Jul 27 19:51:40.839023 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:40.838991    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:51:54.896721 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:54.896660    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-3ab3c697-kserve-68d4dc58bc-hmp46" podUID="d7d5cafe-1b6b-401b-958f-fd77224686c2" containerName="main" containerID="cri-o://ce1ddb820ae216a025c8f4bedb7c356576858683832895096187bee6351f4f8d" gracePeriod=45
Jul 27 19:51:55.147134 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:55.147074    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-3ab3c697-kserve-68d4dc58bc-hmp46"
Jul 27 19:51:55.271067 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:55.271033    2558 generic.go:356] "Generic (PLEG): container finished" podID="d7d5cafe-1b6b-401b-958f-fd77224686c2" containerID="ce1ddb820ae216a025c8f4bedb7c356576858683832895096187bee6351f4f8d" exitCode=0
Jul 27 19:51:55.271244 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:55.271077    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-3ab3c697-kserve-68d4dc58bc-hmp46"
Jul 27 19:51:55.271244 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:55.271114    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-3ab3c697-kserve-68d4dc58bc-hmp46" event={"ID":"d7d5cafe-1b6b-401b-958f-fd77224686c2","Type":"ContainerDied","Data":"ce1ddb820ae216a025c8f4bedb7c356576858683832895096187bee6351f4f8d"}
Jul 27 19:51:55.271244 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:55.271158    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-3ab3c697-kserve-68d4dc58bc-hmp46" event={"ID":"d7d5cafe-1b6b-401b-958f-fd77224686c2","Type":"ContainerDied","Data":"f6dc7a5d69880cf488c989fbe92c9b617a1b5734dbc6c1bdb0515e683f383a16"}
Jul 27 19:51:55.271244 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:55.271177    2558 scope.go:117] "RemoveContainer" containerID="ce1ddb820ae216a025c8f4bedb7c356576858683832895096187bee6351f4f8d"
Jul 27 19:51:55.279861 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:55.279843    2558 scope.go:117] "RemoveContainer" containerID="ce1ddb820ae216a025c8f4bedb7c356576858683832895096187bee6351f4f8d"
Jul 27 19:51:55.280104 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:51:55.280087    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce1ddb820ae216a025c8f4bedb7c356576858683832895096187bee6351f4f8d\": container with ID starting with ce1ddb820ae216a025c8f4bedb7c356576858683832895096187bee6351f4f8d not found: ID does not exist" containerID="ce1ddb820ae216a025c8f4bedb7c356576858683832895096187bee6351f4f8d"
Jul 27 19:51:55.280155 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:55.280114    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce1ddb820ae216a025c8f4bedb7c356576858683832895096187bee6351f4f8d"} err="failed to get container status \"ce1ddb820ae216a025c8f4bedb7c356576858683832895096187bee6351f4f8d\": rpc error: code = NotFound desc = could not find container \"ce1ddb820ae216a025c8f4bedb7c356576858683832895096187bee6351f4f8d\": container with ID starting with ce1ddb820ae216a025c8f4bedb7c356576858683832895096187bee6351f4f8d not found: ID does not exist"
Jul 27 19:51:55.289527 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:55.289508    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d7d5cafe-1b6b-401b-958f-fd77224686c2-home\") pod \"d7d5cafe-1b6b-401b-958f-fd77224686c2\" (UID: \"d7d5cafe-1b6b-401b-958f-fd77224686c2\") "
Jul 27 19:51:55.289626 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:55.289562    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7d5cafe-1b6b-401b-958f-fd77224686c2-tmp-dir\") pod \"d7d5cafe-1b6b-401b-958f-fd77224686c2\" (UID: \"d7d5cafe-1b6b-401b-958f-fd77224686c2\") "
Jul 27 19:51:55.289626 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:55.289589    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d7d5cafe-1b6b-401b-958f-fd77224686c2-model-cache\") pod \"d7d5cafe-1b6b-401b-958f-fd77224686c2\" (UID: \"d7d5cafe-1b6b-401b-958f-fd77224686c2\") "
Jul 27 19:51:55.289730 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:55.289633    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qctxp\" (UniqueName: \"kubernetes.io/projected/d7d5cafe-1b6b-401b-958f-fd77224686c2-kube-api-access-qctxp\") pod \"d7d5cafe-1b6b-401b-958f-fd77224686c2\" (UID: \"d7d5cafe-1b6b-401b-958f-fd77224686c2\") "
Jul 27 19:51:55.289730 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:55.289658    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7d5cafe-1b6b-401b-958f-fd77224686c2-tls-certs\") pod \"d7d5cafe-1b6b-401b-958f-fd77224686c2\" (UID: \"d7d5cafe-1b6b-401b-958f-fd77224686c2\") "
Jul 27 19:51:55.289730 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:55.289685    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d7d5cafe-1b6b-401b-958f-fd77224686c2-dshm\") pod \"d7d5cafe-1b6b-401b-958f-fd77224686c2\" (UID: \"d7d5cafe-1b6b-401b-958f-fd77224686c2\") "
Jul 27 19:51:55.289880 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:55.289802    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7d5cafe-1b6b-401b-958f-fd77224686c2-home" (OuterVolumeSpecName: "home") pod "d7d5cafe-1b6b-401b-958f-fd77224686c2" (UID: "d7d5cafe-1b6b-401b-958f-fd77224686c2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:51:55.289880 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:55.289862    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7d5cafe-1b6b-401b-958f-fd77224686c2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d7d5cafe-1b6b-401b-958f-fd77224686c2" (UID: "d7d5cafe-1b6b-401b-958f-fd77224686c2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:51:55.289968 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:55.289876    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7d5cafe-1b6b-401b-958f-fd77224686c2-model-cache" (OuterVolumeSpecName: "model-cache") pod "d7d5cafe-1b6b-401b-958f-fd77224686c2" (UID: "d7d5cafe-1b6b-401b-958f-fd77224686c2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:51:55.290006 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:55.289996    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d7d5cafe-1b6b-401b-958f-fd77224686c2-home\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:51:55.290042 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:55.290014    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7d5cafe-1b6b-401b-958f-fd77224686c2-tmp-dir\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:51:55.290042 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:55.290029    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d7d5cafe-1b6b-401b-958f-fd77224686c2-model-cache\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:51:55.291890 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:55.291861    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7d5cafe-1b6b-401b-958f-fd77224686c2-dshm" (OuterVolumeSpecName: "dshm") pod "d7d5cafe-1b6b-401b-958f-fd77224686c2" (UID: "d7d5cafe-1b6b-401b-958f-fd77224686c2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:51:55.291890 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:55.291863    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7d5cafe-1b6b-401b-958f-fd77224686c2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d7d5cafe-1b6b-401b-958f-fd77224686c2" (UID: "d7d5cafe-1b6b-401b-958f-fd77224686c2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:51:55.292020 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:55.291906    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7d5cafe-1b6b-401b-958f-fd77224686c2-kube-api-access-qctxp" (OuterVolumeSpecName: "kube-api-access-qctxp") pod "d7d5cafe-1b6b-401b-958f-fd77224686c2" (UID: "d7d5cafe-1b6b-401b-958f-fd77224686c2"). InnerVolumeSpecName "kube-api-access-qctxp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:51:55.390693 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:55.390667    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qctxp\" (UniqueName: \"kubernetes.io/projected/d7d5cafe-1b6b-401b-958f-fd77224686c2-kube-api-access-qctxp\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:51:55.390693 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:55.390693    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7d5cafe-1b6b-401b-958f-fd77224686c2-tls-certs\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:51:55.390866 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:55.390705    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d7d5cafe-1b6b-401b-958f-fd77224686c2-dshm\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:51:55.592755 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:55.592724    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-3ab3c697-kserve-68d4dc58bc-hmp46"]
Jul 27 19:51:55.596057 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:55.596033    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-3ab3c697-kserve-68d4dc58bc-hmp46"]
Jul 27 19:51:55.674368 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:55.674336    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7d5cafe-1b6b-401b-958f-fd77224686c2" path="/var/lib/kubelet/pods/d7d5cafe-1b6b-401b-958f-fd77224686c2/volumes"
Jul 27 19:51:55.840253 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:55.840161    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-3ab3c697-kserve-5999486c47-xqdf4" podUID="ada3592b-a1a9-42b2-a854-6cc7d68b6135" containerName="main" containerID="cri-o://6355df9a84c7fef1ee7c4390a2e12d54804d4e0e8b256ce9599a46bfc0a2a08a" gracePeriod=45
Jul 27 19:51:56.092367 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:56.092310    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-3ab3c697-kserve-5999486c47-xqdf4"
Jul 27 19:51:56.197498 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:56.197470    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ada3592b-a1a9-42b2-a854-6cc7d68b6135-tls-certs\") pod \"ada3592b-a1a9-42b2-a854-6cc7d68b6135\" (UID: \"ada3592b-a1a9-42b2-a854-6cc7d68b6135\") "
Jul 27 19:51:56.197498 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:56.197503    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ada3592b-a1a9-42b2-a854-6cc7d68b6135-dshm\") pod \"ada3592b-a1a9-42b2-a854-6cc7d68b6135\" (UID: \"ada3592b-a1a9-42b2-a854-6cc7d68b6135\") "
Jul 27 19:51:56.197685 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:56.197542    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tzccx\" (UniqueName: \"kubernetes.io/projected/ada3592b-a1a9-42b2-a854-6cc7d68b6135-kube-api-access-tzccx\") pod \"ada3592b-a1a9-42b2-a854-6cc7d68b6135\" (UID: \"ada3592b-a1a9-42b2-a854-6cc7d68b6135\") "
Jul 27 19:51:56.197685 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:56.197621    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ada3592b-a1a9-42b2-a854-6cc7d68b6135-home\") pod \"ada3592b-a1a9-42b2-a854-6cc7d68b6135\" (UID: \"ada3592b-a1a9-42b2-a854-6cc7d68b6135\") "
Jul 27 19:51:56.197685 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:56.197655    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ada3592b-a1a9-42b2-a854-6cc7d68b6135-tmp-dir\") pod \"ada3592b-a1a9-42b2-a854-6cc7d68b6135\" (UID: \"ada3592b-a1a9-42b2-a854-6cc7d68b6135\") "
Jul 27 19:51:56.197850 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:56.197696    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ada3592b-a1a9-42b2-a854-6cc7d68b6135-model-cache\") pod \"ada3592b-a1a9-42b2-a854-6cc7d68b6135\" (UID: \"ada3592b-a1a9-42b2-a854-6cc7d68b6135\") "
Jul 27 19:51:56.197972 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:56.197945    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ada3592b-a1a9-42b2-a854-6cc7d68b6135-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ada3592b-a1a9-42b2-a854-6cc7d68b6135" (UID: "ada3592b-a1a9-42b2-a854-6cc7d68b6135"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:51:56.197972 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:56.197957    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ada3592b-a1a9-42b2-a854-6cc7d68b6135-home" (OuterVolumeSpecName: "home") pod "ada3592b-a1a9-42b2-a854-6cc7d68b6135" (UID: "ada3592b-a1a9-42b2-a854-6cc7d68b6135"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:51:56.198112 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:56.197980    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ada3592b-a1a9-42b2-a854-6cc7d68b6135-model-cache" (OuterVolumeSpecName: "model-cache") pod "ada3592b-a1a9-42b2-a854-6cc7d68b6135" (UID: "ada3592b-a1a9-42b2-a854-6cc7d68b6135"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:51:56.199742 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:56.199722    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ada3592b-a1a9-42b2-a854-6cc7d68b6135-dshm" (OuterVolumeSpecName: "dshm") pod "ada3592b-a1a9-42b2-a854-6cc7d68b6135" (UID: "ada3592b-a1a9-42b2-a854-6cc7d68b6135"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:51:56.199848 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:56.199830    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ada3592b-a1a9-42b2-a854-6cc7d68b6135-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ada3592b-a1a9-42b2-a854-6cc7d68b6135" (UID: "ada3592b-a1a9-42b2-a854-6cc7d68b6135"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:51:56.199901 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:56.199884    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ada3592b-a1a9-42b2-a854-6cc7d68b6135-kube-api-access-tzccx" (OuterVolumeSpecName: "kube-api-access-tzccx") pod "ada3592b-a1a9-42b2-a854-6cc7d68b6135" (UID: "ada3592b-a1a9-42b2-a854-6cc7d68b6135"). InnerVolumeSpecName "kube-api-access-tzccx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:51:56.279033 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:56.279000    2558 generic.go:356] "Generic (PLEG): container finished" podID="ada3592b-a1a9-42b2-a854-6cc7d68b6135" containerID="6355df9a84c7fef1ee7c4390a2e12d54804d4e0e8b256ce9599a46bfc0a2a08a" exitCode=0
Jul 27 19:51:56.279209 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:56.279059    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-3ab3c697-kserve-5999486c47-xqdf4"
Jul 27 19:51:56.279209 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:56.279083    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-3ab3c697-kserve-5999486c47-xqdf4" event={"ID":"ada3592b-a1a9-42b2-a854-6cc7d68b6135","Type":"ContainerDied","Data":"6355df9a84c7fef1ee7c4390a2e12d54804d4e0e8b256ce9599a46bfc0a2a08a"}
Jul 27 19:51:56.279209 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:56.279124    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-3ab3c697-kserve-5999486c47-xqdf4" event={"ID":"ada3592b-a1a9-42b2-a854-6cc7d68b6135","Type":"ContainerDied","Data":"cc1d47bf5c2f5de541a46a6a05de389817e379055659dd8a832994df0cbc57ee"}
Jul 27 19:51:56.279209 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:56.279142    2558 scope.go:117] "RemoveContainer" containerID="6355df9a84c7fef1ee7c4390a2e12d54804d4e0e8b256ce9599a46bfc0a2a08a"
Jul 27 19:51:56.288567 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:56.288551    2558 scope.go:117] "RemoveContainer" containerID="6355df9a84c7fef1ee7c4390a2e12d54804d4e0e8b256ce9599a46bfc0a2a08a"
Jul 27 19:51:56.288802 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:51:56.288783    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6355df9a84c7fef1ee7c4390a2e12d54804d4e0e8b256ce9599a46bfc0a2a08a\": container with ID starting with 6355df9a84c7fef1ee7c4390a2e12d54804d4e0e8b256ce9599a46bfc0a2a08a not found: ID does not exist" containerID="6355df9a84c7fef1ee7c4390a2e12d54804d4e0e8b256ce9599a46bfc0a2a08a"
Jul 27 19:51:56.288853 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:56.288809    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6355df9a84c7fef1ee7c4390a2e12d54804d4e0e8b256ce9599a46bfc0a2a08a"} err="failed to get container status \"6355df9a84c7fef1ee7c4390a2e12d54804d4e0e8b256ce9599a46bfc0a2a08a\": rpc error: code = NotFound desc = could not find container \"6355df9a84c7fef1ee7c4390a2e12d54804d4e0e8b256ce9599a46bfc0a2a08a\": container with ID starting with 6355df9a84c7fef1ee7c4390a2e12d54804d4e0e8b256ce9599a46bfc0a2a08a not found: ID does not exist"
Jul 27 19:51:56.298856 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:56.298835    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tzccx\" (UniqueName: \"kubernetes.io/projected/ada3592b-a1a9-42b2-a854-6cc7d68b6135-kube-api-access-tzccx\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:51:56.298856 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:56.298856    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ada3592b-a1a9-42b2-a854-6cc7d68b6135-home\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:51:56.298998 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:56.298866    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ada3592b-a1a9-42b2-a854-6cc7d68b6135-tmp-dir\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:51:56.298998 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:56.298875    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ada3592b-a1a9-42b2-a854-6cc7d68b6135-model-cache\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:51:56.298998 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:56.298883    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ada3592b-a1a9-42b2-a854-6cc7d68b6135-tls-certs\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:51:56.298998 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:56.298890    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ada3592b-a1a9-42b2-a854-6cc7d68b6135-dshm\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:51:56.300058 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:56.300040    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-3ab3c697-kserve-5999486c47-xqdf4"]
Jul 27 19:51:56.303328 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:56.303300    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-3ab3c697-kserve-5999486c47-xqdf4"]
Jul 27 19:51:57.674814 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:51:57.674777    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ada3592b-a1a9-42b2-a854-6cc7d68b6135" path="/var/lib/kubelet/pods/ada3592b-a1a9-42b2-a854-6cc7d68b6135/volumes"
Jul 27 19:54:18.300268 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:54:18.300230    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:54:49.777337 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:54:49.777241    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:54:52.633244 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:54:52.633190    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:54:52.735582 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:54:52.735538    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:55:11.111250 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:55:11.111216    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:55:14.071712 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:55:14.071677    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:55:14.147916 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:55:14.147881    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:55:14.569093 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:55:14.569058    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:55:20.383518 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:55:20.383492    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:55:20.488018 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:55:20.487984    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:55:20.631004 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:55:20.630971    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:55:20.946018 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:55:20.945979    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:55:21.141354 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:55:21.141322    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:55:22.899076 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:55:22.899045    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:55:22.988921 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:55:22.988885    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:55:23.058207 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:55:23.058178    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:55:23.586349 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:55:23.586312    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:55:23.681384 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:55:23.681350    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:55:23.866130 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:55:23.866102    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:55:51.473693 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:55:51.473645    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:55:51.572501 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:55:51.572464    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:55:58.875957 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:55:58.875920    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:55:58.934081 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:55:58.934047    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:55:58.996906 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:55:58.996870    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:55:59.062371 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:55:59.062334    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:55:59.182025 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:55:59.181909    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:55:59.432902 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:55:59.432805    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:55:59.666716 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:55:59.666685    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:55:59.735454 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:55:59.735352    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:55:59.889343 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:55:59.889305    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:56:00.209922 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:00.209888    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:56:00.537360 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:00.537243    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:56:02.570294 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:02.570260    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:56:02.671116 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:02.671080    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:56:02.770947 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:02.770915    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:56:03.139975 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:03.139940    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:56:03.232502 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:03.232458    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-f82daff9-d47b5b556-wp58w"]
Jul 27 19:56:03.232820 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:03.232807    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ada3592b-a1a9-42b2-a854-6cc7d68b6135" containerName="main"
Jul 27 19:56:03.232873 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:03.232821    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="ada3592b-a1a9-42b2-a854-6cc7d68b6135" containerName="main"
Jul 27 19:56:03.232873 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:03.232835    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7d5cafe-1b6b-401b-958f-fd77224686c2" containerName="main"
Jul 27 19:56:03.232873 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:03.232841    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7d5cafe-1b6b-401b-958f-fd77224686c2" containerName="main"
Jul 27 19:56:03.232980 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:03.232904    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7d5cafe-1b6b-401b-958f-fd77224686c2" containerName="main"
Jul 27 19:56:03.232980 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:03.232917    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="ada3592b-a1a9-42b2-a854-6cc7d68b6135" containerName="main"
Jul 27 19:56:03.236103 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:03.236074    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-f82daff9-d47b5b556-wp58w"
Jul 27 19:56:03.237241 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:03.237210    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:56:03.238345 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:03.238330    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-f82daff9\""
Jul 27 19:56:03.246826 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:03.246762    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-f82daff9-d47b5b556-wp58w"]
Jul 27 19:56:03.255023 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:03.255000    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-f82daff9-789c7894b5-vdk5g"]
Jul 27 19:56:03.258511 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:03.258492    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-f82daff9-789c7894b5-vdk5g"
Jul 27 19:56:03.260675 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:03.260657    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-f82daff9-dockercfg-btnbv\""
Jul 27 19:56:03.268137 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:03.268113    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-f82daff9-789c7894b5-vdk5g"]
Jul 27 19:56:03.303813 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:03.303788    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1be6e165-5c02-43ea-ba27-c76a2099f483-plugins-config-volume\") pod \"payload-processing-e2e-migrate-f82daff9-789c7894b5-vdk5g\" (UID: \"1be6e165-5c02-43ea-ba27-c76a2099f483\") " pod="openshift-ingress/payload-processing-e2e-migrate-f82daff9-789c7894b5-vdk5g"
Jul 27 19:56:03.303961 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:03.303826    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/43db2a86-e9b8-4d1b-8677-52f4f99906cf-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-f82daff9-d47b5b556-wp58w\" (UID: \"43db2a86-e9b8-4d1b-8677-52f4f99906cf\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-f82daff9-d47b5b556-wp58w"
Jul 27 19:56:03.303961 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:03.303852    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5d4nv\" (UniqueName: \"kubernetes.io/projected/1be6e165-5c02-43ea-ba27-c76a2099f483-kube-api-access-5d4nv\") pod \"payload-processing-e2e-migrate-f82daff9-789c7894b5-vdk5g\" (UID: \"1be6e165-5c02-43ea-ba27-c76a2099f483\") " pod="openshift-ingress/payload-processing-e2e-migrate-f82daff9-789c7894b5-vdk5g"
Jul 27 19:56:03.303961 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:03.303937    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqpbm\" (UniqueName: \"kubernetes.io/projected/43db2a86-e9b8-4d1b-8677-52f4f99906cf-kube-api-access-kqpbm\") pod \"payload-pre-processing-e2e-migrate-f82daff9-d47b5b556-wp58w\" (UID: \"43db2a86-e9b8-4d1b-8677-52f4f99906cf\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-f82daff9-d47b5b556-wp58w"
Jul 27 19:56:03.404837 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:03.404757    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqpbm\" (UniqueName: \"kubernetes.io/projected/43db2a86-e9b8-4d1b-8677-52f4f99906cf-kube-api-access-kqpbm\") pod \"payload-pre-processing-e2e-migrate-f82daff9-d47b5b556-wp58w\" (UID: \"43db2a86-e9b8-4d1b-8677-52f4f99906cf\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-f82daff9-d47b5b556-wp58w"
Jul 27 19:56:03.404975 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:03.404850    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1be6e165-5c02-43ea-ba27-c76a2099f483-plugins-config-volume\") pod \"payload-processing-e2e-migrate-f82daff9-789c7894b5-vdk5g\" (UID: \"1be6e165-5c02-43ea-ba27-c76a2099f483\") " pod="openshift-ingress/payload-processing-e2e-migrate-f82daff9-789c7894b5-vdk5g"
Jul 27 19:56:03.404975 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:03.404869    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/43db2a86-e9b8-4d1b-8677-52f4f99906cf-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-f82daff9-d47b5b556-wp58w\" (UID: \"43db2a86-e9b8-4d1b-8677-52f4f99906cf\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-f82daff9-d47b5b556-wp58w"
Jul 27 19:56:03.404975 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:03.404884    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5d4nv\" (UniqueName: \"kubernetes.io/projected/1be6e165-5c02-43ea-ba27-c76a2099f483-kube-api-access-5d4nv\") pod \"payload-processing-e2e-migrate-f82daff9-789c7894b5-vdk5g\" (UID: \"1be6e165-5c02-43ea-ba27-c76a2099f483\") " pod="openshift-ingress/payload-processing-e2e-migrate-f82daff9-789c7894b5-vdk5g"
Jul 27 19:56:03.405552 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:03.405534    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/43db2a86-e9b8-4d1b-8677-52f4f99906cf-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-f82daff9-d47b5b556-wp58w\" (UID: \"43db2a86-e9b8-4d1b-8677-52f4f99906cf\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-f82daff9-d47b5b556-wp58w"
Jul 27 19:56:03.405618 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:03.405591    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1be6e165-5c02-43ea-ba27-c76a2099f483-plugins-config-volume\") pod \"payload-processing-e2e-migrate-f82daff9-789c7894b5-vdk5g\" (UID: \"1be6e165-5c02-43ea-ba27-c76a2099f483\") " pod="openshift-ingress/payload-processing-e2e-migrate-f82daff9-789c7894b5-vdk5g"
Jul 27 19:56:03.413507 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:03.413486    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqpbm\" (UniqueName: \"kubernetes.io/projected/43db2a86-e9b8-4d1b-8677-52f4f99906cf-kube-api-access-kqpbm\") pod \"payload-pre-processing-e2e-migrate-f82daff9-d47b5b556-wp58w\" (UID: \"43db2a86-e9b8-4d1b-8677-52f4f99906cf\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-f82daff9-d47b5b556-wp58w"
Jul 27 19:56:03.413607 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:03.413589    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5d4nv\" (UniqueName: \"kubernetes.io/projected/1be6e165-5c02-43ea-ba27-c76a2099f483-kube-api-access-5d4nv\") pod \"payload-processing-e2e-migrate-f82daff9-789c7894b5-vdk5g\" (UID: \"1be6e165-5c02-43ea-ba27-c76a2099f483\") " pod="openshift-ingress/payload-processing-e2e-migrate-f82daff9-789c7894b5-vdk5g"
Jul 27 19:56:03.556816 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:03.556778    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-f82daff9-d47b5b556-wp58w"
Jul 27 19:56:03.571035 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:03.571012    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-f82daff9-789c7894b5-vdk5g"
Jul 27 19:56:03.707382 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:56:03.707352    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43db2a86_e9b8_4d1b_8677_52f4f99906cf.slice/crio-884a1d3ed259b2c583dca38f3ab19a50510d5edd88200dde39928bc6007bbbba WatchSource:0}: Error finding container 884a1d3ed259b2c583dca38f3ab19a50510d5edd88200dde39928bc6007bbbba: Status 404 returned error can't find the container with id 884a1d3ed259b2c583dca38f3ab19a50510d5edd88200dde39928bc6007bbbba
Jul 27 19:56:03.707898 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:03.707869    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-f82daff9-d47b5b556-wp58w"]
Jul 27 19:56:03.713900 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:03.713879    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 19:56:03.735425 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:03.735382    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-f82daff9-789c7894b5-vdk5g"]
Jul 27 19:56:03.737515 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:56:03.737483    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1be6e165_5c02_43ea_ba27_c76a2099f483.slice/crio-62f04579c1f76a42cb2f2add26cac40e50547cd144e4c0d77d535f530893bb3a WatchSource:0}: Error finding container 62f04579c1f76a42cb2f2add26cac40e50547cd144e4c0d77d535f530893bb3a: Status 404 returned error can't find the container with id 62f04579c1f76a42cb2f2add26cac40e50547cd144e4c0d77d535f530893bb3a
Jul 27 19:56:04.627438 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:04.619813    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-f82daff9-d47b5b556-wp58w" event={"ID":"43db2a86-e9b8-4d1b-8677-52f4f99906cf","Type":"ContainerStarted","Data":"884a1d3ed259b2c583dca38f3ab19a50510d5edd88200dde39928bc6007bbbba"}
Jul 27 19:56:04.627438 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:04.626377    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-f82daff9-789c7894b5-vdk5g" event={"ID":"1be6e165-5c02-43ea-ba27-c76a2099f483","Type":"ContainerStarted","Data":"62f04579c1f76a42cb2f2add26cac40e50547cd144e4c0d77d535f530893bb3a"}
Jul 27 19:56:06.640652 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:06.640614    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-f82daff9-d47b5b556-wp58w" event={"ID":"43db2a86-e9b8-4d1b-8677-52f4f99906cf","Type":"ContainerStarted","Data":"a0f78ff2a5303152152426bcbf8334138dc899b369b3ca50cc00e9cd88212aa1"}
Jul 27 19:56:06.641139 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:06.640682    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-f82daff9-d47b5b556-wp58w"
Jul 27 19:56:06.642379 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:06.642353    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-f82daff9-789c7894b5-vdk5g" event={"ID":"1be6e165-5c02-43ea-ba27-c76a2099f483","Type":"ContainerStarted","Data":"f0572b9772f2e975e8a5f9061d16fc0d08fa46322c31efd5dd32959904476f14"}
Jul 27 19:56:06.642520 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:06.642425    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-f82daff9-789c7894b5-vdk5g"
Jul 27 19:56:06.658023 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:06.657968    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-f82daff9-d47b5b556-wp58w" podStartSLOduration=1.784142213 podStartE2EDuration="3.657954146s" totalImagesPullingTime="1.873811933s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:56:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:56:06.656060539 +0000 UTC m=+1779.528252517" watchObservedRunningTime="2026-07-27 19:56:06.657954146 +0000 UTC m=+1779.530146119"
Jul 27 19:56:06.671746 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:06.671687    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-f82daff9-789c7894b5-vdk5g" podStartSLOduration=1.82946027 podStartE2EDuration="3.671669355s" totalImagesPullingTime="1.842209085s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:56:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:56:06.670326015 +0000 UTC m=+1779.542517989" watchObservedRunningTime="2026-07-27 19:56:06.671669355 +0000 UTC m=+1779.543861329"
Jul 27 19:56:17.650405 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:17.650316    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-f82daff9-d47b5b556-wp58w"
Jul 27 19:56:17.650882 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:17.650596    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-f82daff9-789c7894b5-vdk5g"
Jul 27 19:56:27.689319 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:27.689288    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5dfc2_103bab23-bca3-472a-974b-7731f83c5bfb/ovn-acl-logging/0.log"
Jul 27 19:56:27.694742 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:27.694718    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5dfc2_103bab23-bca3-472a-974b-7731f83c5bfb/ovn-acl-logging/0.log"
Jul 27 19:56:37.921527 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:37.921468    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:56:37.944377 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:37.944340    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-f82daff9-789c7894b5-vdk5g"]
Jul 27 19:56:37.944653 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:37.944625    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-f82daff9-789c7894b5-vdk5g" podUID="1be6e165-5c02-43ea-ba27-c76a2099f483" containerName="payload-processing" containerID="cri-o://f0572b9772f2e975e8a5f9061d16fc0d08fa46322c31efd5dd32959904476f14" gracePeriod=30
Jul 27 19:56:37.950253 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:37.950152    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-f82daff9-d47b5b556-wp58w"]
Jul 27 19:56:37.950477 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:37.950391    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-f82daff9-d47b5b556-wp58w" podUID="43db2a86-e9b8-4d1b-8677-52f4f99906cf" containerName="payload-pre-processing" containerID="cri-o://a0f78ff2a5303152152426bcbf8334138dc899b369b3ca50cc00e9cd88212aa1" gracePeriod=30
Jul 27 19:56:38.025209 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:38.025164    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:56:38.096898 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:38.096841    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:56:38.210437 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:38.210339    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:56:38.249112 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:38.249085    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-f82daff9-789c7894b5-vdk5g"
Jul 27 19:56:38.251884 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:38.251860    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-f82daff9-d47b5b556-wp58w"
Jul 27 19:56:38.429890 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:38.429851    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/43db2a86-e9b8-4d1b-8677-52f4f99906cf-plugins-config-volume\") pod \"43db2a86-e9b8-4d1b-8677-52f4f99906cf\" (UID: \"43db2a86-e9b8-4d1b-8677-52f4f99906cf\") "
Jul 27 19:56:38.430080 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:38.429908    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5d4nv\" (UniqueName: \"kubernetes.io/projected/1be6e165-5c02-43ea-ba27-c76a2099f483-kube-api-access-5d4nv\") pod \"1be6e165-5c02-43ea-ba27-c76a2099f483\" (UID: \"1be6e165-5c02-43ea-ba27-c76a2099f483\") "
Jul 27 19:56:38.430080 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:38.430004    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1be6e165-5c02-43ea-ba27-c76a2099f483-plugins-config-volume\") pod \"1be6e165-5c02-43ea-ba27-c76a2099f483\" (UID: \"1be6e165-5c02-43ea-ba27-c76a2099f483\") "
Jul 27 19:56:38.430080 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:38.430042    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kqpbm\" (UniqueName: \"kubernetes.io/projected/43db2a86-e9b8-4d1b-8677-52f4f99906cf-kube-api-access-kqpbm\") pod \"43db2a86-e9b8-4d1b-8677-52f4f99906cf\" (UID: \"43db2a86-e9b8-4d1b-8677-52f4f99906cf\") "
Jul 27 19:56:38.430395 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:38.430359    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43db2a86-e9b8-4d1b-8677-52f4f99906cf-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "43db2a86-e9b8-4d1b-8677-52f4f99906cf" (UID: "43db2a86-e9b8-4d1b-8677-52f4f99906cf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:56:38.430532 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:38.430434    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1be6e165-5c02-43ea-ba27-c76a2099f483-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1be6e165-5c02-43ea-ba27-c76a2099f483" (UID: "1be6e165-5c02-43ea-ba27-c76a2099f483"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:56:38.432449 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:38.432405    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1be6e165-5c02-43ea-ba27-c76a2099f483-kube-api-access-5d4nv" (OuterVolumeSpecName: "kube-api-access-5d4nv") pod "1be6e165-5c02-43ea-ba27-c76a2099f483" (UID: "1be6e165-5c02-43ea-ba27-c76a2099f483"). InnerVolumeSpecName "kube-api-access-5d4nv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:56:38.432545 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:38.432452    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43db2a86-e9b8-4d1b-8677-52f4f99906cf-kube-api-access-kqpbm" (OuterVolumeSpecName: "kube-api-access-kqpbm") pod "43db2a86-e9b8-4d1b-8677-52f4f99906cf" (UID: "43db2a86-e9b8-4d1b-8677-52f4f99906cf"). InnerVolumeSpecName "kube-api-access-kqpbm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:56:38.469951 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:38.469848    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:56:38.530961 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:38.530917    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1be6e165-5c02-43ea-ba27-c76a2099f483-plugins-config-volume\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:56:38.530961 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:38.530957    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kqpbm\" (UniqueName: \"kubernetes.io/projected/43db2a86-e9b8-4d1b-8677-52f4f99906cf-kube-api-access-kqpbm\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:56:38.530961 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:38.530966    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/43db2a86-e9b8-4d1b-8677-52f4f99906cf-plugins-config-volume\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:56:38.531138 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:38.530976    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5d4nv\" (UniqueName: \"kubernetes.io/projected/1be6e165-5c02-43ea-ba27-c76a2099f483-kube-api-access-5d4nv\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:56:38.654043 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:38.654008    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:56:38.817877 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:38.817792    2558 generic.go:356] "Generic (PLEG): container finished" podID="1be6e165-5c02-43ea-ba27-c76a2099f483" containerID="f0572b9772f2e975e8a5f9061d16fc0d08fa46322c31efd5dd32959904476f14" exitCode=0
Jul 27 19:56:38.817877 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:38.817858    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-f82daff9-789c7894b5-vdk5g"
Jul 27 19:56:38.818105 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:38.817874    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-f82daff9-789c7894b5-vdk5g" event={"ID":"1be6e165-5c02-43ea-ba27-c76a2099f483","Type":"ContainerDied","Data":"f0572b9772f2e975e8a5f9061d16fc0d08fa46322c31efd5dd32959904476f14"}
Jul 27 19:56:38.818105 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:38.817935    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-f82daff9-789c7894b5-vdk5g" event={"ID":"1be6e165-5c02-43ea-ba27-c76a2099f483","Type":"ContainerDied","Data":"62f04579c1f76a42cb2f2add26cac40e50547cd144e4c0d77d535f530893bb3a"}
Jul 27 19:56:38.818105 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:38.817965    2558 scope.go:117] "RemoveContainer" containerID="f0572b9772f2e975e8a5f9061d16fc0d08fa46322c31efd5dd32959904476f14"
Jul 27 19:56:38.819730 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:38.819523    2558 generic.go:356] "Generic (PLEG): container finished" podID="43db2a86-e9b8-4d1b-8677-52f4f99906cf" containerID="a0f78ff2a5303152152426bcbf8334138dc899b369b3ca50cc00e9cd88212aa1" exitCode=0
Jul 27 19:56:38.819730 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:38.819580    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-f82daff9-d47b5b556-wp58w" event={"ID":"43db2a86-e9b8-4d1b-8677-52f4f99906cf","Type":"ContainerDied","Data":"a0f78ff2a5303152152426bcbf8334138dc899b369b3ca50cc00e9cd88212aa1"}
Jul 27 19:56:38.819730 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:38.819630    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-f82daff9-d47b5b556-wp58w" event={"ID":"43db2a86-e9b8-4d1b-8677-52f4f99906cf","Type":"ContainerDied","Data":"884a1d3ed259b2c583dca38f3ab19a50510d5edd88200dde39928bc6007bbbba"}
Jul 27 19:56:38.819935 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:38.819744    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-f82daff9-d47b5b556-wp58w"
Jul 27 19:56:38.829819 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:38.829778    2558 scope.go:117] "RemoveContainer" containerID="f0572b9772f2e975e8a5f9061d16fc0d08fa46322c31efd5dd32959904476f14"
Jul 27 19:56:38.830169 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:56:38.830142    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0572b9772f2e975e8a5f9061d16fc0d08fa46322c31efd5dd32959904476f14\": container with ID starting with f0572b9772f2e975e8a5f9061d16fc0d08fa46322c31efd5dd32959904476f14 not found: ID does not exist" containerID="f0572b9772f2e975e8a5f9061d16fc0d08fa46322c31efd5dd32959904476f14"
Jul 27 19:56:38.830258 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:38.830176    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0572b9772f2e975e8a5f9061d16fc0d08fa46322c31efd5dd32959904476f14"} err="failed to get container status \"f0572b9772f2e975e8a5f9061d16fc0d08fa46322c31efd5dd32959904476f14\": rpc error: code = NotFound desc = could not find container \"f0572b9772f2e975e8a5f9061d16fc0d08fa46322c31efd5dd32959904476f14\": container with ID starting with f0572b9772f2e975e8a5f9061d16fc0d08fa46322c31efd5dd32959904476f14 not found: ID does not exist"
Jul 27 19:56:38.830258 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:38.830197    2558 scope.go:117] "RemoveContainer" containerID="a0f78ff2a5303152152426bcbf8334138dc899b369b3ca50cc00e9cd88212aa1"
Jul 27 19:56:38.840087 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:38.840062    2558 scope.go:117] "RemoveContainer" containerID="a0f78ff2a5303152152426bcbf8334138dc899b369b3ca50cc00e9cd88212aa1"
Jul 27 19:56:38.840433 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:56:38.840397    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0f78ff2a5303152152426bcbf8334138dc899b369b3ca50cc00e9cd88212aa1\": container with ID starting with a0f78ff2a5303152152426bcbf8334138dc899b369b3ca50cc00e9cd88212aa1 not found: ID does not exist" containerID="a0f78ff2a5303152152426bcbf8334138dc899b369b3ca50cc00e9cd88212aa1"
Jul 27 19:56:38.840499 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:38.840446    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0f78ff2a5303152152426bcbf8334138dc899b369b3ca50cc00e9cd88212aa1"} err="failed to get container status \"a0f78ff2a5303152152426bcbf8334138dc899b369b3ca50cc00e9cd88212aa1\": rpc error: code = NotFound desc = could not find container \"a0f78ff2a5303152152426bcbf8334138dc899b369b3ca50cc00e9cd88212aa1\": container with ID starting with a0f78ff2a5303152152426bcbf8334138dc899b369b3ca50cc00e9cd88212aa1 not found: ID does not exist"
Jul 27 19:56:38.844384 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:38.844325    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-f82daff9-789c7894b5-vdk5g"]
Jul 27 19:56:38.849018 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:38.848990    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-f82daff9-789c7894b5-vdk5g"]
Jul 27 19:56:38.864104 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:38.864071    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-f82daff9-d47b5b556-wp58w"]
Jul 27 19:56:38.870636 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:38.870603    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-f82daff9-d47b5b556-wp58w"]
Jul 27 19:56:38.964406 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:38.964359    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:56:39.375791 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:39.375756    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:56:39.675311 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:39.675227    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1be6e165-5c02-43ea-ba27-c76a2099f483" path="/var/lib/kubelet/pods/1be6e165-5c02-43ea-ba27-c76a2099f483/volumes"
Jul 27 19:56:39.675638 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:39.675624    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43db2a86-e9b8-4d1b-8677-52f4f99906cf" path="/var/lib/kubelet/pods/43db2a86-e9b8-4d1b-8677-52f4f99906cf/volumes"
Jul 27 19:56:41.315308 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:41.315274    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:56:41.416661 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:41.416604    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:56:41.516384 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:41.516342    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:56:41.932551 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:41.932518    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:56:42.030197 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:42.030151    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-6e159107-5867c45f5b-pdf5r"]
Jul 27 19:56:42.030687 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:42.030665    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43db2a86-e9b8-4d1b-8677-52f4f99906cf" containerName="payload-pre-processing"
Jul 27 19:56:42.030758 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:42.030692    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="43db2a86-e9b8-4d1b-8677-52f4f99906cf" containerName="payload-pre-processing"
Jul 27 19:56:42.030758 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:42.030703    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1be6e165-5c02-43ea-ba27-c76a2099f483" containerName="payload-processing"
Jul 27 19:56:42.030758 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:42.030713    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="1be6e165-5c02-43ea-ba27-c76a2099f483" containerName="payload-processing"
Jul 27 19:56:42.030852 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:42.030805    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="43db2a86-e9b8-4d1b-8677-52f4f99906cf" containerName="payload-pre-processing"
Jul 27 19:56:42.030852 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:42.030814    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="1be6e165-5c02-43ea-ba27-c76a2099f483" containerName="payload-processing"
Jul 27 19:56:42.031994 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:42.031956    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:56:42.036958 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:42.036940    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-6e159107-5867c45f5b-pdf5r"
Jul 27 19:56:42.039446 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:42.039403    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-6e159107\""
Jul 27 19:56:42.039557 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:42.039433    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-6e159107-dockercfg-j8j54\""
Jul 27 19:56:42.046352 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:42.046323    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-6e159107-5867c45f5b-pdf5r"]
Jul 27 19:56:42.060118 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:42.060086    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndg7c\" (UniqueName: \"kubernetes.io/projected/50f626c1-0dd7-49b6-84a0-b91b65e40b4a-kube-api-access-ndg7c\") pod \"payload-pre-processing-e2e-ait-6e159107-5867c45f5b-pdf5r\" (UID: \"50f626c1-0dd7-49b6-84a0-b91b65e40b4a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-6e159107-5867c45f5b-pdf5r"
Jul 27 19:56:42.060363 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:42.060187    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/50f626c1-0dd7-49b6-84a0-b91b65e40b4a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-6e159107-5867c45f5b-pdf5r\" (UID: \"50f626c1-0dd7-49b6-84a0-b91b65e40b4a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-6e159107-5867c45f5b-pdf5r"
Jul 27 19:56:42.064454 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:42.064387    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-6e159107-6dccb7cc84-gzdpf"]
Jul 27 19:56:42.069454 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:42.069433    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-6e159107-6dccb7cc84-gzdpf"
Jul 27 19:56:42.078043 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:42.078020    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-6e159107-6dccb7cc84-gzdpf"]
Jul 27 19:56:42.161243 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:42.161204    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndg7c\" (UniqueName: \"kubernetes.io/projected/50f626c1-0dd7-49b6-84a0-b91b65e40b4a-kube-api-access-ndg7c\") pod \"payload-pre-processing-e2e-ait-6e159107-5867c45f5b-pdf5r\" (UID: \"50f626c1-0dd7-49b6-84a0-b91b65e40b4a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-6e159107-5867c45f5b-pdf5r"
Jul 27 19:56:42.161486 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:42.161286    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/47799aa2-485f-47d4-9a94-16c1e5fe7c81-plugins-config-volume\") pod \"payload-processing-e2e-ait-6e159107-6dccb7cc84-gzdpf\" (UID: \"47799aa2-485f-47d4-9a94-16c1e5fe7c81\") " pod="openshift-ingress/payload-processing-e2e-ait-6e159107-6dccb7cc84-gzdpf"
Jul 27 19:56:42.161486 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:42.161315    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/50f626c1-0dd7-49b6-84a0-b91b65e40b4a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-6e159107-5867c45f5b-pdf5r\" (UID: \"50f626c1-0dd7-49b6-84a0-b91b65e40b4a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-6e159107-5867c45f5b-pdf5r"
Jul 27 19:56:42.161486 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:42.161349    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjmfn\" (UniqueName: \"kubernetes.io/projected/47799aa2-485f-47d4-9a94-16c1e5fe7c81-kube-api-access-vjmfn\") pod \"payload-processing-e2e-ait-6e159107-6dccb7cc84-gzdpf\" (UID: \"47799aa2-485f-47d4-9a94-16c1e5fe7c81\") " pod="openshift-ingress/payload-processing-e2e-ait-6e159107-6dccb7cc84-gzdpf"
Jul 27 19:56:42.162028 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:42.162008    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/50f626c1-0dd7-49b6-84a0-b91b65e40b4a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-6e159107-5867c45f5b-pdf5r\" (UID: \"50f626c1-0dd7-49b6-84a0-b91b65e40b4a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-6e159107-5867c45f5b-pdf5r"
Jul 27 19:56:42.169916 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:42.169880    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndg7c\" (UniqueName: \"kubernetes.io/projected/50f626c1-0dd7-49b6-84a0-b91b65e40b4a-kube-api-access-ndg7c\") pod \"payload-pre-processing-e2e-ait-6e159107-5867c45f5b-pdf5r\" (UID: \"50f626c1-0dd7-49b6-84a0-b91b65e40b4a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-6e159107-5867c45f5b-pdf5r"
Jul 27 19:56:42.261915 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:42.261819    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/47799aa2-485f-47d4-9a94-16c1e5fe7c81-plugins-config-volume\") pod \"payload-processing-e2e-ait-6e159107-6dccb7cc84-gzdpf\" (UID: \"47799aa2-485f-47d4-9a94-16c1e5fe7c81\") " pod="openshift-ingress/payload-processing-e2e-ait-6e159107-6dccb7cc84-gzdpf"
Jul 27 19:56:42.261915 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:42.261875    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjmfn\" (UniqueName: \"kubernetes.io/projected/47799aa2-485f-47d4-9a94-16c1e5fe7c81-kube-api-access-vjmfn\") pod \"payload-processing-e2e-ait-6e159107-6dccb7cc84-gzdpf\" (UID: \"47799aa2-485f-47d4-9a94-16c1e5fe7c81\") " pod="openshift-ingress/payload-processing-e2e-ait-6e159107-6dccb7cc84-gzdpf"
Jul 27 19:56:42.262699 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:42.262664    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/47799aa2-485f-47d4-9a94-16c1e5fe7c81-plugins-config-volume\") pod \"payload-processing-e2e-ait-6e159107-6dccb7cc84-gzdpf\" (UID: \"47799aa2-485f-47d4-9a94-16c1e5fe7c81\") " pod="openshift-ingress/payload-processing-e2e-ait-6e159107-6dccb7cc84-gzdpf"
Jul 27 19:56:42.270107 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:42.270077    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjmfn\" (UniqueName: \"kubernetes.io/projected/47799aa2-485f-47d4-9a94-16c1e5fe7c81-kube-api-access-vjmfn\") pod \"payload-processing-e2e-ait-6e159107-6dccb7cc84-gzdpf\" (UID: \"47799aa2-485f-47d4-9a94-16c1e5fe7c81\") " pod="openshift-ingress/payload-processing-e2e-ait-6e159107-6dccb7cc84-gzdpf"
Jul 27 19:56:42.348754 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:42.348722    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-6e159107-5867c45f5b-pdf5r"
Jul 27 19:56:42.380259 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:42.380221    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-6e159107-6dccb7cc84-gzdpf"
Jul 27 19:56:42.513399 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:42.513369    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-6e159107-5867c45f5b-pdf5r"]
Jul 27 19:56:42.516497 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:56:42.516253    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50f626c1_0dd7_49b6_84a0_b91b65e40b4a.slice/crio-57052abba060e3c003b4ff4a83a9d186a3deab900d5b261bfe7990b095e012fb WatchSource:0}: Error finding container 57052abba060e3c003b4ff4a83a9d186a3deab900d5b261bfe7990b095e012fb: Status 404 returned error can't find the container with id 57052abba060e3c003b4ff4a83a9d186a3deab900d5b261bfe7990b095e012fb
Jul 27 19:56:42.549155 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:42.549125    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-6e159107-6dccb7cc84-gzdpf"]
Jul 27 19:56:42.553771 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:56:42.553728    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47799aa2_485f_47d4_9a94_16c1e5fe7c81.slice/crio-5b52911fcc8746f594e348d4978c52b2ddf2edb16128d0d74f83d0d01e679c8e WatchSource:0}: Error finding container 5b52911fcc8746f594e348d4978c52b2ddf2edb16128d0d74f83d0d01e679c8e: Status 404 returned error can't find the container with id 5b52911fcc8746f594e348d4978c52b2ddf2edb16128d0d74f83d0d01e679c8e
Jul 27 19:56:42.848589 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:42.848490    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-6e159107-6dccb7cc84-gzdpf" event={"ID":"47799aa2-485f-47d4-9a94-16c1e5fe7c81","Type":"ContainerStarted","Data":"358c13265aea98c698fc1083369117cfd27561a5e51d806a3646ceea8cabaf68"}
Jul 27 19:56:42.848772 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:42.848595    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-6e159107-6dccb7cc84-gzdpf" event={"ID":"47799aa2-485f-47d4-9a94-16c1e5fe7c81","Type":"ContainerStarted","Data":"5b52911fcc8746f594e348d4978c52b2ddf2edb16128d0d74f83d0d01e679c8e"}
Jul 27 19:56:42.848772 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:42.848746    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-6e159107-6dccb7cc84-gzdpf"
Jul 27 19:56:42.850096 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:42.850072    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-6e159107-5867c45f5b-pdf5r" event={"ID":"50f626c1-0dd7-49b6-84a0-b91b65e40b4a","Type":"ContainerStarted","Data":"af1d9b624831874641f80db30e97f721dfdec37c5159c0863b21add19a02bddc"}
Jul 27 19:56:42.850206 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:42.850117    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-6e159107-5867c45f5b-pdf5r" event={"ID":"50f626c1-0dd7-49b6-84a0-b91b65e40b4a","Type":"ContainerStarted","Data":"57052abba060e3c003b4ff4a83a9d186a3deab900d5b261bfe7990b095e012fb"}
Jul 27 19:56:42.850206 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:42.850163    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-6e159107-5867c45f5b-pdf5r"
Jul 27 19:56:42.881873 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:42.881816    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-6e159107-6dccb7cc84-gzdpf" podStartSLOduration=0.881786878 podStartE2EDuration="881.786878ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:56:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:56:42.878590915 +0000 UTC m=+1815.750782889" watchObservedRunningTime="2026-07-27 19:56:42.881786878 +0000 UTC m=+1815.753978843"
Jul 27 19:56:42.898224 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:42.898160    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-6e159107-5867c45f5b-pdf5r" podStartSLOduration=0.898144823 podStartE2EDuration="898.144823ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:56:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:56:42.896658086 +0000 UTC m=+1815.768850067" watchObservedRunningTime="2026-07-27 19:56:42.898144823 +0000 UTC m=+1815.770336796"
Jul 27 19:56:53.857710 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:53.857678    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-6e159107-6dccb7cc84-gzdpf"
Jul 27 19:56:53.858151 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:56:53.858127    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-6e159107-5867c45f5b-pdf5r"
Jul 27 19:57:17.483908 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:17.483872    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-6e159107-6dccb7cc84-gzdpf"]
Jul 27 19:57:17.486788 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:17.484141    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-6e159107-6dccb7cc84-gzdpf" podUID="47799aa2-485f-47d4-9a94-16c1e5fe7c81" containerName="payload-processing" containerID="cri-o://358c13265aea98c698fc1083369117cfd27561a5e51d806a3646ceea8cabaf68" gracePeriod=30
Jul 27 19:57:17.491178 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:17.491144    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-6e159107-5867c45f5b-pdf5r"]
Jul 27 19:57:17.491709 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:17.491356    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-6e159107-5867c45f5b-pdf5r" podUID="50f626c1-0dd7-49b6-84a0-b91b65e40b4a" containerName="payload-pre-processing" containerID="cri-o://af1d9b624831874641f80db30e97f721dfdec37c5159c0863b21add19a02bddc" gracePeriod=30
Jul 27 19:57:17.504649 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:17.504394    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:57:17.624384 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:17.624339    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:57:17.759459 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:17.759392    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:57:17.768666 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:17.768639    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-6e159107-6dccb7cc84-gzdpf"
Jul 27 19:57:17.771592 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:17.771575    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-6e159107-5867c45f5b-pdf5r"
Jul 27 19:57:17.889377 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:17.889348    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vjmfn\" (UniqueName: \"kubernetes.io/projected/47799aa2-485f-47d4-9a94-16c1e5fe7c81-kube-api-access-vjmfn\") pod \"47799aa2-485f-47d4-9a94-16c1e5fe7c81\" (UID: \"47799aa2-485f-47d4-9a94-16c1e5fe7c81\") "
Jul 27 19:57:17.889377 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:17.889382    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ndg7c\" (UniqueName: \"kubernetes.io/projected/50f626c1-0dd7-49b6-84a0-b91b65e40b4a-kube-api-access-ndg7c\") pod \"50f626c1-0dd7-49b6-84a0-b91b65e40b4a\" (UID: \"50f626c1-0dd7-49b6-84a0-b91b65e40b4a\") "
Jul 27 19:57:17.889600 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:17.889469    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/50f626c1-0dd7-49b6-84a0-b91b65e40b4a-plugins-config-volume\") pod \"50f626c1-0dd7-49b6-84a0-b91b65e40b4a\" (UID: \"50f626c1-0dd7-49b6-84a0-b91b65e40b4a\") "
Jul 27 19:57:17.889600 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:17.889563    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/47799aa2-485f-47d4-9a94-16c1e5fe7c81-plugins-config-volume\") pod \"47799aa2-485f-47d4-9a94-16c1e5fe7c81\" (UID: \"47799aa2-485f-47d4-9a94-16c1e5fe7c81\") "
Jul 27 19:57:17.889886 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:17.889864    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50f626c1-0dd7-49b6-84a0-b91b65e40b4a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "50f626c1-0dd7-49b6-84a0-b91b65e40b4a" (UID: "50f626c1-0dd7-49b6-84a0-b91b65e40b4a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:57:17.889971 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:17.889912    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/47799aa2-485f-47d4-9a94-16c1e5fe7c81-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "47799aa2-485f-47d4-9a94-16c1e5fe7c81" (UID: "47799aa2-485f-47d4-9a94-16c1e5fe7c81"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:57:17.891582 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:17.891564    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50f626c1-0dd7-49b6-84a0-b91b65e40b4a-kube-api-access-ndg7c" (OuterVolumeSpecName: "kube-api-access-ndg7c") pod "50f626c1-0dd7-49b6-84a0-b91b65e40b4a" (UID: "50f626c1-0dd7-49b6-84a0-b91b65e40b4a"). InnerVolumeSpecName "kube-api-access-ndg7c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:57:17.891672 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:17.891658    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47799aa2-485f-47d4-9a94-16c1e5fe7c81-kube-api-access-vjmfn" (OuterVolumeSpecName: "kube-api-access-vjmfn") pod "47799aa2-485f-47d4-9a94-16c1e5fe7c81" (UID: "47799aa2-485f-47d4-9a94-16c1e5fe7c81"). InnerVolumeSpecName "kube-api-access-vjmfn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:57:17.990674 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:17.990636    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/50f626c1-0dd7-49b6-84a0-b91b65e40b4a-plugins-config-volume\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:57:17.990674 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:17.990665    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/47799aa2-485f-47d4-9a94-16c1e5fe7c81-plugins-config-volume\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:57:17.990674 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:17.990674    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vjmfn\" (UniqueName: \"kubernetes.io/projected/47799aa2-485f-47d4-9a94-16c1e5fe7c81-kube-api-access-vjmfn\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:57:17.990674 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:17.990683    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ndg7c\" (UniqueName: \"kubernetes.io/projected/50f626c1-0dd7-49b6-84a0-b91b65e40b4a-kube-api-access-ndg7c\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:57:18.042390 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:18.042305    2558 generic.go:356] "Generic (PLEG): container finished" podID="50f626c1-0dd7-49b6-84a0-b91b65e40b4a" containerID="af1d9b624831874641f80db30e97f721dfdec37c5159c0863b21add19a02bddc" exitCode=0
Jul 27 19:57:18.042390 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:18.042354    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-6e159107-5867c45f5b-pdf5r"
Jul 27 19:57:18.042597 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:18.042397    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-6e159107-5867c45f5b-pdf5r" event={"ID":"50f626c1-0dd7-49b6-84a0-b91b65e40b4a","Type":"ContainerDied","Data":"af1d9b624831874641f80db30e97f721dfdec37c5159c0863b21add19a02bddc"}
Jul 27 19:57:18.042597 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:18.042465    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-6e159107-5867c45f5b-pdf5r" event={"ID":"50f626c1-0dd7-49b6-84a0-b91b65e40b4a","Type":"ContainerDied","Data":"57052abba060e3c003b4ff4a83a9d186a3deab900d5b261bfe7990b095e012fb"}
Jul 27 19:57:18.042597 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:18.042496    2558 scope.go:117] "RemoveContainer" containerID="af1d9b624831874641f80db30e97f721dfdec37c5159c0863b21add19a02bddc"
Jul 27 19:57:18.043969 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:18.043918    2558 generic.go:356] "Generic (PLEG): container finished" podID="47799aa2-485f-47d4-9a94-16c1e5fe7c81" containerID="358c13265aea98c698fc1083369117cfd27561a5e51d806a3646ceea8cabaf68" exitCode=0
Jul 27 19:57:18.044064 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:18.043972    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-6e159107-6dccb7cc84-gzdpf" event={"ID":"47799aa2-485f-47d4-9a94-16c1e5fe7c81","Type":"ContainerDied","Data":"358c13265aea98c698fc1083369117cfd27561a5e51d806a3646ceea8cabaf68"}
Jul 27 19:57:18.044064 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:18.044023    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-6e159107-6dccb7cc84-gzdpf" event={"ID":"47799aa2-485f-47d4-9a94-16c1e5fe7c81","Type":"ContainerDied","Data":"5b52911fcc8746f594e348d4978c52b2ddf2edb16128d0d74f83d0d01e679c8e"}
Jul 27 19:57:18.044064 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:18.043978    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-6e159107-6dccb7cc84-gzdpf"
Jul 27 19:57:18.052138 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:18.052122    2558 scope.go:117] "RemoveContainer" containerID="af1d9b624831874641f80db30e97f721dfdec37c5159c0863b21add19a02bddc"
Jul 27 19:57:18.052381 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:57:18.052358    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af1d9b624831874641f80db30e97f721dfdec37c5159c0863b21add19a02bddc\": container with ID starting with af1d9b624831874641f80db30e97f721dfdec37c5159c0863b21add19a02bddc not found: ID does not exist" containerID="af1d9b624831874641f80db30e97f721dfdec37c5159c0863b21add19a02bddc"
Jul 27 19:57:18.052510 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:18.052385    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af1d9b624831874641f80db30e97f721dfdec37c5159c0863b21add19a02bddc"} err="failed to get container status \"af1d9b624831874641f80db30e97f721dfdec37c5159c0863b21add19a02bddc\": rpc error: code = NotFound desc = could not find container \"af1d9b624831874641f80db30e97f721dfdec37c5159c0863b21add19a02bddc\": container with ID starting with af1d9b624831874641f80db30e97f721dfdec37c5159c0863b21add19a02bddc not found: ID does not exist"
Jul 27 19:57:18.052510 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:18.052402    2558 scope.go:117] "RemoveContainer" containerID="358c13265aea98c698fc1083369117cfd27561a5e51d806a3646ceea8cabaf68"
Jul 27 19:57:18.060591 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:18.060573    2558 scope.go:117] "RemoveContainer" containerID="358c13265aea98c698fc1083369117cfd27561a5e51d806a3646ceea8cabaf68"
Jul 27 19:57:18.060845 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:57:18.060827    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"358c13265aea98c698fc1083369117cfd27561a5e51d806a3646ceea8cabaf68\": container with ID starting with 358c13265aea98c698fc1083369117cfd27561a5e51d806a3646ceea8cabaf68 not found: ID does not exist" containerID="358c13265aea98c698fc1083369117cfd27561a5e51d806a3646ceea8cabaf68"
Jul 27 19:57:18.060899 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:18.060864    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"358c13265aea98c698fc1083369117cfd27561a5e51d806a3646ceea8cabaf68"} err="failed to get container status \"358c13265aea98c698fc1083369117cfd27561a5e51d806a3646ceea8cabaf68\": rpc error: code = NotFound desc = could not find container \"358c13265aea98c698fc1083369117cfd27561a5e51d806a3646ceea8cabaf68\": container with ID starting with 358c13265aea98c698fc1083369117cfd27561a5e51d806a3646ceea8cabaf68 not found: ID does not exist"
Jul 27 19:57:18.065149 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:18.065128    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-6e159107-5867c45f5b-pdf5r"]
Jul 27 19:57:18.069060 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:18.069042    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-6e159107-5867c45f5b-pdf5r"]
Jul 27 19:57:18.079334 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:18.079313    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-6e159107-6dccb7cc84-gzdpf"]
Jul 27 19:57:18.083204 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:18.083185    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-6e159107-6dccb7cc84-gzdpf"]
Jul 27 19:57:18.901385 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:18.901350    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:57:19.076430 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:19.076381    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:57:19.289774 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:19.289694    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:57:19.579720 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:19.579636    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:57:19.674845 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:19.674814    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="47799aa2-485f-47d4-9a94-16c1e5fe7c81" path="/var/lib/kubelet/pods/47799aa2-485f-47d4-9a94-16c1e5fe7c81/volumes"
Jul 27 19:57:19.675173 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:19.675160    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50f626c1-0dd7-49b6-84a0-b91b65e40b4a" path="/var/lib/kubelet/pods/50f626c1-0dd7-49b6-84a0-b91b65e40b4a/volumes"
Jul 27 19:57:19.852341 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:19.852257    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:57:21.874271 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:21.874238    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:57:21.947917 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:21.947882    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:57:22.023085 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:22.023040    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:57:22.193430 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:22.193324    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:57:22.555690 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:22.555604    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:57:22.648982 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:22.648948    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:57:22.654726 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:22.654697    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-ee5db6ec-5d9487884d-b4tb5"]
Jul 27 19:57:22.655133 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:22.655116    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47799aa2-485f-47d4-9a94-16c1e5fe7c81" containerName="payload-processing"
Jul 27 19:57:22.655239 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:22.655135    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="47799aa2-485f-47d4-9a94-16c1e5fe7c81" containerName="payload-processing"
Jul 27 19:57:22.655239 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:22.655176    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50f626c1-0dd7-49b6-84a0-b91b65e40b4a" containerName="payload-pre-processing"
Jul 27 19:57:22.655239 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:22.655184    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="50f626c1-0dd7-49b6-84a0-b91b65e40b4a" containerName="payload-pre-processing"
Jul 27 19:57:22.655459 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:22.655269    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="47799aa2-485f-47d4-9a94-16c1e5fe7c81" containerName="payload-processing"
Jul 27 19:57:22.655459 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:22.655286    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="50f626c1-0dd7-49b6-84a0-b91b65e40b4a" containerName="payload-pre-processing"
Jul 27 19:57:22.660001 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:22.659981    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-ee5db6ec-5d9487884d-b4tb5"
Jul 27 19:57:22.662956 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:22.662935    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-ee5db6ec\""
Jul 27 19:57:22.670939 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:22.670916    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-ee5db6ec-5d9487884d-b4tb5"]
Jul 27 19:57:22.685137 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:22.685109    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-ee5db6ec-75689d8fbb-qnqc4"]
Jul 27 19:57:22.688923 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:22.688907    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-ee5db6ec-75689d8fbb-qnqc4"
Jul 27 19:57:22.691455 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:22.691406    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-ee5db6ec-dockercfg-wbpp9\""
Jul 27 19:57:22.695831 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:22.695811    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-ee5db6ec-75689d8fbb-qnqc4"]
Jul 27 19:57:22.835705 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:22.835610    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8bgx\" (UniqueName: \"kubernetes.io/projected/9dcd505d-5eec-49e4-b6d6-5163c36793eb-kube-api-access-b8bgx\") pod \"payload-pre-processing-e2e-derive-ee5db6ec-5d9487884d-b4tb5\" (UID: \"9dcd505d-5eec-49e4-b6d6-5163c36793eb\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-ee5db6ec-5d9487884d-b4tb5"
Jul 27 19:57:22.835705 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:22.835655    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ea500cb-5ab4-4a06-94f3-a5d7e5f38f48-plugins-config-volume\") pod \"payload-processing-e2e-derive-ee5db6ec-75689d8fbb-qnqc4\" (UID: \"5ea500cb-5ab4-4a06-94f3-a5d7e5f38f48\") " pod="openshift-ingress/payload-processing-e2e-derive-ee5db6ec-75689d8fbb-qnqc4"
Jul 27 19:57:22.835989 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:22.835739    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9dcd505d-5eec-49e4-b6d6-5163c36793eb-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-ee5db6ec-5d9487884d-b4tb5\" (UID: \"9dcd505d-5eec-49e4-b6d6-5163c36793eb\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-ee5db6ec-5d9487884d-b4tb5"
Jul 27 19:57:22.835989 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:22.835782    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vl4v\" (UniqueName: \"kubernetes.io/projected/5ea500cb-5ab4-4a06-94f3-a5d7e5f38f48-kube-api-access-2vl4v\") pod \"payload-processing-e2e-derive-ee5db6ec-75689d8fbb-qnqc4\" (UID: \"5ea500cb-5ab4-4a06-94f3-a5d7e5f38f48\") " pod="openshift-ingress/payload-processing-e2e-derive-ee5db6ec-75689d8fbb-qnqc4"
Jul 27 19:57:22.936650 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:22.936615    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9dcd505d-5eec-49e4-b6d6-5163c36793eb-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-ee5db6ec-5d9487884d-b4tb5\" (UID: \"9dcd505d-5eec-49e4-b6d6-5163c36793eb\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-ee5db6ec-5d9487884d-b4tb5"
Jul 27 19:57:22.936650 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:22.936654    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vl4v\" (UniqueName: \"kubernetes.io/projected/5ea500cb-5ab4-4a06-94f3-a5d7e5f38f48-kube-api-access-2vl4v\") pod \"payload-processing-e2e-derive-ee5db6ec-75689d8fbb-qnqc4\" (UID: \"5ea500cb-5ab4-4a06-94f3-a5d7e5f38f48\") " pod="openshift-ingress/payload-processing-e2e-derive-ee5db6ec-75689d8fbb-qnqc4"
Jul 27 19:57:22.937229 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:22.936715    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8bgx\" (UniqueName: \"kubernetes.io/projected/9dcd505d-5eec-49e4-b6d6-5163c36793eb-kube-api-access-b8bgx\") pod \"payload-pre-processing-e2e-derive-ee5db6ec-5d9487884d-b4tb5\" (UID: \"9dcd505d-5eec-49e4-b6d6-5163c36793eb\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-ee5db6ec-5d9487884d-b4tb5"
Jul 27 19:57:22.937229 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:22.936736    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ea500cb-5ab4-4a06-94f3-a5d7e5f38f48-plugins-config-volume\") pod \"payload-processing-e2e-derive-ee5db6ec-75689d8fbb-qnqc4\" (UID: \"5ea500cb-5ab4-4a06-94f3-a5d7e5f38f48\") " pod="openshift-ingress/payload-processing-e2e-derive-ee5db6ec-75689d8fbb-qnqc4"
Jul 27 19:57:22.937401 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:22.937381    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ea500cb-5ab4-4a06-94f3-a5d7e5f38f48-plugins-config-volume\") pod \"payload-processing-e2e-derive-ee5db6ec-75689d8fbb-qnqc4\" (UID: \"5ea500cb-5ab4-4a06-94f3-a5d7e5f38f48\") " pod="openshift-ingress/payload-processing-e2e-derive-ee5db6ec-75689d8fbb-qnqc4"
Jul 27 19:57:22.937564 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:22.937543    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9dcd505d-5eec-49e4-b6d6-5163c36793eb-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-ee5db6ec-5d9487884d-b4tb5\" (UID: \"9dcd505d-5eec-49e4-b6d6-5163c36793eb\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-ee5db6ec-5d9487884d-b4tb5"
Jul 27 19:57:22.952103 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:22.952069    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8bgx\" (UniqueName: \"kubernetes.io/projected/9dcd505d-5eec-49e4-b6d6-5163c36793eb-kube-api-access-b8bgx\") pod \"payload-pre-processing-e2e-derive-ee5db6ec-5d9487884d-b4tb5\" (UID: \"9dcd505d-5eec-49e4-b6d6-5163c36793eb\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-ee5db6ec-5d9487884d-b4tb5"
Jul 27 19:57:22.952739 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:22.952722    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vl4v\" (UniqueName: \"kubernetes.io/projected/5ea500cb-5ab4-4a06-94f3-a5d7e5f38f48-kube-api-access-2vl4v\") pod \"payload-processing-e2e-derive-ee5db6ec-75689d8fbb-qnqc4\" (UID: \"5ea500cb-5ab4-4a06-94f3-a5d7e5f38f48\") " pod="openshift-ingress/payload-processing-e2e-derive-ee5db6ec-75689d8fbb-qnqc4"
Jul 27 19:57:22.972617 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:22.972591    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-ee5db6ec-5d9487884d-b4tb5"
Jul 27 19:57:22.999724 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:22.999702    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-ee5db6ec-75689d8fbb-qnqc4"
Jul 27 19:57:23.130164 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:23.130129    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-ee5db6ec-5d9487884d-b4tb5"]
Jul 27 19:57:23.134584 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:57:23.134549    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9dcd505d_5eec_49e4_b6d6_5163c36793eb.slice/crio-5d1bcb60375dbfe6853e7792c9ed9a76b05a6108c9e3904cddd74f906e73623a WatchSource:0}: Error finding container 5d1bcb60375dbfe6853e7792c9ed9a76b05a6108c9e3904cddd74f906e73623a: Status 404 returned error can't find the container with id 5d1bcb60375dbfe6853e7792c9ed9a76b05a6108c9e3904cddd74f906e73623a
Jul 27 19:57:23.160670 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:23.160642    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-ee5db6ec-75689d8fbb-qnqc4"]
Jul 27 19:57:23.166541 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:57:23.166510    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ea500cb_5ab4_4a06_94f3_a5d7e5f38f48.slice/crio-38c968dd2abb556fb22bf545385e0e9016d7443458800a5dcfb23344b0f18f41 WatchSource:0}: Error finding container 38c968dd2abb556fb22bf545385e0e9016d7443458800a5dcfb23344b0f18f41: Status 404 returned error can't find the container with id 38c968dd2abb556fb22bf545385e0e9016d7443458800a5dcfb23344b0f18f41
Jul 27 19:57:24.091159 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:24.091122    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-ee5db6ec-5d9487884d-b4tb5" event={"ID":"9dcd505d-5eec-49e4-b6d6-5163c36793eb","Type":"ContainerStarted","Data":"657410af161cdcc592709bc6c4af5a50787629ee7861d9c0ec66752ae2d93101"}
Jul 27 19:57:24.091617 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:24.091179    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-ee5db6ec-5d9487884d-b4tb5"
Jul 27 19:57:24.091617 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:24.091204    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-ee5db6ec-5d9487884d-b4tb5" event={"ID":"9dcd505d-5eec-49e4-b6d6-5163c36793eb","Type":"ContainerStarted","Data":"5d1bcb60375dbfe6853e7792c9ed9a76b05a6108c9e3904cddd74f906e73623a"}
Jul 27 19:57:24.092798 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:24.092775    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-ee5db6ec-75689d8fbb-qnqc4" event={"ID":"5ea500cb-5ab4-4a06-94f3-a5d7e5f38f48","Type":"ContainerStarted","Data":"1a96f4a1dd49f859c714946e8abffd7c5f0c3c33fa49862ae884e5c5d2cc900c"}
Jul 27 19:57:24.092934 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:24.092820    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-ee5db6ec-75689d8fbb-qnqc4" event={"ID":"5ea500cb-5ab4-4a06-94f3-a5d7e5f38f48","Type":"ContainerStarted","Data":"38c968dd2abb556fb22bf545385e0e9016d7443458800a5dcfb23344b0f18f41"}
Jul 27 19:57:24.092934 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:24.092862    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-ee5db6ec-75689d8fbb-qnqc4"
Jul 27 19:57:24.107423 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:24.107357    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-ee5db6ec-5d9487884d-b4tb5" podStartSLOduration=2.107342881 podStartE2EDuration="2.107342881s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:57:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:57:24.105316534 +0000 UTC m=+1856.977508508" watchObservedRunningTime="2026-07-27 19:57:24.107342881 +0000 UTC m=+1856.979534854"
Jul 27 19:57:24.119665 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:24.119610    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-ee5db6ec-75689d8fbb-qnqc4" podStartSLOduration=2.119591486 podStartE2EDuration="2.119591486s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:57:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:57:24.118884835 +0000 UTC m=+1856.991076810" watchObservedRunningTime="2026-07-27 19:57:24.119591486 +0000 UTC m=+1856.991783461"
Jul 27 19:57:35.100620 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:35.100581    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-ee5db6ec-75689d8fbb-qnqc4"
Jul 27 19:57:35.100997 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:35.100927    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-ee5db6ec-5d9487884d-b4tb5"
Jul 27 19:57:57.168534 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:57.168431    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:57:57.285777 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:57.285741    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-ee5db6ec-75689d8fbb-qnqc4"]
Jul 27 19:57:57.286053 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:57.286010    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-ee5db6ec-75689d8fbb-qnqc4" podUID="5ea500cb-5ab4-4a06-94f3-a5d7e5f38f48" containerName="payload-processing" containerID="cri-o://1a96f4a1dd49f859c714946e8abffd7c5f0c3c33fa49862ae884e5c5d2cc900c" gracePeriod=30
Jul 27 19:57:57.291210 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:57.291180    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-ee5db6ec-5d9487884d-b4tb5"]
Jul 27 19:57:57.291612 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:57.291566    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-ee5db6ec-5d9487884d-b4tb5" podUID="9dcd505d-5eec-49e4-b6d6-5163c36793eb" containerName="payload-pre-processing" containerID="cri-o://657410af161cdcc592709bc6c4af5a50787629ee7861d9c0ec66752ae2d93101" gracePeriod=30
Jul 27 19:57:57.316037 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:57.316001    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:57:57.389717 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:57.389685    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:57:57.456615 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:57.456531    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:57:57.573325 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:57.573255    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:57:57.582364 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:57.582342    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-ee5db6ec-75689d8fbb-qnqc4"
Jul 27 19:57:57.585234 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:57.585218    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-ee5db6ec-5d9487884d-b4tb5"
Jul 27 19:57:57.650174 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:57.650151    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2vl4v\" (UniqueName: \"kubernetes.io/projected/5ea500cb-5ab4-4a06-94f3-a5d7e5f38f48-kube-api-access-2vl4v\") pod \"5ea500cb-5ab4-4a06-94f3-a5d7e5f38f48\" (UID: \"5ea500cb-5ab4-4a06-94f3-a5d7e5f38f48\") "
Jul 27 19:57:57.650324 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:57.650186    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b8bgx\" (UniqueName: \"kubernetes.io/projected/9dcd505d-5eec-49e4-b6d6-5163c36793eb-kube-api-access-b8bgx\") pod \"9dcd505d-5eec-49e4-b6d6-5163c36793eb\" (UID: \"9dcd505d-5eec-49e4-b6d6-5163c36793eb\") "
Jul 27 19:57:57.650324 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:57.650263    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ea500cb-5ab4-4a06-94f3-a5d7e5f38f48-plugins-config-volume\") pod \"5ea500cb-5ab4-4a06-94f3-a5d7e5f38f48\" (UID: \"5ea500cb-5ab4-4a06-94f3-a5d7e5f38f48\") "
Jul 27 19:57:57.650324 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:57.650281    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9dcd505d-5eec-49e4-b6d6-5163c36793eb-plugins-config-volume\") pod \"9dcd505d-5eec-49e4-b6d6-5163c36793eb\" (UID: \"9dcd505d-5eec-49e4-b6d6-5163c36793eb\") "
Jul 27 19:57:57.650689 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:57.650665    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ea500cb-5ab4-4a06-94f3-a5d7e5f38f48-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5ea500cb-5ab4-4a06-94f3-a5d7e5f38f48" (UID: "5ea500cb-5ab4-4a06-94f3-a5d7e5f38f48"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:57:57.650758 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:57.650696    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9dcd505d-5eec-49e4-b6d6-5163c36793eb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9dcd505d-5eec-49e4-b6d6-5163c36793eb" (UID: "9dcd505d-5eec-49e4-b6d6-5163c36793eb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:57:57.652426 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:57.652390    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ea500cb-5ab4-4a06-94f3-a5d7e5f38f48-kube-api-access-2vl4v" (OuterVolumeSpecName: "kube-api-access-2vl4v") pod "5ea500cb-5ab4-4a06-94f3-a5d7e5f38f48" (UID: "5ea500cb-5ab4-4a06-94f3-a5d7e5f38f48"). InnerVolumeSpecName "kube-api-access-2vl4v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:57:57.652500 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:57.652397    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9dcd505d-5eec-49e4-b6d6-5163c36793eb-kube-api-access-b8bgx" (OuterVolumeSpecName: "kube-api-access-b8bgx") pod "9dcd505d-5eec-49e4-b6d6-5163c36793eb" (UID: "9dcd505d-5eec-49e4-b6d6-5163c36793eb"). InnerVolumeSpecName "kube-api-access-b8bgx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:57:57.751339 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:57.751275    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2vl4v\" (UniqueName: \"kubernetes.io/projected/5ea500cb-5ab4-4a06-94f3-a5d7e5f38f48-kube-api-access-2vl4v\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:57:57.751339 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:57.751301    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b8bgx\" (UniqueName: \"kubernetes.io/projected/9dcd505d-5eec-49e4-b6d6-5163c36793eb-kube-api-access-b8bgx\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:57:57.751339 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:57.751309    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ea500cb-5ab4-4a06-94f3-a5d7e5f38f48-plugins-config-volume\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:57:57.751339 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:57.751318    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9dcd505d-5eec-49e4-b6d6-5163c36793eb-plugins-config-volume\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 19:57:57.824157 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:57.824120    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:57:58.261881 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:58.261840    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:57:58.278010 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:58.277980    2558 generic.go:356] "Generic (PLEG): container finished" podID="9dcd505d-5eec-49e4-b6d6-5163c36793eb" containerID="657410af161cdcc592709bc6c4af5a50787629ee7861d9c0ec66752ae2d93101" exitCode=0
Jul 27 19:57:58.278155 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:58.278030    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-ee5db6ec-5d9487884d-b4tb5"
Jul 27 19:57:58.278155 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:58.278077    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-ee5db6ec-5d9487884d-b4tb5" event={"ID":"9dcd505d-5eec-49e4-b6d6-5163c36793eb","Type":"ContainerDied","Data":"657410af161cdcc592709bc6c4af5a50787629ee7861d9c0ec66752ae2d93101"}
Jul 27 19:57:58.278155 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:58.278136    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-ee5db6ec-5d9487884d-b4tb5" event={"ID":"9dcd505d-5eec-49e4-b6d6-5163c36793eb","Type":"ContainerDied","Data":"5d1bcb60375dbfe6853e7792c9ed9a76b05a6108c9e3904cddd74f906e73623a"}
Jul 27 19:57:58.278293 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:58.278169    2558 scope.go:117] "RemoveContainer" containerID="657410af161cdcc592709bc6c4af5a50787629ee7861d9c0ec66752ae2d93101"
Jul 27 19:57:58.279658 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:58.279622    2558 generic.go:356] "Generic (PLEG): container finished" podID="5ea500cb-5ab4-4a06-94f3-a5d7e5f38f48" containerID="1a96f4a1dd49f859c714946e8abffd7c5f0c3c33fa49862ae884e5c5d2cc900c" exitCode=0
Jul 27 19:57:58.279779 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:58.279684    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-ee5db6ec-75689d8fbb-qnqc4"
Jul 27 19:57:58.279779 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:58.279720    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-ee5db6ec-75689d8fbb-qnqc4" event={"ID":"5ea500cb-5ab4-4a06-94f3-a5d7e5f38f48","Type":"ContainerDied","Data":"1a96f4a1dd49f859c714946e8abffd7c5f0c3c33fa49862ae884e5c5d2cc900c"}
Jul 27 19:57:58.279779 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:58.279762    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-ee5db6ec-75689d8fbb-qnqc4" event={"ID":"5ea500cb-5ab4-4a06-94f3-a5d7e5f38f48","Type":"ContainerDied","Data":"38c968dd2abb556fb22bf545385e0e9016d7443458800a5dcfb23344b0f18f41"}
Jul 27 19:57:58.287557 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:58.287319    2558 scope.go:117] "RemoveContainer" containerID="657410af161cdcc592709bc6c4af5a50787629ee7861d9c0ec66752ae2d93101"
Jul 27 19:57:58.287639 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:57:58.287616    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"657410af161cdcc592709bc6c4af5a50787629ee7861d9c0ec66752ae2d93101\": container with ID starting with 657410af161cdcc592709bc6c4af5a50787629ee7861d9c0ec66752ae2d93101 not found: ID does not exist" containerID="657410af161cdcc592709bc6c4af5a50787629ee7861d9c0ec66752ae2d93101"
Jul 27 19:57:58.287758 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:58.287646    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"657410af161cdcc592709bc6c4af5a50787629ee7861d9c0ec66752ae2d93101"} err="failed to get container status \"657410af161cdcc592709bc6c4af5a50787629ee7861d9c0ec66752ae2d93101\": rpc error: code = NotFound desc = could not find container \"657410af161cdcc592709bc6c4af5a50787629ee7861d9c0ec66752ae2d93101\": container with ID starting with 657410af161cdcc592709bc6c4af5a50787629ee7861d9c0ec66752ae2d93101 not found: ID does not exist"
Jul 27 19:57:58.287758 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:58.287669    2558 scope.go:117] "RemoveContainer" containerID="1a96f4a1dd49f859c714946e8abffd7c5f0c3c33fa49862ae884e5c5d2cc900c"
Jul 27 19:57:58.296194 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:58.296174    2558 scope.go:117] "RemoveContainer" containerID="1a96f4a1dd49f859c714946e8abffd7c5f0c3c33fa49862ae884e5c5d2cc900c"
Jul 27 19:57:58.296522 ip-10-0-129-235 kubenswrapper[2558]: E0727 19:57:58.296492    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a96f4a1dd49f859c714946e8abffd7c5f0c3c33fa49862ae884e5c5d2cc900c\": container with ID starting with 1a96f4a1dd49f859c714946e8abffd7c5f0c3c33fa49862ae884e5c5d2cc900c not found: ID does not exist" containerID="1a96f4a1dd49f859c714946e8abffd7c5f0c3c33fa49862ae884e5c5d2cc900c"
Jul 27 19:57:58.296606 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:58.296529    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a96f4a1dd49f859c714946e8abffd7c5f0c3c33fa49862ae884e5c5d2cc900c"} err="failed to get container status \"1a96f4a1dd49f859c714946e8abffd7c5f0c3c33fa49862ae884e5c5d2cc900c\": rpc error: code = NotFound desc = could not find container \"1a96f4a1dd49f859c714946e8abffd7c5f0c3c33fa49862ae884e5c5d2cc900c\": container with ID starting with 1a96f4a1dd49f859c714946e8abffd7c5f0c3c33fa49862ae884e5c5d2cc900c not found: ID does not exist"
Jul 27 19:57:58.303856 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:58.303752    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-ee5db6ec-5d9487884d-b4tb5"]
Jul 27 19:57:58.309785 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:58.309752    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-ee5db6ec-5d9487884d-b4tb5"]
Jul 27 19:57:58.321251 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:58.321227    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-ee5db6ec-75689d8fbb-qnqc4"]
Jul 27 19:57:58.328677 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:58.328646    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-ee5db6ec-75689d8fbb-qnqc4"]
Jul 27 19:57:58.334362 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:58.334337    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:57:58.400532 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:58.400503    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:57:59.675458 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:59.675399    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ea500cb-5ab4-4a06-94f3-a5d7e5f38f48" path="/var/lib/kubelet/pods/5ea500cb-5ab4-4a06-94f3-a5d7e5f38f48/volumes"
Jul 27 19:57:59.675833 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:59.675786    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9dcd505d-5eec-49e4-b6d6-5163c36793eb" path="/var/lib/kubelet/pods/9dcd505d-5eec-49e4-b6d6-5163c36793eb/volumes"
Jul 27 19:57:59.825441 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:57:59.825382    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:58:06.205062 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:58:06.205022    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:58:06.308774 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:58:06.308735    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:58:06.410132 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:58:06.410099    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:58:06.507624 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:58:06.507533    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:58:06.900937 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:58:06.900904    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:58:32.844090 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:58:32.844051    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:58:39.392265 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:58:39.392224    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:58:39.453585 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:58:39.453547    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:58:39.527658 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:58:39.527615    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:58:39.627431 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:58:39.627379    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:58:55.416541 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:58:55.416507    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:59:01.540940 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:01.540898    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:59:01.640620 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:01.640587    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:59:01.740616 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:01.740574    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:59:12.277759 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:12.277726    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:59:13.015119 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:13.015090    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:59:19.615808 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:19.615726    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:59:21.184241 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:21.184210    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:59:22.343053 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:22.343014    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:59:28.112084 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:28.112043    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:59:28.208989 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:28.208951    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:59:28.773019 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:28.772991    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:59:35.345294 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:35.345263    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:59:35.446208 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:35.446174    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:59:35.574854 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:35.574825    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:59:35.880214 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:35.880183    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:59:36.059994 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:36.059947    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:59:38.644546 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:38.644512    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:59:38.745798 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:38.745767    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:59:38.849573 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:38.849531    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:59:38.988683 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:38.988598    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:59:39.237113 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:39.237083    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:59:39.317655 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:39.317564    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:59:39.322917 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:39.322886    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c7847b84-7c6c58cc9b-jqjwq"]
Jul 27 19:59:39.323449 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:39.323406    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ea500cb-5ab4-4a06-94f3-a5d7e5f38f48" containerName="payload-processing"
Jul 27 19:59:39.323560 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:39.323451    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ea500cb-5ab4-4a06-94f3-a5d7e5f38f48" containerName="payload-processing"
Jul 27 19:59:39.323560 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:39.323488    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9dcd505d-5eec-49e4-b6d6-5163c36793eb" containerName="payload-pre-processing"
Jul 27 19:59:39.323560 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:39.323496    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="9dcd505d-5eec-49e4-b6d6-5163c36793eb" containerName="payload-pre-processing"
Jul 27 19:59:39.323713 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:39.323597    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="9dcd505d-5eec-49e4-b6d6-5163c36793eb" containerName="payload-pre-processing"
Jul 27 19:59:39.323713 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:39.323609    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ea500cb-5ab4-4a06-94f3-a5d7e5f38f48" containerName="payload-processing"
Jul 27 19:59:39.326952 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:39.326931    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-c7847b84-7c6c58cc9b-jqjwq"
Jul 27 19:59:39.329900 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:39.329872    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-c7847b84\""
Jul 27 19:59:39.331330 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:39.331313    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-c7847b84-dockercfg-smd5m\""
Jul 27 19:59:39.336354 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:39.336329    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c7847b84-7c6c58cc9b-jqjwq"]
Jul 27 19:59:39.341836 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:39.341812    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c7847b84-6cd6dcf8bb-z2k44"]
Jul 27 19:59:39.345267 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:39.345247    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-c7847b84-6cd6dcf8bb-z2k44"
Jul 27 19:59:39.354474 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:39.354451    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c7847b84-6cd6dcf8bb-z2k44"]
Jul 27 19:59:39.459654 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:39.459621    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9v62\" (UniqueName: \"kubernetes.io/projected/23996096-3828-40c8-9d8d-ca60e7623ab3-kube-api-access-v9v62\") pod \"payload-processing-e2e-mt-c7847b84-6cd6dcf8bb-z2k44\" (UID: \"23996096-3828-40c8-9d8d-ca60e7623ab3\") " pod="openshift-ingress/payload-processing-e2e-mt-c7847b84-6cd6dcf8bb-z2k44"
Jul 27 19:59:39.459654 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:39.459651    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5a8306d6-27db-46a1-b426-368b854f0eb1-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-c7847b84-7c6c58cc9b-jqjwq\" (UID: \"5a8306d6-27db-46a1-b426-368b854f0eb1\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c7847b84-7c6c58cc9b-jqjwq"
Jul 27 19:59:39.459843 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:39.459695    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6pg5d\" (UniqueName: \"kubernetes.io/projected/5a8306d6-27db-46a1-b426-368b854f0eb1-kube-api-access-6pg5d\") pod \"payload-pre-processing-e2e-mt-c7847b84-7c6c58cc9b-jqjwq\" (UID: \"5a8306d6-27db-46a1-b426-368b854f0eb1\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c7847b84-7c6c58cc9b-jqjwq"
Jul 27 19:59:39.459843 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:39.459752    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/23996096-3828-40c8-9d8d-ca60e7623ab3-plugins-config-volume\") pod \"payload-processing-e2e-mt-c7847b84-6cd6dcf8bb-z2k44\" (UID: \"23996096-3828-40c8-9d8d-ca60e7623ab3\") " pod="openshift-ingress/payload-processing-e2e-mt-c7847b84-6cd6dcf8bb-z2k44"
Jul 27 19:59:39.561201 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:39.561153    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9v62\" (UniqueName: \"kubernetes.io/projected/23996096-3828-40c8-9d8d-ca60e7623ab3-kube-api-access-v9v62\") pod \"payload-processing-e2e-mt-c7847b84-6cd6dcf8bb-z2k44\" (UID: \"23996096-3828-40c8-9d8d-ca60e7623ab3\") " pod="openshift-ingress/payload-processing-e2e-mt-c7847b84-6cd6dcf8bb-z2k44"
Jul 27 19:59:39.561201 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:39.561201    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5a8306d6-27db-46a1-b426-368b854f0eb1-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-c7847b84-7c6c58cc9b-jqjwq\" (UID: \"5a8306d6-27db-46a1-b426-368b854f0eb1\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c7847b84-7c6c58cc9b-jqjwq"
Jul 27 19:59:39.561453 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:39.561248    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pg5d\" (UniqueName: \"kubernetes.io/projected/5a8306d6-27db-46a1-b426-368b854f0eb1-kube-api-access-6pg5d\") pod \"payload-pre-processing-e2e-mt-c7847b84-7c6c58cc9b-jqjwq\" (UID: \"5a8306d6-27db-46a1-b426-368b854f0eb1\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c7847b84-7c6c58cc9b-jqjwq"
Jul 27 19:59:39.561453 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:39.561272    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/23996096-3828-40c8-9d8d-ca60e7623ab3-plugins-config-volume\") pod \"payload-processing-e2e-mt-c7847b84-6cd6dcf8bb-z2k44\" (UID: \"23996096-3828-40c8-9d8d-ca60e7623ab3\") " pod="openshift-ingress/payload-processing-e2e-mt-c7847b84-6cd6dcf8bb-z2k44"
Jul 27 19:59:39.561945 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:39.561927    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/23996096-3828-40c8-9d8d-ca60e7623ab3-plugins-config-volume\") pod \"payload-processing-e2e-mt-c7847b84-6cd6dcf8bb-z2k44\" (UID: \"23996096-3828-40c8-9d8d-ca60e7623ab3\") " pod="openshift-ingress/payload-processing-e2e-mt-c7847b84-6cd6dcf8bb-z2k44"
Jul 27 19:59:39.561999 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:39.561982    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5a8306d6-27db-46a1-b426-368b854f0eb1-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-c7847b84-7c6c58cc9b-jqjwq\" (UID: \"5a8306d6-27db-46a1-b426-368b854f0eb1\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c7847b84-7c6c58cc9b-jqjwq"
Jul 27 19:59:39.572967 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:39.572903    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6pg5d\" (UniqueName: \"kubernetes.io/projected/5a8306d6-27db-46a1-b426-368b854f0eb1-kube-api-access-6pg5d\") pod \"payload-pre-processing-e2e-mt-c7847b84-7c6c58cc9b-jqjwq\" (UID: \"5a8306d6-27db-46a1-b426-368b854f0eb1\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c7847b84-7c6c58cc9b-jqjwq"
Jul 27 19:59:39.572967 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:39.572922    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9v62\" (UniqueName: \"kubernetes.io/projected/23996096-3828-40c8-9d8d-ca60e7623ab3-kube-api-access-v9v62\") pod \"payload-processing-e2e-mt-c7847b84-6cd6dcf8bb-z2k44\" (UID: \"23996096-3828-40c8-9d8d-ca60e7623ab3\") " pod="openshift-ingress/payload-processing-e2e-mt-c7847b84-6cd6dcf8bb-z2k44"
Jul 27 19:59:39.638895 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:39.638863    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-c7847b84-7c6c58cc9b-jqjwq"
Jul 27 19:59:39.656203 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:39.656179    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-c7847b84-6cd6dcf8bb-z2k44"
Jul 27 19:59:39.788516 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:39.788402    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c7847b84-7c6c58cc9b-jqjwq"]
Jul 27 19:59:39.792150 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:59:39.792121    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a8306d6_27db_46a1_b426_368b854f0eb1.slice/crio-56d969f7c5663c75b72a542847ebbef2b22b20d3e79ab20e3d4818011e78eb8e WatchSource:0}: Error finding container 56d969f7c5663c75b72a542847ebbef2b22b20d3e79ab20e3d4818011e78eb8e: Status 404 returned error can't find the container with id 56d969f7c5663c75b72a542847ebbef2b22b20d3e79ab20e3d4818011e78eb8e
Jul 27 19:59:39.818049 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:39.818024    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c7847b84-6cd6dcf8bb-z2k44"]
Jul 27 19:59:39.819616 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:39.819591    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-c7847b84-7c6c58cc9b-jqjwq" event={"ID":"5a8306d6-27db-46a1-b426-368b854f0eb1","Type":"ContainerStarted","Data":"56d969f7c5663c75b72a542847ebbef2b22b20d3e79ab20e3d4818011e78eb8e"}
Jul 27 19:59:39.821694 ip-10-0-129-235 kubenswrapper[2558]: W0727 19:59:39.821666    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23996096_3828_40c8_9d8d_ca60e7623ab3.slice/crio-318883279ccf22c8dfe1e232adc17b6af5219a393e11a768ed38fa40de7fe5a7 WatchSource:0}: Error finding container 318883279ccf22c8dfe1e232adc17b6af5219a393e11a768ed38fa40de7fe5a7: Status 404 returned error can't find the container with id 318883279ccf22c8dfe1e232adc17b6af5219a393e11a768ed38fa40de7fe5a7
Jul 27 19:59:40.827461 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:40.827390    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-c7847b84-6cd6dcf8bb-z2k44" event={"ID":"23996096-3828-40c8-9d8d-ca60e7623ab3","Type":"ContainerStarted","Data":"5a6fa8ba98e3e07b8843ef7c0693558cb00c08653ad89c256038bd199dc86547"}
Jul 27 19:59:40.827947 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:40.827489    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-c7847b84-6cd6dcf8bb-z2k44"
Jul 27 19:59:40.827947 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:40.827511    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-c7847b84-6cd6dcf8bb-z2k44" event={"ID":"23996096-3828-40c8-9d8d-ca60e7623ab3","Type":"ContainerStarted","Data":"318883279ccf22c8dfe1e232adc17b6af5219a393e11a768ed38fa40de7fe5a7"}
Jul 27 19:59:40.829041 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:40.829012    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-c7847b84-7c6c58cc9b-jqjwq" event={"ID":"5a8306d6-27db-46a1-b426-368b854f0eb1","Type":"ContainerStarted","Data":"e8690058f723f118fd09631ba6cc18ee51baa203badc8172fbfcd1bde5efaa95"}
Jul 27 19:59:40.829178 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:40.829160    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-c7847b84-7c6c58cc9b-jqjwq"
Jul 27 19:59:40.843681 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:40.843634    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-c7847b84-6cd6dcf8bb-z2k44" podStartSLOduration=1.843618396 podStartE2EDuration="1.843618396s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:59:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:59:40.841668097 +0000 UTC m=+1993.713860074" watchObservedRunningTime="2026-07-27 19:59:40.843618396 +0000 UTC m=+1993.715810369"
Jul 27 19:59:40.857099 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:40.857058    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-c7847b84-7c6c58cc9b-jqjwq" podStartSLOduration=1.857044448 podStartE2EDuration="1.857044448s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:59:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:59:40.855551522 +0000 UTC m=+1993.727743497" watchObservedRunningTime="2026-07-27 19:59:40.857044448 +0000 UTC m=+1993.729236422"
Jul 27 19:59:43.134069 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:43.134031    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:59:43.762039 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:43.761999    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:59:43.861056 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:43.861014    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:59:44.191573 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:44.191525    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:59:50.309036 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:50.309000    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:59:50.451695 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:50.451635    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 19:59:51.845025 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:51.844990    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-c7847b84-6cd6dcf8bb-z2k44"
Jul 27 19:59:51.845485 ip-10-0-129-235 kubenswrapper[2558]: I0727 19:59:51.845100    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-c7847b84-7c6c58cc9b-jqjwq"
Jul 27 20:00:00.786740 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:00.786705    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:00:00.815937 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:00.807722    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c7847b84-6cd6dcf8bb-z2k44"]
Jul 27 20:00:00.815937 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:00.808165    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-c7847b84-6cd6dcf8bb-z2k44" podUID="23996096-3828-40c8-9d8d-ca60e7623ab3" containerName="payload-processing" containerID="cri-o://5a6fa8ba98e3e07b8843ef7c0693558cb00c08653ad89c256038bd199dc86547" gracePeriod=30
Jul 27 20:00:00.824136 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:00.823828    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c7847b84-7c6c58cc9b-jqjwq"]
Jul 27 20:00:00.824298 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:00.824182    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-c7847b84-7c6c58cc9b-jqjwq" podUID="5a8306d6-27db-46a1-b426-368b854f0eb1" containerName="payload-pre-processing" containerID="cri-o://e8690058f723f118fd09631ba6cc18ee51baa203badc8172fbfcd1bde5efaa95" gracePeriod=30
Jul 27 20:00:00.858142 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:00.858114    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:00:00.957025 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:00.956991    2558 generic.go:356] "Generic (PLEG): container finished" podID="23996096-3828-40c8-9d8d-ca60e7623ab3" containerID="5a6fa8ba98e3e07b8843ef7c0693558cb00c08653ad89c256038bd199dc86547" exitCode=0
Jul 27 20:00:00.957204 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:00.957074    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-c7847b84-6cd6dcf8bb-z2k44" event={"ID":"23996096-3828-40c8-9d8d-ca60e7623ab3","Type":"ContainerDied","Data":"5a6fa8ba98e3e07b8843ef7c0693558cb00c08653ad89c256038bd199dc86547"}
Jul 27 20:00:00.959020 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:00.958994    2558 generic.go:356] "Generic (PLEG): container finished" podID="5a8306d6-27db-46a1-b426-368b854f0eb1" containerID="e8690058f723f118fd09631ba6cc18ee51baa203badc8172fbfcd1bde5efaa95" exitCode=0
Jul 27 20:00:00.959160 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:00.959028    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-c7847b84-7c6c58cc9b-jqjwq" event={"ID":"5a8306d6-27db-46a1-b426-368b854f0eb1","Type":"ContainerDied","Data":"e8690058f723f118fd09631ba6cc18ee51baa203badc8172fbfcd1bde5efaa95"}
Jul 27 20:00:01.015819 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:01.015750    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:00:01.070146 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:01.070126    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-c7847b84-6cd6dcf8bb-z2k44"
Jul 27 20:00:01.100100 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:01.100080    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-c7847b84-7c6c58cc9b-jqjwq"
Jul 27 20:00:01.145572 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:01.145543    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6pg5d\" (UniqueName: \"kubernetes.io/projected/5a8306d6-27db-46a1-b426-368b854f0eb1-kube-api-access-6pg5d\") pod \"5a8306d6-27db-46a1-b426-368b854f0eb1\" (UID: \"5a8306d6-27db-46a1-b426-368b854f0eb1\") "
Jul 27 20:00:01.145738 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:01.145577    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/23996096-3828-40c8-9d8d-ca60e7623ab3-plugins-config-volume\") pod \"23996096-3828-40c8-9d8d-ca60e7623ab3\" (UID: \"23996096-3828-40c8-9d8d-ca60e7623ab3\") "
Jul 27 20:00:01.145738 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:01.145606    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5a8306d6-27db-46a1-b426-368b854f0eb1-plugins-config-volume\") pod \"5a8306d6-27db-46a1-b426-368b854f0eb1\" (UID: \"5a8306d6-27db-46a1-b426-368b854f0eb1\") "
Jul 27 20:00:01.145867 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:01.145743    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v9v62\" (UniqueName: \"kubernetes.io/projected/23996096-3828-40c8-9d8d-ca60e7623ab3-kube-api-access-v9v62\") pod \"23996096-3828-40c8-9d8d-ca60e7623ab3\" (UID: \"23996096-3828-40c8-9d8d-ca60e7623ab3\") "
Jul 27 20:00:01.145952 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:01.145931    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23996096-3828-40c8-9d8d-ca60e7623ab3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "23996096-3828-40c8-9d8d-ca60e7623ab3" (UID: "23996096-3828-40c8-9d8d-ca60e7623ab3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 20:00:01.145996 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:01.145957    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a8306d6-27db-46a1-b426-368b854f0eb1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5a8306d6-27db-46a1-b426-368b854f0eb1" (UID: "5a8306d6-27db-46a1-b426-368b854f0eb1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 20:00:01.146081 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:01.146067    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/23996096-3828-40c8-9d8d-ca60e7623ab3-plugins-config-volume\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:00:01.146147 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:01.146083    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5a8306d6-27db-46a1-b426-368b854f0eb1-plugins-config-volume\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:00:01.147972 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:01.147952    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23996096-3828-40c8-9d8d-ca60e7623ab3-kube-api-access-v9v62" (OuterVolumeSpecName: "kube-api-access-v9v62") pod "23996096-3828-40c8-9d8d-ca60e7623ab3" (UID: "23996096-3828-40c8-9d8d-ca60e7623ab3"). InnerVolumeSpecName "kube-api-access-v9v62". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:00:01.148265 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:01.148235    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a8306d6-27db-46a1-b426-368b854f0eb1-kube-api-access-6pg5d" (OuterVolumeSpecName: "kube-api-access-6pg5d") pod "5a8306d6-27db-46a1-b426-368b854f0eb1" (UID: "5a8306d6-27db-46a1-b426-368b854f0eb1"). InnerVolumeSpecName "kube-api-access-6pg5d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:00:01.247040 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:01.247008    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6pg5d\" (UniqueName: \"kubernetes.io/projected/5a8306d6-27db-46a1-b426-368b854f0eb1-kube-api-access-6pg5d\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:00:01.247040 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:01.247034    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v9v62\" (UniqueName: \"kubernetes.io/projected/23996096-3828-40c8-9d8d-ca60e7623ab3-kube-api-access-v9v62\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:00:01.371825 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:01.371775    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:00:01.965938 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:01.965849    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-c7847b84-7c6c58cc9b-jqjwq"
Jul 27 20:00:01.965938 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:01.965885    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-c7847b84-7c6c58cc9b-jqjwq" event={"ID":"5a8306d6-27db-46a1-b426-368b854f0eb1","Type":"ContainerDied","Data":"56d969f7c5663c75b72a542847ebbef2b22b20d3e79ab20e3d4818011e78eb8e"}
Jul 27 20:00:01.966391 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:01.965942    2558 scope.go:117] "RemoveContainer" containerID="e8690058f723f118fd09631ba6cc18ee51baa203badc8172fbfcd1bde5efaa95"
Jul 27 20:00:01.967399 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:01.967376    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-c7847b84-6cd6dcf8bb-z2k44" event={"ID":"23996096-3828-40c8-9d8d-ca60e7623ab3","Type":"ContainerDied","Data":"318883279ccf22c8dfe1e232adc17b6af5219a393e11a768ed38fa40de7fe5a7"}
Jul 27 20:00:01.967509 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:01.967406    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-c7847b84-6cd6dcf8bb-z2k44"
Jul 27 20:00:01.975601 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:01.975570    2558 scope.go:117] "RemoveContainer" containerID="5a6fa8ba98e3e07b8843ef7c0693558cb00c08653ad89c256038bd199dc86547"
Jul 27 20:00:01.984880 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:01.984861    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c7847b84-7c6c58cc9b-jqjwq"]
Jul 27 20:00:01.994117 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:01.994093    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c7847b84-7c6c58cc9b-jqjwq"]
Jul 27 20:00:02.004206 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:02.004182    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c7847b84-6cd6dcf8bb-z2k44"]
Jul 27 20:00:02.007997 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:02.007974    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c7847b84-6cd6dcf8bb-z2k44"]
Jul 27 20:00:03.674397 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:03.674364    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23996096-3828-40c8-9d8d-ca60e7623ab3" path="/var/lib/kubelet/pods/23996096-3828-40c8-9d8d-ca60e7623ab3/volumes"
Jul 27 20:00:03.674789 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:03.674757    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a8306d6-27db-46a1-b426-368b854f0eb1" path="/var/lib/kubelet/pods/5a8306d6-27db-46a1-b426-368b854f0eb1/volumes"
Jul 27 20:00:07.530853 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:07.530815    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:00:07.595074 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:07.595042    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:00:07.775712 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:07.775678    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:00:07.899679 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:07.899642    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:00:07.994390 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:07.994351    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:00:08.109917 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:08.109879    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:00:14.470669 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:14.470628    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:00:14.579991 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:14.579958    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:00:14.734077 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:14.733981    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:00:14.888894 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:14.888862    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:00:21.021253 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:21.021213    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:00:21.092548 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:21.092513    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:00:28.039368 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:28.039319    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:00:28.169726 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:28.169693    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:00:28.259264 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:28.259228    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:00:30.944363 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:30.944326    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:00:31.024291 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:31.024257    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:00:31.080988 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:31.080945    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:00:31.316254 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:31.316168    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:00:31.629296 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:31.629259    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:00:31.726568 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:31.726540    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:00:31.731056 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:31.731031    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-73760660-67c49cc76-blhp6"]
Jul 27 20:00:31.731398 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:31.731385    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23996096-3828-40c8-9d8d-ca60e7623ab3" containerName="payload-processing"
Jul 27 20:00:31.731464 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:31.731401    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="23996096-3828-40c8-9d8d-ca60e7623ab3" containerName="payload-processing"
Jul 27 20:00:31.731464 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:31.731436    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a8306d6-27db-46a1-b426-368b854f0eb1" containerName="payload-pre-processing"
Jul 27 20:00:31.731464 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:31.731445    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a8306d6-27db-46a1-b426-368b854f0eb1" containerName="payload-pre-processing"
Jul 27 20:00:31.731558 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:31.731528    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a8306d6-27db-46a1-b426-368b854f0eb1" containerName="payload-pre-processing"
Jul 27 20:00:31.731558 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:31.731539    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="23996096-3828-40c8-9d8d-ca60e7623ab3" containerName="payload-processing"
Jul 27 20:00:31.734543 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:31.734528    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-73760660-67c49cc76-blhp6"
Jul 27 20:00:31.736753 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:31.736736    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-73760660\""
Jul 27 20:00:31.736862 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:31.736770    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-73760660-dockercfg-56qc7\""
Jul 27 20:00:31.745394 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:31.745371    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-73760660-67c49cc76-blhp6"]
Jul 27 20:00:31.757732 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:31.757705    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-73760660-8ddd65d58-h9p57"]
Jul 27 20:00:31.761470 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:31.761452    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-73760660-8ddd65d58-h9p57"
Jul 27 20:00:31.769872 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:31.769852    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-73760660-8ddd65d58-h9p57"]
Jul 27 20:00:31.809055 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:31.809027    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9baf5281-a1d8-433e-aec3-a36d644cea81-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-73760660-67c49cc76-blhp6\" (UID: \"9baf5281-a1d8-433e-aec3-a36d644cea81\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-73760660-67c49cc76-blhp6"
Jul 27 20:00:31.809190 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:31.809090    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mj2lw\" (UniqueName: \"kubernetes.io/projected/9baf5281-a1d8-433e-aec3-a36d644cea81-kube-api-access-mj2lw\") pod \"payload-pre-processing-e2e-mt-73760660-67c49cc76-blhp6\" (UID: \"9baf5281-a1d8-433e-aec3-a36d644cea81\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-73760660-67c49cc76-blhp6"
Jul 27 20:00:31.910226 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:31.910153    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9baf5281-a1d8-433e-aec3-a36d644cea81-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-73760660-67c49cc76-blhp6\" (UID: \"9baf5281-a1d8-433e-aec3-a36d644cea81\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-73760660-67c49cc76-blhp6"
Jul 27 20:00:31.910226 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:31.910194    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mj2lw\" (UniqueName: \"kubernetes.io/projected/9baf5281-a1d8-433e-aec3-a36d644cea81-kube-api-access-mj2lw\") pod \"payload-pre-processing-e2e-mt-73760660-67c49cc76-blhp6\" (UID: \"9baf5281-a1d8-433e-aec3-a36d644cea81\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-73760660-67c49cc76-blhp6"
Jul 27 20:00:31.910401 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:31.910335    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcttx\" (UniqueName: \"kubernetes.io/projected/ca113aca-92ad-4082-a364-02390fb0a233-kube-api-access-qcttx\") pod \"payload-processing-e2e-mt-73760660-8ddd65d58-h9p57\" (UID: \"ca113aca-92ad-4082-a364-02390fb0a233\") " pod="openshift-ingress/payload-processing-e2e-mt-73760660-8ddd65d58-h9p57"
Jul 27 20:00:31.910487 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:31.910467    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ca113aca-92ad-4082-a364-02390fb0a233-plugins-config-volume\") pod \"payload-processing-e2e-mt-73760660-8ddd65d58-h9p57\" (UID: \"ca113aca-92ad-4082-a364-02390fb0a233\") " pod="openshift-ingress/payload-processing-e2e-mt-73760660-8ddd65d58-h9p57"
Jul 27 20:00:31.910916 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:31.910898    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9baf5281-a1d8-433e-aec3-a36d644cea81-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-73760660-67c49cc76-blhp6\" (UID: \"9baf5281-a1d8-433e-aec3-a36d644cea81\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-73760660-67c49cc76-blhp6"
Jul 27 20:00:31.918228 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:31.918203    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mj2lw\" (UniqueName: \"kubernetes.io/projected/9baf5281-a1d8-433e-aec3-a36d644cea81-kube-api-access-mj2lw\") pod \"payload-pre-processing-e2e-mt-73760660-67c49cc76-blhp6\" (UID: \"9baf5281-a1d8-433e-aec3-a36d644cea81\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-73760660-67c49cc76-blhp6"
Jul 27 20:00:32.011491 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:32.011461    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ca113aca-92ad-4082-a364-02390fb0a233-plugins-config-volume\") pod \"payload-processing-e2e-mt-73760660-8ddd65d58-h9p57\" (UID: \"ca113aca-92ad-4082-a364-02390fb0a233\") " pod="openshift-ingress/payload-processing-e2e-mt-73760660-8ddd65d58-h9p57"
Jul 27 20:00:32.011920 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:32.011529    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcttx\" (UniqueName: \"kubernetes.io/projected/ca113aca-92ad-4082-a364-02390fb0a233-kube-api-access-qcttx\") pod \"payload-processing-e2e-mt-73760660-8ddd65d58-h9p57\" (UID: \"ca113aca-92ad-4082-a364-02390fb0a233\") " pod="openshift-ingress/payload-processing-e2e-mt-73760660-8ddd65d58-h9p57"
Jul 27 20:00:32.012244 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:32.012219    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ca113aca-92ad-4082-a364-02390fb0a233-plugins-config-volume\") pod \"payload-processing-e2e-mt-73760660-8ddd65d58-h9p57\" (UID: \"ca113aca-92ad-4082-a364-02390fb0a233\") " pod="openshift-ingress/payload-processing-e2e-mt-73760660-8ddd65d58-h9p57"
Jul 27 20:00:32.018925 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:32.018898    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcttx\" (UniqueName: \"kubernetes.io/projected/ca113aca-92ad-4082-a364-02390fb0a233-kube-api-access-qcttx\") pod \"payload-processing-e2e-mt-73760660-8ddd65d58-h9p57\" (UID: \"ca113aca-92ad-4082-a364-02390fb0a233\") " pod="openshift-ingress/payload-processing-e2e-mt-73760660-8ddd65d58-h9p57"
Jul 27 20:00:32.045878 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:32.045857    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-73760660-67c49cc76-blhp6"
Jul 27 20:00:32.072155 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:32.072126    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-73760660-8ddd65d58-h9p57"
Jul 27 20:00:32.194609 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:32.194576    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-73760660-67c49cc76-blhp6"]
Jul 27 20:00:32.196675 ip-10-0-129-235 kubenswrapper[2558]: W0727 20:00:32.196645    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9baf5281_a1d8_433e_aec3_a36d644cea81.slice/crio-1257568fcda1e12668b984f4c9bac85df3841275bbb3be33869217c5c7140f3a WatchSource:0}: Error finding container 1257568fcda1e12668b984f4c9bac85df3841275bbb3be33869217c5c7140f3a: Status 404 returned error can't find the container with id 1257568fcda1e12668b984f4c9bac85df3841275bbb3be33869217c5c7140f3a
Jul 27 20:00:32.229039 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:32.229014    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-73760660-8ddd65d58-h9p57"]
Jul 27 20:00:32.230281 ip-10-0-129-235 kubenswrapper[2558]: W0727 20:00:32.230256    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca113aca_92ad_4082_a364_02390fb0a233.slice/crio-73a4e00c9b64b919a6d08fb32150d645ea2b46c733a8d1db536812723d3ebd08 WatchSource:0}: Error finding container 73a4e00c9b64b919a6d08fb32150d645ea2b46c733a8d1db536812723d3ebd08: Status 404 returned error can't find the container with id 73a4e00c9b64b919a6d08fb32150d645ea2b46c733a8d1db536812723d3ebd08
Jul 27 20:00:33.132503 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:33.132470    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-73760660-67c49cc76-blhp6" event={"ID":"9baf5281-a1d8-433e-aec3-a36d644cea81","Type":"ContainerStarted","Data":"9789ab2342e086c01ff4af65ff96bfdffe35ab928c817ca35785d839cac76738"}
Jul 27 20:00:33.132990 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:33.132530    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-73760660-67c49cc76-blhp6" event={"ID":"9baf5281-a1d8-433e-aec3-a36d644cea81","Type":"ContainerStarted","Data":"1257568fcda1e12668b984f4c9bac85df3841275bbb3be33869217c5c7140f3a"}
Jul 27 20:00:33.132990 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:33.132689    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-73760660-67c49cc76-blhp6"
Jul 27 20:00:33.134070 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:33.134045    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-73760660-8ddd65d58-h9p57" event={"ID":"ca113aca-92ad-4082-a364-02390fb0a233","Type":"ContainerStarted","Data":"9fd8030ede68c5e064198058f2a3885173d334e28a4503acdec4fa6e7e80ee16"}
Jul 27 20:00:33.134175 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:33.134085    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-73760660-8ddd65d58-h9p57" event={"ID":"ca113aca-92ad-4082-a364-02390fb0a233","Type":"ContainerStarted","Data":"73a4e00c9b64b919a6d08fb32150d645ea2b46c733a8d1db536812723d3ebd08"}
Jul 27 20:00:33.134175 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:33.134112    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-73760660-8ddd65d58-h9p57"
Jul 27 20:00:33.149334 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:33.149290    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-73760660-67c49cc76-blhp6" podStartSLOduration=2.149272739 podStartE2EDuration="2.149272739s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:00:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:00:33.146735308 +0000 UTC m=+2046.018927281" watchObservedRunningTime="2026-07-27 20:00:33.149272739 +0000 UTC m=+2046.021464715"
Jul 27 20:00:33.162306 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:33.162257    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-73760660-8ddd65d58-h9p57" podStartSLOduration=2.162240802 podStartE2EDuration="2.162240802s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:00:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:00:33.161909637 +0000 UTC m=+2046.034101618" watchObservedRunningTime="2026-07-27 20:00:33.162240802 +0000 UTC m=+2046.034432776"
Jul 27 20:00:35.426145 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:35.426106    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:00:39.328074 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:39.328042    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-73760660/e2e-lifecycle-model-73760660-kserve-d457fbc6c-tdbgg"]
Jul 27 20:00:39.331988 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:39.331967    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-73760660/e2e-lifecycle-model-73760660-kserve-d457fbc6c-tdbgg"
Jul 27 20:00:39.335238 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:39.335219    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-73760660\"/\"openshift-service-ca.crt\""
Jul 27 20:00:39.335350 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:39.335235    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-73760660\"/\"e2e-lifecycle-model-73760660-kserve-self-signed-certs\""
Jul 27 20:00:39.335350 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:39.335244    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-73760660\"/\"default-dockercfg-cq8cs\""
Jul 27 20:00:39.335350 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:39.335236    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-73760660\"/\"kube-root-ca.crt\""
Jul 27 20:00:39.342271 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:39.342247    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-73760660/e2e-lifecycle-model-73760660-kserve-d457fbc6c-tdbgg"]
Jul 27 20:00:39.466068 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:39.466029    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:00:39.479170 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:39.479141    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3158745-09b9-4840-9b53-e048749424f6-dshm\") pod \"e2e-lifecycle-model-73760660-kserve-d457fbc6c-tdbgg\" (UID: \"d3158745-09b9-4840-9b53-e048749424f6\") " pod="ai-tenant-e2e-mt-73760660/e2e-lifecycle-model-73760660-kserve-d457fbc6c-tdbgg"
Jul 27 20:00:39.479321 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:39.479204    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3158745-09b9-4840-9b53-e048749424f6-model-cache\") pod \"e2e-lifecycle-model-73760660-kserve-d457fbc6c-tdbgg\" (UID: \"d3158745-09b9-4840-9b53-e048749424f6\") " pod="ai-tenant-e2e-mt-73760660/e2e-lifecycle-model-73760660-kserve-d457fbc6c-tdbgg"
Jul 27 20:00:39.479321 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:39.479234    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3158745-09b9-4840-9b53-e048749424f6-tls-certs\") pod \"e2e-lifecycle-model-73760660-kserve-d457fbc6c-tdbgg\" (UID: \"d3158745-09b9-4840-9b53-e048749424f6\") " pod="ai-tenant-e2e-mt-73760660/e2e-lifecycle-model-73760660-kserve-d457fbc6c-tdbgg"
Jul 27 20:00:39.479321 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:39.479301    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3158745-09b9-4840-9b53-e048749424f6-tmp-dir\") pod \"e2e-lifecycle-model-73760660-kserve-d457fbc6c-tdbgg\" (UID: \"d3158745-09b9-4840-9b53-e048749424f6\") " pod="ai-tenant-e2e-mt-73760660/e2e-lifecycle-model-73760660-kserve-d457fbc6c-tdbgg"
Jul 27 20:00:39.479497 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:39.479352    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxmtv\" (UniqueName: \"kubernetes.io/projected/d3158745-09b9-4840-9b53-e048749424f6-kube-api-access-qxmtv\") pod \"e2e-lifecycle-model-73760660-kserve-d457fbc6c-tdbgg\" (UID: \"d3158745-09b9-4840-9b53-e048749424f6\") " pod="ai-tenant-e2e-mt-73760660/e2e-lifecycle-model-73760660-kserve-d457fbc6c-tdbgg"
Jul 27 20:00:39.479497 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:39.479387    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3158745-09b9-4840-9b53-e048749424f6-home\") pod \"e2e-lifecycle-model-73760660-kserve-d457fbc6c-tdbgg\" (UID: \"d3158745-09b9-4840-9b53-e048749424f6\") " pod="ai-tenant-e2e-mt-73760660/e2e-lifecycle-model-73760660-kserve-d457fbc6c-tdbgg"
Jul 27 20:00:39.580197 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:39.580112    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3158745-09b9-4840-9b53-e048749424f6-home\") pod \"e2e-lifecycle-model-73760660-kserve-d457fbc6c-tdbgg\" (UID: \"d3158745-09b9-4840-9b53-e048749424f6\") " pod="ai-tenant-e2e-mt-73760660/e2e-lifecycle-model-73760660-kserve-d457fbc6c-tdbgg"
Jul 27 20:00:39.580197 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:39.580174    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3158745-09b9-4840-9b53-e048749424f6-dshm\") pod \"e2e-lifecycle-model-73760660-kserve-d457fbc6c-tdbgg\" (UID: \"d3158745-09b9-4840-9b53-e048749424f6\") " pod="ai-tenant-e2e-mt-73760660/e2e-lifecycle-model-73760660-kserve-d457fbc6c-tdbgg"
Jul 27 20:00:39.580469 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:39.580202    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3158745-09b9-4840-9b53-e048749424f6-model-cache\") pod \"e2e-lifecycle-model-73760660-kserve-d457fbc6c-tdbgg\" (UID: \"d3158745-09b9-4840-9b53-e048749424f6\") " pod="ai-tenant-e2e-mt-73760660/e2e-lifecycle-model-73760660-kserve-d457fbc6c-tdbgg"
Jul 27 20:00:39.580469 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:39.580219    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3158745-09b9-4840-9b53-e048749424f6-tls-certs\") pod \"e2e-lifecycle-model-73760660-kserve-d457fbc6c-tdbgg\" (UID: \"d3158745-09b9-4840-9b53-e048749424f6\") " pod="ai-tenant-e2e-mt-73760660/e2e-lifecycle-model-73760660-kserve-d457fbc6c-tdbgg"
Jul 27 20:00:39.580469 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:39.580246    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3158745-09b9-4840-9b53-e048749424f6-tmp-dir\") pod \"e2e-lifecycle-model-73760660-kserve-d457fbc6c-tdbgg\" (UID: \"d3158745-09b9-4840-9b53-e048749424f6\") " pod="ai-tenant-e2e-mt-73760660/e2e-lifecycle-model-73760660-kserve-d457fbc6c-tdbgg"
Jul 27 20:00:39.580469 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:39.580297    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxmtv\" (UniqueName: \"kubernetes.io/projected/d3158745-09b9-4840-9b53-e048749424f6-kube-api-access-qxmtv\") pod \"e2e-lifecycle-model-73760660-kserve-d457fbc6c-tdbgg\" (UID: \"d3158745-09b9-4840-9b53-e048749424f6\") " pod="ai-tenant-e2e-mt-73760660/e2e-lifecycle-model-73760660-kserve-d457fbc6c-tdbgg"
Jul 27 20:00:39.580673 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:39.580599    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3158745-09b9-4840-9b53-e048749424f6-model-cache\") pod \"e2e-lifecycle-model-73760660-kserve-d457fbc6c-tdbgg\" (UID: \"d3158745-09b9-4840-9b53-e048749424f6\") " pod="ai-tenant-e2e-mt-73760660/e2e-lifecycle-model-73760660-kserve-d457fbc6c-tdbgg"
Jul 27 20:00:39.580673 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:39.580607    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3158745-09b9-4840-9b53-e048749424f6-tmp-dir\") pod \"e2e-lifecycle-model-73760660-kserve-d457fbc6c-tdbgg\" (UID: \"d3158745-09b9-4840-9b53-e048749424f6\") " pod="ai-tenant-e2e-mt-73760660/e2e-lifecycle-model-73760660-kserve-d457fbc6c-tdbgg"
Jul 27 20:00:39.580792 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:39.580776    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3158745-09b9-4840-9b53-e048749424f6-home\") pod \"e2e-lifecycle-model-73760660-kserve-d457fbc6c-tdbgg\" (UID: \"d3158745-09b9-4840-9b53-e048749424f6\") " pod="ai-tenant-e2e-mt-73760660/e2e-lifecycle-model-73760660-kserve-d457fbc6c-tdbgg"
Jul 27 20:00:39.582730 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:39.582709    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3158745-09b9-4840-9b53-e048749424f6-dshm\") pod \"e2e-lifecycle-model-73760660-kserve-d457fbc6c-tdbgg\" (UID: \"d3158745-09b9-4840-9b53-e048749424f6\") " pod="ai-tenant-e2e-mt-73760660/e2e-lifecycle-model-73760660-kserve-d457fbc6c-tdbgg"
Jul 27 20:00:39.583013 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:39.582995    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3158745-09b9-4840-9b53-e048749424f6-tls-certs\") pod \"e2e-lifecycle-model-73760660-kserve-d457fbc6c-tdbgg\" (UID: \"d3158745-09b9-4840-9b53-e048749424f6\") " pod="ai-tenant-e2e-mt-73760660/e2e-lifecycle-model-73760660-kserve-d457fbc6c-tdbgg"
Jul 27 20:00:39.588098 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:39.588074    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxmtv\" (UniqueName: \"kubernetes.io/projected/d3158745-09b9-4840-9b53-e048749424f6-kube-api-access-qxmtv\") pod \"e2e-lifecycle-model-73760660-kserve-d457fbc6c-tdbgg\" (UID: \"d3158745-09b9-4840-9b53-e048749424f6\") " pod="ai-tenant-e2e-mt-73760660/e2e-lifecycle-model-73760660-kserve-d457fbc6c-tdbgg"
Jul 27 20:00:39.643264 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:39.643235    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-73760660/e2e-lifecycle-model-73760660-kserve-d457fbc6c-tdbgg"
Jul 27 20:00:39.779042 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:39.778996    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-73760660/e2e-lifecycle-model-73760660-kserve-d457fbc6c-tdbgg"]
Jul 27 20:00:39.782223 ip-10-0-129-235 kubenswrapper[2558]: W0727 20:00:39.782200    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd3158745_09b9_4840_9b53_e048749424f6.slice/crio-2f9543eb61cf5b6a7fc58484802da35ff87de260e475505ccab06c62e137a490 WatchSource:0}: Error finding container 2f9543eb61cf5b6a7fc58484802da35ff87de260e475505ccab06c62e137a490: Status 404 returned error can't find the container with id 2f9543eb61cf5b6a7fc58484802da35ff87de260e475505ccab06c62e137a490
Jul 27 20:00:40.174475 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:40.174428    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-73760660/e2e-lifecycle-model-73760660-kserve-d457fbc6c-tdbgg" event={"ID":"d3158745-09b9-4840-9b53-e048749424f6","Type":"ContainerStarted","Data":"98b42b1167fc718a0663226dc040df3ed89ad863185f74930475cf7fb3dfeae7"}
Jul 27 20:00:40.174475 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:40.174476    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-73760660/e2e-lifecycle-model-73760660-kserve-d457fbc6c-tdbgg" event={"ID":"d3158745-09b9-4840-9b53-e048749424f6","Type":"ContainerStarted","Data":"2f9543eb61cf5b6a7fc58484802da35ff87de260e475505ccab06c62e137a490"}
Jul 27 20:00:40.192264 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:40.192215    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-73760660/e2e-lifecycle-model-73760660-kserve-d457fbc6c-tdbgg" podStartSLOduration=1.192201185 podStartE2EDuration="1.192201185s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:00:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:00:40.189742956 +0000 UTC m=+2053.061934931" watchObservedRunningTime="2026-07-27 20:00:40.192201185 +0000 UTC m=+2053.064393159"
Jul 27 20:00:44.143624 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:44.143592    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-73760660-8ddd65d58-h9p57"
Jul 27 20:00:44.144101 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:44.143666    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-73760660-67c49cc76-blhp6"
Jul 27 20:00:49.643836 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:49.643725    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-73760660/e2e-lifecycle-model-73760660-kserve-d457fbc6c-tdbgg"
Jul 27 20:00:49.643836 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:49.643800    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-73760660/e2e-lifecycle-model-73760660-kserve-d457fbc6c-tdbgg"
Jul 27 20:00:49.656429 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:49.656381    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-73760660/e2e-lifecycle-model-73760660-kserve-d457fbc6c-tdbgg"
Jul 27 20:00:50.242318 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:00:50.242285    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-73760660/e2e-lifecycle-model-73760660-kserve-d457fbc6c-tdbgg"
Jul 27 20:01:20.082066 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:20.082022    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-73760660-8ddd65d58-h9p57"]
Jul 27 20:01:20.082709 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:20.082286    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-73760660-8ddd65d58-h9p57" podUID="ca113aca-92ad-4082-a364-02390fb0a233" containerName="payload-processing" containerID="cri-o://9fd8030ede68c5e064198058f2a3885173d334e28a4503acdec4fa6e7e80ee16" gracePeriod=30
Jul 27 20:01:20.085355 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:20.085327    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-73760660-67c49cc76-blhp6"]
Jul 27 20:01:20.085650 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:20.085603    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-73760660-67c49cc76-blhp6" podUID="9baf5281-a1d8-433e-aec3-a36d644cea81" containerName="payload-pre-processing" containerID="cri-o://9789ab2342e086c01ff4af65ff96bfdffe35ab928c817ca35785d839cac76738" gracePeriod=30
Jul 27 20:01:20.154122 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:20.154067    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:01:20.251404 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:20.251334    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:01:20.374454 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:20.374403    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-73760660-8ddd65d58-h9p57"
Jul 27 20:01:20.377547 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:20.377523    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-73760660-67c49cc76-blhp6"
Jul 27 20:01:20.402689 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:20.402656    2558 generic.go:356] "Generic (PLEG): container finished" podID="ca113aca-92ad-4082-a364-02390fb0a233" containerID="9fd8030ede68c5e064198058f2a3885173d334e28a4503acdec4fa6e7e80ee16" exitCode=0
Jul 27 20:01:20.402891 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:20.402699    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-73760660-8ddd65d58-h9p57"
Jul 27 20:01:20.402891 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:20.402734    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-73760660-8ddd65d58-h9p57" event={"ID":"ca113aca-92ad-4082-a364-02390fb0a233","Type":"ContainerDied","Data":"9fd8030ede68c5e064198058f2a3885173d334e28a4503acdec4fa6e7e80ee16"}
Jul 27 20:01:20.402891 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:20.402793    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-73760660-8ddd65d58-h9p57" event={"ID":"ca113aca-92ad-4082-a364-02390fb0a233","Type":"ContainerDied","Data":"73a4e00c9b64b919a6d08fb32150d645ea2b46c733a8d1db536812723d3ebd08"}
Jul 27 20:01:20.402891 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:20.402823    2558 scope.go:117] "RemoveContainer" containerID="9fd8030ede68c5e064198058f2a3885173d334e28a4503acdec4fa6e7e80ee16"
Jul 27 20:01:20.404212 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:20.404182    2558 generic.go:356] "Generic (PLEG): container finished" podID="9baf5281-a1d8-433e-aec3-a36d644cea81" containerID="9789ab2342e086c01ff4af65ff96bfdffe35ab928c817ca35785d839cac76738" exitCode=0
Jul 27 20:01:20.404355 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:20.404224    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-73760660-67c49cc76-blhp6"
Jul 27 20:01:20.404355 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:20.404222    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-73760660-67c49cc76-blhp6" event={"ID":"9baf5281-a1d8-433e-aec3-a36d644cea81","Type":"ContainerDied","Data":"9789ab2342e086c01ff4af65ff96bfdffe35ab928c817ca35785d839cac76738"}
Jul 27 20:01:20.404355 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:20.404276    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-73760660-67c49cc76-blhp6" event={"ID":"9baf5281-a1d8-433e-aec3-a36d644cea81","Type":"ContainerDied","Data":"1257568fcda1e12668b984f4c9bac85df3841275bbb3be33869217c5c7140f3a"}
Jul 27 20:01:20.415218 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:20.415192    2558 scope.go:117] "RemoveContainer" containerID="9fd8030ede68c5e064198058f2a3885173d334e28a4503acdec4fa6e7e80ee16"
Jul 27 20:01:20.415612 ip-10-0-129-235 kubenswrapper[2558]: E0727 20:01:20.415587    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9fd8030ede68c5e064198058f2a3885173d334e28a4503acdec4fa6e7e80ee16\": container with ID starting with 9fd8030ede68c5e064198058f2a3885173d334e28a4503acdec4fa6e7e80ee16 not found: ID does not exist" containerID="9fd8030ede68c5e064198058f2a3885173d334e28a4503acdec4fa6e7e80ee16"
Jul 27 20:01:20.415732 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:20.415621    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9fd8030ede68c5e064198058f2a3885173d334e28a4503acdec4fa6e7e80ee16"} err="failed to get container status \"9fd8030ede68c5e064198058f2a3885173d334e28a4503acdec4fa6e7e80ee16\": rpc error: code = NotFound desc = could not find container \"9fd8030ede68c5e064198058f2a3885173d334e28a4503acdec4fa6e7e80ee16\": container with ID starting with 9fd8030ede68c5e064198058f2a3885173d334e28a4503acdec4fa6e7e80ee16 not found: ID does not exist"
Jul 27 20:01:20.415732 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:20.415641    2558 scope.go:117] "RemoveContainer" containerID="9789ab2342e086c01ff4af65ff96bfdffe35ab928c817ca35785d839cac76738"
Jul 27 20:01:20.426652 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:20.426623    2558 scope.go:117] "RemoveContainer" containerID="9789ab2342e086c01ff4af65ff96bfdffe35ab928c817ca35785d839cac76738"
Jul 27 20:01:20.427070 ip-10-0-129-235 kubenswrapper[2558]: E0727 20:01:20.427039    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9789ab2342e086c01ff4af65ff96bfdffe35ab928c817ca35785d839cac76738\": container with ID starting with 9789ab2342e086c01ff4af65ff96bfdffe35ab928c817ca35785d839cac76738 not found: ID does not exist" containerID="9789ab2342e086c01ff4af65ff96bfdffe35ab928c817ca35785d839cac76738"
Jul 27 20:01:20.427152 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:20.427082    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9789ab2342e086c01ff4af65ff96bfdffe35ab928c817ca35785d839cac76738"} err="failed to get container status \"9789ab2342e086c01ff4af65ff96bfdffe35ab928c817ca35785d839cac76738\": rpc error: code = NotFound desc = could not find container \"9789ab2342e086c01ff4af65ff96bfdffe35ab928c817ca35785d839cac76738\": container with ID starting with 9789ab2342e086c01ff4af65ff96bfdffe35ab928c817ca35785d839cac76738 not found: ID does not exist"
Jul 27 20:01:20.452476 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:20.452398    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qcttx\" (UniqueName: \"kubernetes.io/projected/ca113aca-92ad-4082-a364-02390fb0a233-kube-api-access-qcttx\") pod \"ca113aca-92ad-4082-a364-02390fb0a233\" (UID: \"ca113aca-92ad-4082-a364-02390fb0a233\") "
Jul 27 20:01:20.452694 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:20.452528    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mj2lw\" (UniqueName: \"kubernetes.io/projected/9baf5281-a1d8-433e-aec3-a36d644cea81-kube-api-access-mj2lw\") pod \"9baf5281-a1d8-433e-aec3-a36d644cea81\" (UID: \"9baf5281-a1d8-433e-aec3-a36d644cea81\") "
Jul 27 20:01:20.452694 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:20.452576    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9baf5281-a1d8-433e-aec3-a36d644cea81-plugins-config-volume\") pod \"9baf5281-a1d8-433e-aec3-a36d644cea81\" (UID: \"9baf5281-a1d8-433e-aec3-a36d644cea81\") "
Jul 27 20:01:20.452694 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:20.452640    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ca113aca-92ad-4082-a364-02390fb0a233-plugins-config-volume\") pod \"ca113aca-92ad-4082-a364-02390fb0a233\" (UID: \"ca113aca-92ad-4082-a364-02390fb0a233\") "
Jul 27 20:01:20.453018 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:20.452988    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9baf5281-a1d8-433e-aec3-a36d644cea81-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9baf5281-a1d8-433e-aec3-a36d644cea81" (UID: "9baf5281-a1d8-433e-aec3-a36d644cea81"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 20:01:20.453124 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:20.453048    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca113aca-92ad-4082-a364-02390fb0a233-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ca113aca-92ad-4082-a364-02390fb0a233" (UID: "ca113aca-92ad-4082-a364-02390fb0a233"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 20:01:20.454825 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:20.454801    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca113aca-92ad-4082-a364-02390fb0a233-kube-api-access-qcttx" (OuterVolumeSpecName: "kube-api-access-qcttx") pod "ca113aca-92ad-4082-a364-02390fb0a233" (UID: "ca113aca-92ad-4082-a364-02390fb0a233"). InnerVolumeSpecName "kube-api-access-qcttx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:01:20.454825 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:20.454809    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9baf5281-a1d8-433e-aec3-a36d644cea81-kube-api-access-mj2lw" (OuterVolumeSpecName: "kube-api-access-mj2lw") pod "9baf5281-a1d8-433e-aec3-a36d644cea81" (UID: "9baf5281-a1d8-433e-aec3-a36d644cea81"). InnerVolumeSpecName "kube-api-access-mj2lw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:01:20.553508 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:20.553466    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ca113aca-92ad-4082-a364-02390fb0a233-plugins-config-volume\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:01:20.553508 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:20.553499    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qcttx\" (UniqueName: \"kubernetes.io/projected/ca113aca-92ad-4082-a364-02390fb0a233-kube-api-access-qcttx\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:01:20.553508 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:20.553509    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mj2lw\" (UniqueName: \"kubernetes.io/projected/9baf5281-a1d8-433e-aec3-a36d644cea81-kube-api-access-mj2lw\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:01:20.553740 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:20.553520    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9baf5281-a1d8-433e-aec3-a36d644cea81-plugins-config-volume\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:01:20.729707 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:20.729669    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-73760660-8ddd65d58-h9p57"]
Jul 27 20:01:20.733174 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:20.733135    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-73760660-8ddd65d58-h9p57"]
Jul 27 20:01:20.743554 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:20.743507    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-73760660-67c49cc76-blhp6"]
Jul 27 20:01:20.745744 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:20.745716    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-73760660-67c49cc76-blhp6"]
Jul 27 20:01:21.675493 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:21.675456    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9baf5281-a1d8-433e-aec3-a36d644cea81" path="/var/lib/kubelet/pods/9baf5281-a1d8-433e-aec3-a36d644cea81/volumes"
Jul 27 20:01:21.675893 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:21.675837    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca113aca-92ad-4082-a364-02390fb0a233" path="/var/lib/kubelet/pods/ca113aca-92ad-4082-a364-02390fb0a233/volumes"
Jul 27 20:01:23.332052 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:23.332018    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:01:27.724603 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:27.724574    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5dfc2_103bab23-bca3-472a-974b-7731f83c5bfb/ovn-acl-logging/0.log"
Jul 27 20:01:27.733535 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:27.733509    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5dfc2_103bab23-bca3-472a-974b-7731f83c5bfb/ovn-acl-logging/0.log"
Jul 27 20:01:28.279513 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:28.279475    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-73760660/e2e-lifecycle-model-73760660-kserve-d457fbc6c-tdbgg"]
Jul 27 20:01:28.785728 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:28.785693    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:01:43.301501 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:43.301405    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-73760660/e2e-lifecycle-model-73760660-kserve-d457fbc6c-tdbgg" podUID="d3158745-09b9-4840-9b53-e048749424f6" containerName="main" containerID="cri-o://98b42b1167fc718a0663226dc040df3ed89ad863185f74930475cf7fb3dfeae7" gracePeriod=45
Jul 27 20:01:43.542935 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:43.542905    2558 generic.go:356] "Generic (PLEG): container finished" podID="d3158745-09b9-4840-9b53-e048749424f6" containerID="98b42b1167fc718a0663226dc040df3ed89ad863185f74930475cf7fb3dfeae7" exitCode=0
Jul 27 20:01:43.543074 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:43.542974    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-73760660/e2e-lifecycle-model-73760660-kserve-d457fbc6c-tdbgg" event={"ID":"d3158745-09b9-4840-9b53-e048749424f6","Type":"ContainerDied","Data":"98b42b1167fc718a0663226dc040df3ed89ad863185f74930475cf7fb3dfeae7"}
Jul 27 20:01:43.543074 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:43.543013    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-73760660/e2e-lifecycle-model-73760660-kserve-d457fbc6c-tdbgg" event={"ID":"d3158745-09b9-4840-9b53-e048749424f6","Type":"ContainerDied","Data":"2f9543eb61cf5b6a7fc58484802da35ff87de260e475505ccab06c62e137a490"}
Jul 27 20:01:43.543074 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:43.543024    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2f9543eb61cf5b6a7fc58484802da35ff87de260e475505ccab06c62e137a490"
Jul 27 20:01:43.554037 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:43.553986    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-73760660/e2e-lifecycle-model-73760660-kserve-d457fbc6c-tdbgg"
Jul 27 20:01:43.575336 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:43.575308    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3158745-09b9-4840-9b53-e048749424f6-tls-certs\") pod \"d3158745-09b9-4840-9b53-e048749424f6\" (UID: \"d3158745-09b9-4840-9b53-e048749424f6\") "
Jul 27 20:01:43.575511 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:43.575376    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qxmtv\" (UniqueName: \"kubernetes.io/projected/d3158745-09b9-4840-9b53-e048749424f6-kube-api-access-qxmtv\") pod \"d3158745-09b9-4840-9b53-e048749424f6\" (UID: \"d3158745-09b9-4840-9b53-e048749424f6\") "
Jul 27 20:01:43.575511 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:43.575434    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3158745-09b9-4840-9b53-e048749424f6-model-cache\") pod \"d3158745-09b9-4840-9b53-e048749424f6\" (UID: \"d3158745-09b9-4840-9b53-e048749424f6\") "
Jul 27 20:01:43.575511 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:43.575472    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3158745-09b9-4840-9b53-e048749424f6-dshm\") pod \"d3158745-09b9-4840-9b53-e048749424f6\" (UID: \"d3158745-09b9-4840-9b53-e048749424f6\") "
Jul 27 20:01:43.575511 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:43.575503    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3158745-09b9-4840-9b53-e048749424f6-home\") pod \"d3158745-09b9-4840-9b53-e048749424f6\" (UID: \"d3158745-09b9-4840-9b53-e048749424f6\") "
Jul 27 20:01:43.575846 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:43.575555    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3158745-09b9-4840-9b53-e048749424f6-tmp-dir\") pod \"d3158745-09b9-4840-9b53-e048749424f6\" (UID: \"d3158745-09b9-4840-9b53-e048749424f6\") "
Jul 27 20:01:43.576499 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:43.576444    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3158745-09b9-4840-9b53-e048749424f6-model-cache" (OuterVolumeSpecName: "model-cache") pod "d3158745-09b9-4840-9b53-e048749424f6" (UID: "d3158745-09b9-4840-9b53-e048749424f6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:01:43.576800 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:43.576757    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3158745-09b9-4840-9b53-e048749424f6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d3158745-09b9-4840-9b53-e048749424f6" (UID: "d3158745-09b9-4840-9b53-e048749424f6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:01:43.576930 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:43.576891    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3158745-09b9-4840-9b53-e048749424f6-home" (OuterVolumeSpecName: "home") pod "d3158745-09b9-4840-9b53-e048749424f6" (UID: "d3158745-09b9-4840-9b53-e048749424f6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:01:43.579792 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:43.579708    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3158745-09b9-4840-9b53-e048749424f6-kube-api-access-qxmtv" (OuterVolumeSpecName: "kube-api-access-qxmtv") pod "d3158745-09b9-4840-9b53-e048749424f6" (UID: "d3158745-09b9-4840-9b53-e048749424f6"). InnerVolumeSpecName "kube-api-access-qxmtv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:01:43.579792 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:43.579728    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3158745-09b9-4840-9b53-e048749424f6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d3158745-09b9-4840-9b53-e048749424f6" (UID: "d3158745-09b9-4840-9b53-e048749424f6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 20:01:43.580005 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:43.579980    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3158745-09b9-4840-9b53-e048749424f6-dshm" (OuterVolumeSpecName: "dshm") pod "d3158745-09b9-4840-9b53-e048749424f6" (UID: "d3158745-09b9-4840-9b53-e048749424f6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:01:43.676477 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:43.676437    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3158745-09b9-4840-9b53-e048749424f6-tls-certs\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:01:43.676477 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:43.676469    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qxmtv\" (UniqueName: \"kubernetes.io/projected/d3158745-09b9-4840-9b53-e048749424f6-kube-api-access-qxmtv\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:01:43.676477 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:43.676481    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3158745-09b9-4840-9b53-e048749424f6-model-cache\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:01:43.676477 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:43.676489    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3158745-09b9-4840-9b53-e048749424f6-dshm\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:01:43.676852 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:43.676497    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3158745-09b9-4840-9b53-e048749424f6-home\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:01:43.676852 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:43.676504    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3158745-09b9-4840-9b53-e048749424f6-tmp-dir\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:01:44.548559 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:44.548529    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-73760660/e2e-lifecycle-model-73760660-kserve-d457fbc6c-tdbgg"
Jul 27 20:01:44.564322 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:44.564287    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-73760660/e2e-lifecycle-model-73760660-kserve-d457fbc6c-tdbgg"]
Jul 27 20:01:44.568042 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:44.568004    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-73760660/e2e-lifecycle-model-73760660-kserve-d457fbc6c-tdbgg"]
Jul 27 20:01:45.674357 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:01:45.674320    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3158745-09b9-4840-9b53-e048749424f6" path="/var/lib/kubelet/pods/d3158745-09b9-4840-9b53-e048749424f6/volumes"
Jul 27 20:02:04.675170 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:02:04.675133    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:02:06.355515 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:02:06.355476    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:02:08.218444 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:02:08.218386    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:02:08.285158 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:02:08.285125    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:02:10.510230 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:02:10.510187    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:02:11.220152 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:02:11.220117    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:02:26.051741 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:02:26.051661    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:02:26.119274 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:02:26.119237    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:02:26.306541 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:02:26.306463    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:02:26.546490 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:02:26.546452    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:02:26.941833 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:02:26.941799    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:02:27.614681 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:02:27.614639    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:02:27.712866 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:02:27.712831    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:02:28.722742 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:02:28.722699    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:02:28.820855 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:02:28.820823    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:02:30.145551 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:02:30.145509    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:02:36.157398 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:02:36.157362    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:02:36.259803 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:02:36.259768    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:02:36.827667 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:02:36.827628    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:02:43.141750 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:02:43.141717    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:02:43.248746 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:02:43.248712    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:02:43.341862 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:02:43.341828    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:02:53.844056 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:02:53.844008    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:02:54.168116 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:02:54.168079    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:03:14.455644 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:14.455607    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:03:15.076564 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:15.076523    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:03:21.884860 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:21.884826    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:03:21.985221 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:21.985178    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:03:22.255329 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:22.255229    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:03:22.529370 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:22.529279    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:03:23.693905 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:23.693871    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:03:25.774158 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:25.774123    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:03:25.864655 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:25.864623    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:03:25.950208 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:25.950166    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:03:26.395536 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:26.395502    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:03:26.494620 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:26.494581    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:03:26.496060 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:26.496035    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-79961d-65d6cb99cb-l9k84"]
Jul 27 20:03:26.496442 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:26.496427    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca113aca-92ad-4082-a364-02390fb0a233" containerName="payload-processing"
Jul 27 20:03:26.496491 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:26.496447    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca113aca-92ad-4082-a364-02390fb0a233" containerName="payload-processing"
Jul 27 20:03:26.496491 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:26.496459    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9baf5281-a1d8-433e-aec3-a36d644cea81" containerName="payload-pre-processing"
Jul 27 20:03:26.496491 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:26.496465    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="9baf5281-a1d8-433e-aec3-a36d644cea81" containerName="payload-pre-processing"
Jul 27 20:03:26.496491 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:26.496488    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3158745-09b9-4840-9b53-e048749424f6" containerName="main"
Jul 27 20:03:26.496629 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:26.496493    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3158745-09b9-4840-9b53-e048749424f6" containerName="main"
Jul 27 20:03:26.496629 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:26.496569    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca113aca-92ad-4082-a364-02390fb0a233" containerName="payload-processing"
Jul 27 20:03:26.496629 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:26.496577    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3158745-09b9-4840-9b53-e048749424f6" containerName="main"
Jul 27 20:03:26.496629 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:26.496584    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="9baf5281-a1d8-433e-aec3-a36d644cea81" containerName="payload-pre-processing"
Jul 27 20:03:26.499775 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:26.499760    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-79961d-65d6cb99cb-l9k84"
Jul 27 20:03:26.502344 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:26.502320    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-79961d\""
Jul 27 20:03:26.502485 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:26.502365    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-79961d-dockercfg-mwlv4\""
Jul 27 20:03:26.509342 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:26.509321    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-79961d-65d6cb99cb-l9k84"]
Jul 27 20:03:26.523146 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:26.523115    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-79961d-79f89db66-88g6l"]
Jul 27 20:03:26.527293 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:26.527265    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-79961d-79f89db66-88g6l"
Jul 27 20:03:26.535462 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:26.535359    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-79961d-79f89db66-88g6l"]
Jul 27 20:03:26.580633 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:26.580603    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0561285a-4234-4a70-8b79-ccf03d825bbe-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-79961d-65d6cb99cb-l9k84\" (UID: \"0561285a-4234-4a70-8b79-ccf03d825bbe\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-79961d-65d6cb99cb-l9k84"
Jul 27 20:03:26.580800 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:26.580638    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmv2s\" (UniqueName: \"kubernetes.io/projected/a72f7964-07ae-46a1-924d-6cff28a37d23-kube-api-access-vmv2s\") pod \"payload-processing-e2e-inf-a-79961d-79f89db66-88g6l\" (UID: \"a72f7964-07ae-46a1-924d-6cff28a37d23\") " pod="openshift-ingress/payload-processing-e2e-inf-a-79961d-79f89db66-88g6l"
Jul 27 20:03:26.580800 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:26.580664    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wc5cw\" (UniqueName: \"kubernetes.io/projected/0561285a-4234-4a70-8b79-ccf03d825bbe-kube-api-access-wc5cw\") pod \"payload-pre-processing-e2e-inf-a-79961d-65d6cb99cb-l9k84\" (UID: \"0561285a-4234-4a70-8b79-ccf03d825bbe\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-79961d-65d6cb99cb-l9k84"
Jul 27 20:03:26.580800 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:26.580681    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a72f7964-07ae-46a1-924d-6cff28a37d23-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-79961d-79f89db66-88g6l\" (UID: \"a72f7964-07ae-46a1-924d-6cff28a37d23\") " pod="openshift-ingress/payload-processing-e2e-inf-a-79961d-79f89db66-88g6l"
Jul 27 20:03:26.596213 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:26.596176    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:03:26.681593 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:26.681503    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmv2s\" (UniqueName: \"kubernetes.io/projected/a72f7964-07ae-46a1-924d-6cff28a37d23-kube-api-access-vmv2s\") pod \"payload-processing-e2e-inf-a-79961d-79f89db66-88g6l\" (UID: \"a72f7964-07ae-46a1-924d-6cff28a37d23\") " pod="openshift-ingress/payload-processing-e2e-inf-a-79961d-79f89db66-88g6l"
Jul 27 20:03:26.681593 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:26.681542    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wc5cw\" (UniqueName: \"kubernetes.io/projected/0561285a-4234-4a70-8b79-ccf03d825bbe-kube-api-access-wc5cw\") pod \"payload-pre-processing-e2e-inf-a-79961d-65d6cb99cb-l9k84\" (UID: \"0561285a-4234-4a70-8b79-ccf03d825bbe\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-79961d-65d6cb99cb-l9k84"
Jul 27 20:03:26.681593 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:26.681565    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a72f7964-07ae-46a1-924d-6cff28a37d23-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-79961d-79f89db66-88g6l\" (UID: \"a72f7964-07ae-46a1-924d-6cff28a37d23\") " pod="openshift-ingress/payload-processing-e2e-inf-a-79961d-79f89db66-88g6l"
Jul 27 20:03:26.681896 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:26.681643    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0561285a-4234-4a70-8b79-ccf03d825bbe-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-79961d-65d6cb99cb-l9k84\" (UID: \"0561285a-4234-4a70-8b79-ccf03d825bbe\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-79961d-65d6cb99cb-l9k84"
Jul 27 20:03:26.682342 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:26.682323    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a72f7964-07ae-46a1-924d-6cff28a37d23-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-79961d-79f89db66-88g6l\" (UID: \"a72f7964-07ae-46a1-924d-6cff28a37d23\") " pod="openshift-ingress/payload-processing-e2e-inf-a-79961d-79f89db66-88g6l"
Jul 27 20:03:26.682380 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:26.682335    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0561285a-4234-4a70-8b79-ccf03d825bbe-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-79961d-65d6cb99cb-l9k84\" (UID: \"0561285a-4234-4a70-8b79-ccf03d825bbe\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-79961d-65d6cb99cb-l9k84"
Jul 27 20:03:26.689496 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:26.689466    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmv2s\" (UniqueName: \"kubernetes.io/projected/a72f7964-07ae-46a1-924d-6cff28a37d23-kube-api-access-vmv2s\") pod \"payload-processing-e2e-inf-a-79961d-79f89db66-88g6l\" (UID: \"a72f7964-07ae-46a1-924d-6cff28a37d23\") " pod="openshift-ingress/payload-processing-e2e-inf-a-79961d-79f89db66-88g6l"
Jul 27 20:03:26.689626 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:26.689497    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wc5cw\" (UniqueName: \"kubernetes.io/projected/0561285a-4234-4a70-8b79-ccf03d825bbe-kube-api-access-wc5cw\") pod \"payload-pre-processing-e2e-inf-a-79961d-65d6cb99cb-l9k84\" (UID: \"0561285a-4234-4a70-8b79-ccf03d825bbe\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-79961d-65d6cb99cb-l9k84"
Jul 27 20:03:26.811781 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:26.811750    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-79961d-65d6cb99cb-l9k84"
Jul 27 20:03:26.839074 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:26.839044    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-79961d-79f89db66-88g6l"
Jul 27 20:03:26.960675 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:26.960643    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-79961d-65d6cb99cb-l9k84"]
Jul 27 20:03:26.961467 ip-10-0-129-235 kubenswrapper[2558]: W0727 20:03:26.961439    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0561285a_4234_4a70_8b79_ccf03d825bbe.slice/crio-c5591f8bd869f11d1d39acec540a74f6cc18a6fef5b7dacaaa679932cc9e5a86 WatchSource:0}: Error finding container c5591f8bd869f11d1d39acec540a74f6cc18a6fef5b7dacaaa679932cc9e5a86: Status 404 returned error can't find the container with id c5591f8bd869f11d1d39acec540a74f6cc18a6fef5b7dacaaa679932cc9e5a86
Jul 27 20:03:26.988343 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:26.988321    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-79961d-79f89db66-88g6l"]
Jul 27 20:03:26.995504 ip-10-0-129-235 kubenswrapper[2558]: W0727 20:03:26.995478    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda72f7964_07ae_46a1_924d_6cff28a37d23.slice/crio-62f51c9d5c5075c585518b85372f5d58589a8a15196820984f4c459d000823e7 WatchSource:0}: Error finding container 62f51c9d5c5075c585518b85372f5d58589a8a15196820984f4c459d000823e7: Status 404 returned error can't find the container with id 62f51c9d5c5075c585518b85372f5d58589a8a15196820984f4c459d000823e7
Jul 27 20:03:27.091929 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:27.091889    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-79961d-65d6cb99cb-l9k84" event={"ID":"0561285a-4234-4a70-8b79-ccf03d825bbe","Type":"ContainerStarted","Data":"b60a387536609858f89e7893cec7bab4a9b793fcd1de1d4cc77766cd27d87227"}
Jul 27 20:03:27.092126 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:27.091958    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-79961d-65d6cb99cb-l9k84" event={"ID":"0561285a-4234-4a70-8b79-ccf03d825bbe","Type":"ContainerStarted","Data":"c5591f8bd869f11d1d39acec540a74f6cc18a6fef5b7dacaaa679932cc9e5a86"}
Jul 27 20:03:27.092126 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:27.091993    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-79961d-65d6cb99cb-l9k84"
Jul 27 20:03:27.093359 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:27.093335    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-79961d-79f89db66-88g6l" event={"ID":"a72f7964-07ae-46a1-924d-6cff28a37d23","Type":"ContainerStarted","Data":"82679c5592d01b95827fa1fbc82693f61ca970749793615c399e7f0221d6ee7f"}
Jul 27 20:03:27.093490 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:27.093374    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-79961d-79f89db66-88g6l" event={"ID":"a72f7964-07ae-46a1-924d-6cff28a37d23","Type":"ContainerStarted","Data":"62f51c9d5c5075c585518b85372f5d58589a8a15196820984f4c459d000823e7"}
Jul 27 20:03:27.093490 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:27.093403    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-79961d-79f89db66-88g6l"
Jul 27 20:03:27.109688 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:27.109635    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-79961d-65d6cb99cb-l9k84" podStartSLOduration=1.109621502 podStartE2EDuration="1.109621502s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:03:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:03:27.108870135 +0000 UTC m=+2219.981062109" watchObservedRunningTime="2026-07-27 20:03:27.109621502 +0000 UTC m=+2219.981813476"
Jul 27 20:03:27.126995 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:27.126949    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-79961d-79f89db66-88g6l" podStartSLOduration=1.126933373 podStartE2EDuration="1.126933373s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:03:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:03:27.124432511 +0000 UTC m=+2219.996624487" watchObservedRunningTime="2026-07-27 20:03:27.126933373 +0000 UTC m=+2219.999125348"
Jul 27 20:03:30.267435 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:30.267384    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:03:30.578723 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:30.578629    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:03:30.873582 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:30.873546    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:03:31.042090 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:31.042050    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:03:34.255769 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:34.255733    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:03:34.374668 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:34.374630    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:03:34.472852 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:34.472826    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:03:34.580184 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:34.580077    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:03:35.062572 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:35.062522    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:03:35.173340 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:35.173305    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-08e715-5c98c45688-4tst8"]
Jul 27 20:03:35.176904 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:35.176883    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-08e715-5c98c45688-4tst8"
Jul 27 20:03:35.179382 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:35.179356    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-08e715\""
Jul 27 20:03:35.179539 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:35.179450    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-08e715-dockercfg-4mzrm\""
Jul 27 20:03:35.187773 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:35.187749    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-08e715-5c98c45688-4tst8"]
Jul 27 20:03:35.259546 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:35.259517    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/83c8bb74-e6e3-463c-9039-978c7afa64d1-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-08e715-5c98c45688-4tst8\" (UID: \"83c8bb74-e6e3-463c-9039-978c7afa64d1\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-08e715-5c98c45688-4tst8"
Jul 27 20:03:35.259933 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:35.259570    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rh2p\" (UniqueName: \"kubernetes.io/projected/83c8bb74-e6e3-463c-9039-978c7afa64d1-kube-api-access-4rh2p\") pod \"payload-pre-processing-e2e-inf-b-08e715-5c98c45688-4tst8\" (UID: \"83c8bb74-e6e3-463c-9039-978c7afa64d1\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-08e715-5c98c45688-4tst8"
Jul 27 20:03:35.293827 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:35.293790    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-08e715-85bd995d5b-j8fc8"]
Jul 27 20:03:35.297228 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:35.297213    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-08e715-85bd995d5b-j8fc8"
Jul 27 20:03:35.306290 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:35.306265    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-08e715-85bd995d5b-j8fc8"]
Jul 27 20:03:35.360573 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:35.360486    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sj86t\" (UniqueName: \"kubernetes.io/projected/b6215976-a00f-40dd-8f51-2115bdec63b6-kube-api-access-sj86t\") pod \"payload-processing-e2e-inf-b-08e715-85bd995d5b-j8fc8\" (UID: \"b6215976-a00f-40dd-8f51-2115bdec63b6\") " pod="openshift-ingress/payload-processing-e2e-inf-b-08e715-85bd995d5b-j8fc8"
Jul 27 20:03:35.360573 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:35.360530    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6215976-a00f-40dd-8f51-2115bdec63b6-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-08e715-85bd995d5b-j8fc8\" (UID: \"b6215976-a00f-40dd-8f51-2115bdec63b6\") " pod="openshift-ingress/payload-processing-e2e-inf-b-08e715-85bd995d5b-j8fc8"
Jul 27 20:03:35.360573 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:35.360564    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/83c8bb74-e6e3-463c-9039-978c7afa64d1-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-08e715-5c98c45688-4tst8\" (UID: \"83c8bb74-e6e3-463c-9039-978c7afa64d1\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-08e715-5c98c45688-4tst8"
Jul 27 20:03:35.360785 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:35.360600    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rh2p\" (UniqueName: \"kubernetes.io/projected/83c8bb74-e6e3-463c-9039-978c7afa64d1-kube-api-access-4rh2p\") pod \"payload-pre-processing-e2e-inf-b-08e715-5c98c45688-4tst8\" (UID: \"83c8bb74-e6e3-463c-9039-978c7afa64d1\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-08e715-5c98c45688-4tst8"
Jul 27 20:03:35.361215 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:35.361196    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/83c8bb74-e6e3-463c-9039-978c7afa64d1-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-08e715-5c98c45688-4tst8\" (UID: \"83c8bb74-e6e3-463c-9039-978c7afa64d1\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-08e715-5c98c45688-4tst8"
Jul 27 20:03:35.369224 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:35.369204    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rh2p\" (UniqueName: \"kubernetes.io/projected/83c8bb74-e6e3-463c-9039-978c7afa64d1-kube-api-access-4rh2p\") pod \"payload-pre-processing-e2e-inf-b-08e715-5c98c45688-4tst8\" (UID: \"83c8bb74-e6e3-463c-9039-978c7afa64d1\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-08e715-5c98c45688-4tst8"
Jul 27 20:03:35.399248 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:35.399214    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:03:35.461767 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:35.461729    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6215976-a00f-40dd-8f51-2115bdec63b6-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-08e715-85bd995d5b-j8fc8\" (UID: \"b6215976-a00f-40dd-8f51-2115bdec63b6\") " pod="openshift-ingress/payload-processing-e2e-inf-b-08e715-85bd995d5b-j8fc8"
Jul 27 20:03:35.461937 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:35.461841    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sj86t\" (UniqueName: \"kubernetes.io/projected/b6215976-a00f-40dd-8f51-2115bdec63b6-kube-api-access-sj86t\") pod \"payload-processing-e2e-inf-b-08e715-85bd995d5b-j8fc8\" (UID: \"b6215976-a00f-40dd-8f51-2115bdec63b6\") " pod="openshift-ingress/payload-processing-e2e-inf-b-08e715-85bd995d5b-j8fc8"
Jul 27 20:03:35.462448 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:35.462400    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6215976-a00f-40dd-8f51-2115bdec63b6-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-08e715-85bd995d5b-j8fc8\" (UID: \"b6215976-a00f-40dd-8f51-2115bdec63b6\") " pod="openshift-ingress/payload-processing-e2e-inf-b-08e715-85bd995d5b-j8fc8"
Jul 27 20:03:35.474150 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:35.474118    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sj86t\" (UniqueName: \"kubernetes.io/projected/b6215976-a00f-40dd-8f51-2115bdec63b6-kube-api-access-sj86t\") pod \"payload-processing-e2e-inf-b-08e715-85bd995d5b-j8fc8\" (UID: \"b6215976-a00f-40dd-8f51-2115bdec63b6\") " pod="openshift-ingress/payload-processing-e2e-inf-b-08e715-85bd995d5b-j8fc8"
Jul 27 20:03:35.488842 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:35.488819    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-08e715-5c98c45688-4tst8"
Jul 27 20:03:35.608777 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:35.608744    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-08e715-85bd995d5b-j8fc8"
Jul 27 20:03:35.628302 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:35.628266    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-08e715-5c98c45688-4tst8"]
Jul 27 20:03:35.629841 ip-10-0-129-235 kubenswrapper[2558]: W0727 20:03:35.629806    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83c8bb74_e6e3_463c_9039_978c7afa64d1.slice/crio-582626a6bd7bec9d013c4d8996f29d13c5b4a41e8d2f9b75aed31ee49241cfd9 WatchSource:0}: Error finding container 582626a6bd7bec9d013c4d8996f29d13c5b4a41e8d2f9b75aed31ee49241cfd9: Status 404 returned error can't find the container with id 582626a6bd7bec9d013c4d8996f29d13c5b4a41e8d2f9b75aed31ee49241cfd9
Jul 27 20:03:35.757215 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:35.757183    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-08e715-85bd995d5b-j8fc8"]
Jul 27 20:03:35.758710 ip-10-0-129-235 kubenswrapper[2558]: W0727 20:03:35.758682    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6215976_a00f_40dd_8f51_2115bdec63b6.slice/crio-790e65c14c5676c16c58c2f0747bc8d8150fd051d22abd385ac295d7cd3e5c3f WatchSource:0}: Error finding container 790e65c14c5676c16c58c2f0747bc8d8150fd051d22abd385ac295d7cd3e5c3f: Status 404 returned error can't find the container with id 790e65c14c5676c16c58c2f0747bc8d8150fd051d22abd385ac295d7cd3e5c3f
Jul 27 20:03:36.148130 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:36.148095    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-08e715-85bd995d5b-j8fc8" event={"ID":"b6215976-a00f-40dd-8f51-2115bdec63b6","Type":"ContainerStarted","Data":"0dcdffd6afd5e0a743250a3e62ea0c490ad790d9da1f2db0bd04c9b23f89f362"}
Jul 27 20:03:36.148301 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:36.148154    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-08e715-85bd995d5b-j8fc8" event={"ID":"b6215976-a00f-40dd-8f51-2115bdec63b6","Type":"ContainerStarted","Data":"790e65c14c5676c16c58c2f0747bc8d8150fd051d22abd385ac295d7cd3e5c3f"}
Jul 27 20:03:36.148385 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:36.148310    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-08e715-85bd995d5b-j8fc8"
Jul 27 20:03:36.149714 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:36.149688    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-08e715-5c98c45688-4tst8" event={"ID":"83c8bb74-e6e3-463c-9039-978c7afa64d1","Type":"ContainerStarted","Data":"47de7824f8988f35f1aa0a09c48fceb0c952a6a3fd9d3c36fd764d138532fee3"}
Jul 27 20:03:36.149842 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:36.149736    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-08e715-5c98c45688-4tst8"
Jul 27 20:03:36.149842 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:36.149754    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-08e715-5c98c45688-4tst8" event={"ID":"83c8bb74-e6e3-463c-9039-978c7afa64d1","Type":"ContainerStarted","Data":"582626a6bd7bec9d013c4d8996f29d13c5b4a41e8d2f9b75aed31ee49241cfd9"}
Jul 27 20:03:36.165833 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:36.165792    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-08e715-85bd995d5b-j8fc8" podStartSLOduration=1.165773687 podStartE2EDuration="1.165773687s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:03:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:03:36.163952287 +0000 UTC m=+2229.036144262" watchObservedRunningTime="2026-07-27 20:03:36.165773687 +0000 UTC m=+2229.037965661"
Jul 27 20:03:36.179963 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:36.179911    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-08e715-5c98c45688-4tst8" podStartSLOduration=1.179899093 podStartE2EDuration="1.179899093s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:03:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:03:36.179748713 +0000 UTC m=+2229.051940689" watchObservedRunningTime="2026-07-27 20:03:36.179899093 +0000 UTC m=+2229.052091066"
Jul 27 20:03:38.101546 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:38.101511    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-79961d-79f89db66-88g6l"
Jul 27 20:03:38.101947 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:38.101588    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-79961d-65d6cb99cb-l9k84"
Jul 27 20:03:38.750434 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:38.750376    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:03:47.157096 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:47.157062    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-08e715-85bd995d5b-j8fc8"
Jul 27 20:03:47.157553 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:47.157132    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-08e715-5c98c45688-4tst8"
Jul 27 20:03:53.012712 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:53.012610    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:03:59.528184 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:59.528149    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-79961d/test-model-79961d-kserve-8cc7d675c-8vnmt"]
Jul 27 20:03:59.532899 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:59.532878    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-79961d/test-model-79961d-kserve-8cc7d675c-8vnmt"
Jul 27 20:03:59.535308 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:59.535284    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-79961d\"/\"kube-root-ca.crt\""
Jul 27 20:03:59.535475 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:59.535284    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-79961d\"/\"default-dockercfg-m5hz9\""
Jul 27 20:03:59.535475 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:59.535287    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-79961d\"/\"openshift-service-ca.crt\""
Jul 27 20:03:59.535602 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:59.535466    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-79961d\"/\"test-model-79961d-kserve-self-signed-certs\""
Jul 27 20:03:59.542056 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:59.542033    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-79961d/test-model-79961d-kserve-8cc7d675c-8vnmt"]
Jul 27 20:03:59.680567 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:59.680531    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41aa7071-55f1-4531-9a13-9590e3d930a9-tls-certs\") pod \"test-model-79961d-kserve-8cc7d675c-8vnmt\" (UID: \"41aa7071-55f1-4531-9a13-9590e3d930a9\") " pod="ai-tenant-e2e-inf-a-79961d/test-model-79961d-kserve-8cc7d675c-8vnmt"
Jul 27 20:03:59.680567 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:59.680567    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41aa7071-55f1-4531-9a13-9590e3d930a9-tmp-dir\") pod \"test-model-79961d-kserve-8cc7d675c-8vnmt\" (UID: \"41aa7071-55f1-4531-9a13-9590e3d930a9\") " pod="ai-tenant-e2e-inf-a-79961d/test-model-79961d-kserve-8cc7d675c-8vnmt"
Jul 27 20:03:59.680884 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:59.680586    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41aa7071-55f1-4531-9a13-9590e3d930a9-home\") pod \"test-model-79961d-kserve-8cc7d675c-8vnmt\" (UID: \"41aa7071-55f1-4531-9a13-9590e3d930a9\") " pod="ai-tenant-e2e-inf-a-79961d/test-model-79961d-kserve-8cc7d675c-8vnmt"
Jul 27 20:03:59.680884 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:59.680692    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8vhc\" (UniqueName: \"kubernetes.io/projected/41aa7071-55f1-4531-9a13-9590e3d930a9-kube-api-access-w8vhc\") pod \"test-model-79961d-kserve-8cc7d675c-8vnmt\" (UID: \"41aa7071-55f1-4531-9a13-9590e3d930a9\") " pod="ai-tenant-e2e-inf-a-79961d/test-model-79961d-kserve-8cc7d675c-8vnmt"
Jul 27 20:03:59.680884 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:59.680760    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41aa7071-55f1-4531-9a13-9590e3d930a9-dshm\") pod \"test-model-79961d-kserve-8cc7d675c-8vnmt\" (UID: \"41aa7071-55f1-4531-9a13-9590e3d930a9\") " pod="ai-tenant-e2e-inf-a-79961d/test-model-79961d-kserve-8cc7d675c-8vnmt"
Jul 27 20:03:59.680884 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:59.680786    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41aa7071-55f1-4531-9a13-9590e3d930a9-model-cache\") pod \"test-model-79961d-kserve-8cc7d675c-8vnmt\" (UID: \"41aa7071-55f1-4531-9a13-9590e3d930a9\") " pod="ai-tenant-e2e-inf-a-79961d/test-model-79961d-kserve-8cc7d675c-8vnmt"
Jul 27 20:03:59.702969 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:59.702940    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:03:59.781573 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:59.781479    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41aa7071-55f1-4531-9a13-9590e3d930a9-dshm\") pod \"test-model-79961d-kserve-8cc7d675c-8vnmt\" (UID: \"41aa7071-55f1-4531-9a13-9590e3d930a9\") " pod="ai-tenant-e2e-inf-a-79961d/test-model-79961d-kserve-8cc7d675c-8vnmt"
Jul 27 20:03:59.781573 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:59.781516    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41aa7071-55f1-4531-9a13-9590e3d930a9-model-cache\") pod \"test-model-79961d-kserve-8cc7d675c-8vnmt\" (UID: \"41aa7071-55f1-4531-9a13-9590e3d930a9\") " pod="ai-tenant-e2e-inf-a-79961d/test-model-79961d-kserve-8cc7d675c-8vnmt"
Jul 27 20:03:59.781784 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:59.781602    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41aa7071-55f1-4531-9a13-9590e3d930a9-tls-certs\") pod \"test-model-79961d-kserve-8cc7d675c-8vnmt\" (UID: \"41aa7071-55f1-4531-9a13-9590e3d930a9\") " pod="ai-tenant-e2e-inf-a-79961d/test-model-79961d-kserve-8cc7d675c-8vnmt"
Jul 27 20:03:59.781784 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:59.781626    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41aa7071-55f1-4531-9a13-9590e3d930a9-tmp-dir\") pod \"test-model-79961d-kserve-8cc7d675c-8vnmt\" (UID: \"41aa7071-55f1-4531-9a13-9590e3d930a9\") " pod="ai-tenant-e2e-inf-a-79961d/test-model-79961d-kserve-8cc7d675c-8vnmt"
Jul 27 20:03:59.781784 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:59.781656    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41aa7071-55f1-4531-9a13-9590e3d930a9-home\") pod \"test-model-79961d-kserve-8cc7d675c-8vnmt\" (UID: \"41aa7071-55f1-4531-9a13-9590e3d930a9\") " pod="ai-tenant-e2e-inf-a-79961d/test-model-79961d-kserve-8cc7d675c-8vnmt"
Jul 27 20:03:59.781784 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:59.781688    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8vhc\" (UniqueName: \"kubernetes.io/projected/41aa7071-55f1-4531-9a13-9590e3d930a9-kube-api-access-w8vhc\") pod \"test-model-79961d-kserve-8cc7d675c-8vnmt\" (UID: \"41aa7071-55f1-4531-9a13-9590e3d930a9\") " pod="ai-tenant-e2e-inf-a-79961d/test-model-79961d-kserve-8cc7d675c-8vnmt"
Jul 27 20:03:59.782019 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:59.781996    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41aa7071-55f1-4531-9a13-9590e3d930a9-model-cache\") pod \"test-model-79961d-kserve-8cc7d675c-8vnmt\" (UID: \"41aa7071-55f1-4531-9a13-9590e3d930a9\") " pod="ai-tenant-e2e-inf-a-79961d/test-model-79961d-kserve-8cc7d675c-8vnmt"
Jul 27 20:03:59.782096 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:59.782009    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41aa7071-55f1-4531-9a13-9590e3d930a9-tmp-dir\") pod \"test-model-79961d-kserve-8cc7d675c-8vnmt\" (UID: \"41aa7071-55f1-4531-9a13-9590e3d930a9\") " pod="ai-tenant-e2e-inf-a-79961d/test-model-79961d-kserve-8cc7d675c-8vnmt"
Jul 27 20:03:59.782096 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:59.782071    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41aa7071-55f1-4531-9a13-9590e3d930a9-home\") pod \"test-model-79961d-kserve-8cc7d675c-8vnmt\" (UID: \"41aa7071-55f1-4531-9a13-9590e3d930a9\") " pod="ai-tenant-e2e-inf-a-79961d/test-model-79961d-kserve-8cc7d675c-8vnmt"
Jul 27 20:03:59.783979 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:59.783955    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41aa7071-55f1-4531-9a13-9590e3d930a9-dshm\") pod \"test-model-79961d-kserve-8cc7d675c-8vnmt\" (UID: \"41aa7071-55f1-4531-9a13-9590e3d930a9\") " pod="ai-tenant-e2e-inf-a-79961d/test-model-79961d-kserve-8cc7d675c-8vnmt"
Jul 27 20:03:59.784141 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:59.784125    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41aa7071-55f1-4531-9a13-9590e3d930a9-tls-certs\") pod \"test-model-79961d-kserve-8cc7d675c-8vnmt\" (UID: \"41aa7071-55f1-4531-9a13-9590e3d930a9\") " pod="ai-tenant-e2e-inf-a-79961d/test-model-79961d-kserve-8cc7d675c-8vnmt"
Jul 27 20:03:59.789593 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:59.789574    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8vhc\" (UniqueName: \"kubernetes.io/projected/41aa7071-55f1-4531-9a13-9590e3d930a9-kube-api-access-w8vhc\") pod \"test-model-79961d-kserve-8cc7d675c-8vnmt\" (UID: \"41aa7071-55f1-4531-9a13-9590e3d930a9\") " pod="ai-tenant-e2e-inf-a-79961d/test-model-79961d-kserve-8cc7d675c-8vnmt"
Jul 27 20:03:59.844762 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:59.844733    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-79961d/test-model-79961d-kserve-8cc7d675c-8vnmt"
Jul 27 20:03:59.984672 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:03:59.984642    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-79961d/test-model-79961d-kserve-8cc7d675c-8vnmt"]
Jul 27 20:03:59.986270 ip-10-0-129-235 kubenswrapper[2558]: W0727 20:03:59.986238    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod41aa7071_55f1_4531_9a13_9590e3d930a9.slice/crio-fa0eb6fd3b5f86241b1d0eedbf9100ec8c0a72a538960b2a5e8d2563e93a81a6 WatchSource:0}: Error finding container fa0eb6fd3b5f86241b1d0eedbf9100ec8c0a72a538960b2a5e8d2563e93a81a6: Status 404 returned error can't find the container with id fa0eb6fd3b5f86241b1d0eedbf9100ec8c0a72a538960b2a5e8d2563e93a81a6
Jul 27 20:04:00.286761 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:04:00.286677    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-79961d/test-model-79961d-kserve-8cc7d675c-8vnmt" event={"ID":"41aa7071-55f1-4531-9a13-9590e3d930a9","Type":"ContainerStarted","Data":"b74863ab805d17153545fdeffe54dee202dfa49075c7a351e2318a1f68f300fe"}
Jul 27 20:04:00.286761 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:04:00.286722    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-79961d/test-model-79961d-kserve-8cc7d675c-8vnmt" event={"ID":"41aa7071-55f1-4531-9a13-9590e3d930a9","Type":"ContainerStarted","Data":"fa0eb6fd3b5f86241b1d0eedbf9100ec8c0a72a538960b2a5e8d2563e93a81a6"}
Jul 27 20:04:00.305387 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:04:00.305336    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-79961d/test-model-79961d-kserve-8cc7d675c-8vnmt" podStartSLOduration=1.305321193 podStartE2EDuration="1.305321193s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:03:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:04:00.303028911 +0000 UTC m=+2253.175220888" watchObservedRunningTime="2026-07-27 20:04:00.305321193 +0000 UTC m=+2253.177513166"
Jul 27 20:04:09.845482 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:04:09.845447    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-79961d/test-model-79961d-kserve-8cc7d675c-8vnmt"
Jul 27 20:04:09.845901 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:04:09.845772    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-79961d/test-model-79961d-kserve-8cc7d675c-8vnmt"
Jul 27 20:04:09.858153 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:04:09.858129    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-79961d/test-model-79961d-kserve-8cc7d675c-8vnmt"
Jul 27 20:04:10.361280 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:04:10.361253    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-79961d/test-model-79961d-kserve-8cc7d675c-8vnmt"
Jul 27 20:04:24.516000 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:04:24.515959    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-08e715/test-model-08e715-kserve-84f9d5d5d9-c6hjp"]
Jul 27 20:04:24.520233 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:04:24.520208    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-08e715/test-model-08e715-kserve-84f9d5d5d9-c6hjp"
Jul 27 20:04:24.523591 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:04:24.523566    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-08e715\"/\"default-dockercfg-xchrl\""
Jul 27 20:04:24.523707 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:04:24.523617    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-08e715\"/\"openshift-service-ca.crt\""
Jul 27 20:04:24.523809 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:04:24.523788    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-08e715\"/\"kube-root-ca.crt\""
Jul 27 20:04:24.524033 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:04:24.524018    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-08e715\"/\"test-model-08e715-kserve-self-signed-certs\""
Jul 27 20:04:24.531118 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:04:24.531093    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-08e715/test-model-08e715-kserve-84f9d5d5d9-c6hjp"]
Jul 27 20:04:24.554972 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:04:24.554943    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:04:24.602121 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:04:24.602094    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3afe4f66-1cf4-4ec5-9c93-e27412ec6083-tmp-dir\") pod \"test-model-08e715-kserve-84f9d5d5d9-c6hjp\" (UID: \"3afe4f66-1cf4-4ec5-9c93-e27412ec6083\") " pod="ai-tenant-e2e-inf-b-08e715/test-model-08e715-kserve-84f9d5d5d9-c6hjp"
Jul 27 20:04:24.602258 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:04:24.602124    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3afe4f66-1cf4-4ec5-9c93-e27412ec6083-dshm\") pod \"test-model-08e715-kserve-84f9d5d5d9-c6hjp\" (UID: \"3afe4f66-1cf4-4ec5-9c93-e27412ec6083\") " pod="ai-tenant-e2e-inf-b-08e715/test-model-08e715-kserve-84f9d5d5d9-c6hjp"
Jul 27 20:04:24.602258 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:04:24.602158    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3afe4f66-1cf4-4ec5-9c93-e27412ec6083-model-cache\") pod \"test-model-08e715-kserve-84f9d5d5d9-c6hjp\" (UID: \"3afe4f66-1cf4-4ec5-9c93-e27412ec6083\") " pod="ai-tenant-e2e-inf-b-08e715/test-model-08e715-kserve-84f9d5d5d9-c6hjp"
Jul 27 20:04:24.602258 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:04:24.602235    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3afe4f66-1cf4-4ec5-9c93-e27412ec6083-home\") pod \"test-model-08e715-kserve-84f9d5d5d9-c6hjp\" (UID: \"3afe4f66-1cf4-4ec5-9c93-e27412ec6083\") " pod="ai-tenant-e2e-inf-b-08e715/test-model-08e715-kserve-84f9d5d5d9-c6hjp"
Jul 27 20:04:24.602369 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:04:24.602258    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v87xx\" (UniqueName: \"kubernetes.io/projected/3afe4f66-1cf4-4ec5-9c93-e27412ec6083-kube-api-access-v87xx\") pod \"test-model-08e715-kserve-84f9d5d5d9-c6hjp\" (UID: \"3afe4f66-1cf4-4ec5-9c93-e27412ec6083\") " pod="ai-tenant-e2e-inf-b-08e715/test-model-08e715-kserve-84f9d5d5d9-c6hjp"
Jul 27 20:04:24.602369 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:04:24.602295    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3afe4f66-1cf4-4ec5-9c93-e27412ec6083-tls-certs\") pod \"test-model-08e715-kserve-84f9d5d5d9-c6hjp\" (UID: \"3afe4f66-1cf4-4ec5-9c93-e27412ec6083\") " pod="ai-tenant-e2e-inf-b-08e715/test-model-08e715-kserve-84f9d5d5d9-c6hjp"
Jul 27 20:04:24.703122 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:04:24.703092    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3afe4f66-1cf4-4ec5-9c93-e27412ec6083-tmp-dir\") pod \"test-model-08e715-kserve-84f9d5d5d9-c6hjp\" (UID: \"3afe4f66-1cf4-4ec5-9c93-e27412ec6083\") " pod="ai-tenant-e2e-inf-b-08e715/test-model-08e715-kserve-84f9d5d5d9-c6hjp"
Jul 27 20:04:24.703257 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:04:24.703127    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3afe4f66-1cf4-4ec5-9c93-e27412ec6083-dshm\") pod \"test-model-08e715-kserve-84f9d5d5d9-c6hjp\" (UID: \"3afe4f66-1cf4-4ec5-9c93-e27412ec6083\") " pod="ai-tenant-e2e-inf-b-08e715/test-model-08e715-kserve-84f9d5d5d9-c6hjp"
Jul 27 20:04:24.703257 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:04:24.703172    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3afe4f66-1cf4-4ec5-9c93-e27412ec6083-model-cache\") pod \"test-model-08e715-kserve-84f9d5d5d9-c6hjp\" (UID: \"3afe4f66-1cf4-4ec5-9c93-e27412ec6083\") " pod="ai-tenant-e2e-inf-b-08e715/test-model-08e715-kserve-84f9d5d5d9-c6hjp"
Jul 27 20:04:24.703257 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:04:24.703221    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3afe4f66-1cf4-4ec5-9c93-e27412ec6083-home\") pod \"test-model-08e715-kserve-84f9d5d5d9-c6hjp\" (UID: \"3afe4f66-1cf4-4ec5-9c93-e27412ec6083\") " pod="ai-tenant-e2e-inf-b-08e715/test-model-08e715-kserve-84f9d5d5d9-c6hjp"
Jul 27 20:04:24.703257 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:04:24.703246    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v87xx\" (UniqueName: \"kubernetes.io/projected/3afe4f66-1cf4-4ec5-9c93-e27412ec6083-kube-api-access-v87xx\") pod \"test-model-08e715-kserve-84f9d5d5d9-c6hjp\" (UID: \"3afe4f66-1cf4-4ec5-9c93-e27412ec6083\") " pod="ai-tenant-e2e-inf-b-08e715/test-model-08e715-kserve-84f9d5d5d9-c6hjp"
Jul 27 20:04:24.703481 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:04:24.703286    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3afe4f66-1cf4-4ec5-9c93-e27412ec6083-tls-certs\") pod \"test-model-08e715-kserve-84f9d5d5d9-c6hjp\" (UID: \"3afe4f66-1cf4-4ec5-9c93-e27412ec6083\") " pod="ai-tenant-e2e-inf-b-08e715/test-model-08e715-kserve-84f9d5d5d9-c6hjp"
Jul 27 20:04:24.703622 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:04:24.703595    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3afe4f66-1cf4-4ec5-9c93-e27412ec6083-model-cache\") pod \"test-model-08e715-kserve-84f9d5d5d9-c6hjp\" (UID: \"3afe4f66-1cf4-4ec5-9c93-e27412ec6083\") " pod="ai-tenant-e2e-inf-b-08e715/test-model-08e715-kserve-84f9d5d5d9-c6hjp"
Jul 27 20:04:24.703749 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:04:24.703654    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3afe4f66-1cf4-4ec5-9c93-e27412ec6083-tmp-dir\") pod \"test-model-08e715-kserve-84f9d5d5d9-c6hjp\" (UID: \"3afe4f66-1cf4-4ec5-9c93-e27412ec6083\") " pod="ai-tenant-e2e-inf-b-08e715/test-model-08e715-kserve-84f9d5d5d9-c6hjp"
Jul 27 20:04:24.703749 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:04:24.703679    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3afe4f66-1cf4-4ec5-9c93-e27412ec6083-home\") pod \"test-model-08e715-kserve-84f9d5d5d9-c6hjp\" (UID: \"3afe4f66-1cf4-4ec5-9c93-e27412ec6083\") " pod="ai-tenant-e2e-inf-b-08e715/test-model-08e715-kserve-84f9d5d5d9-c6hjp"
Jul 27 20:04:24.705619 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:04:24.705596    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3afe4f66-1cf4-4ec5-9c93-e27412ec6083-dshm\") pod \"test-model-08e715-kserve-84f9d5d5d9-c6hjp\" (UID: \"3afe4f66-1cf4-4ec5-9c93-e27412ec6083\") " pod="ai-tenant-e2e-inf-b-08e715/test-model-08e715-kserve-84f9d5d5d9-c6hjp"
Jul 27 20:04:24.705762 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:04:24.705745    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3afe4f66-1cf4-4ec5-9c93-e27412ec6083-tls-certs\") pod \"test-model-08e715-kserve-84f9d5d5d9-c6hjp\" (UID: \"3afe4f66-1cf4-4ec5-9c93-e27412ec6083\") " pod="ai-tenant-e2e-inf-b-08e715/test-model-08e715-kserve-84f9d5d5d9-c6hjp"
Jul 27 20:04:24.711531 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:04:24.711512    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v87xx\" (UniqueName: \"kubernetes.io/projected/3afe4f66-1cf4-4ec5-9c93-e27412ec6083-kube-api-access-v87xx\") pod \"test-model-08e715-kserve-84f9d5d5d9-c6hjp\" (UID: \"3afe4f66-1cf4-4ec5-9c93-e27412ec6083\") " pod="ai-tenant-e2e-inf-b-08e715/test-model-08e715-kserve-84f9d5d5d9-c6hjp"
Jul 27 20:04:24.836489 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:04:24.832748    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-08e715/test-model-08e715-kserve-84f9d5d5d9-c6hjp"
Jul 27 20:04:24.975094 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:04:24.975069    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-08e715/test-model-08e715-kserve-84f9d5d5d9-c6hjp"]
Jul 27 20:04:24.976665 ip-10-0-129-235 kubenswrapper[2558]: W0727 20:04:24.976634    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3afe4f66_1cf4_4ec5_9c93_e27412ec6083.slice/crio-9fb4c085380f8c3c2177609b1a71829434065a9fc9730f1dec35add4af63bd39 WatchSource:0}: Error finding container 9fb4c085380f8c3c2177609b1a71829434065a9fc9730f1dec35add4af63bd39: Status 404 returned error can't find the container with id 9fb4c085380f8c3c2177609b1a71829434065a9fc9730f1dec35add4af63bd39
Jul 27 20:04:25.430778 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:04:25.430741    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-08e715/test-model-08e715-kserve-84f9d5d5d9-c6hjp" event={"ID":"3afe4f66-1cf4-4ec5-9c93-e27412ec6083","Type":"ContainerStarted","Data":"552976168e3c3236792b5652d5611b2ef1c8156d58561ca7bbeee736c4213ed7"}
Jul 27 20:04:25.430988 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:04:25.430787    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-08e715/test-model-08e715-kserve-84f9d5d5d9-c6hjp" event={"ID":"3afe4f66-1cf4-4ec5-9c93-e27412ec6083","Type":"ContainerStarted","Data":"9fb4c085380f8c3c2177609b1a71829434065a9fc9730f1dec35add4af63bd39"}
Jul 27 20:04:25.450465 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:04:25.450390    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-08e715/test-model-08e715-kserve-84f9d5d5d9-c6hjp" podStartSLOduration=1.450373913 podStartE2EDuration="1.450373913s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:04:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:04:25.447083491 +0000 UTC m=+2278.319275464" watchObservedRunningTime="2026-07-27 20:04:25.450373913 +0000 UTC m=+2278.322565887"
Jul 27 20:04:34.836466 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:04:34.836419    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-08e715/test-model-08e715-kserve-84f9d5d5d9-c6hjp"
Jul 27 20:04:34.836466 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:04:34.836474    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-08e715/test-model-08e715-kserve-84f9d5d5d9-c6hjp"
Jul 27 20:04:34.849371 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:04:34.849347    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-08e715/test-model-08e715-kserve-84f9d5d5d9-c6hjp"
Jul 27 20:04:35.499741 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:04:35.499714    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-08e715/test-model-08e715-kserve-84f9d5d5d9-c6hjp"
Jul 27 20:05:03.215062 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:03.215031    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-79961d/test-model-79961d-kserve-8cc7d675c-8vnmt"]
Jul 27 20:05:03.258841 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:03.258800    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:05:04.088627 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:04.088588    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-08e715/test-model-08e715-kserve-84f9d5d5d9-c6hjp"]
Jul 27 20:05:04.108579 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:04.108550    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:05:13.959791 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:13.959749    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:05:14.060617 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:14.060581    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:05:14.672079 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:14.672044    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:05:14.706512 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:14.706481    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-79961d-79f89db66-88g6l"]
Jul 27 20:05:14.706756 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:14.706732    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-79961d-79f89db66-88g6l" podUID="a72f7964-07ae-46a1-924d-6cff28a37d23" containerName="payload-processing" containerID="cri-o://82679c5592d01b95827fa1fbc82693f61ca970749793615c399e7f0221d6ee7f" gracePeriod=30
Jul 27 20:05:14.714460 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:14.714430    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-79961d-65d6cb99cb-l9k84"]
Jul 27 20:05:14.714684 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:14.714657    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-79961d-65d6cb99cb-l9k84" podUID="0561285a-4234-4a70-8b79-ccf03d825bbe" containerName="payload-pre-processing" containerID="cri-o://b60a387536609858f89e7893cec7bab4a9b793fcd1de1d4cc77766cd27d87227" gracePeriod=30
Jul 27 20:05:14.772230 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:14.772199    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:05:14.845673 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:14.845638    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:05:14.910596 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:14.910567    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:05:14.998735 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:14.998708    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-79961d-79f89db66-88g6l"
Jul 27 20:05:15.001785 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:15.001766    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-79961d-65d6cb99cb-l9k84"
Jul 27 20:05:15.176501 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:15.176445    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vmv2s\" (UniqueName: \"kubernetes.io/projected/a72f7964-07ae-46a1-924d-6cff28a37d23-kube-api-access-vmv2s\") pod \"a72f7964-07ae-46a1-924d-6cff28a37d23\" (UID: \"a72f7964-07ae-46a1-924d-6cff28a37d23\") "
Jul 27 20:05:15.176699 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:15.176515    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wc5cw\" (UniqueName: \"kubernetes.io/projected/0561285a-4234-4a70-8b79-ccf03d825bbe-kube-api-access-wc5cw\") pod \"0561285a-4234-4a70-8b79-ccf03d825bbe\" (UID: \"0561285a-4234-4a70-8b79-ccf03d825bbe\") "
Jul 27 20:05:15.176699 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:15.176562    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a72f7964-07ae-46a1-924d-6cff28a37d23-plugins-config-volume\") pod \"a72f7964-07ae-46a1-924d-6cff28a37d23\" (UID: \"a72f7964-07ae-46a1-924d-6cff28a37d23\") "
Jul 27 20:05:15.176699 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:15.176642    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0561285a-4234-4a70-8b79-ccf03d825bbe-plugins-config-volume\") pod \"0561285a-4234-4a70-8b79-ccf03d825bbe\" (UID: \"0561285a-4234-4a70-8b79-ccf03d825bbe\") "
Jul 27 20:05:15.176999 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:15.176974    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a72f7964-07ae-46a1-924d-6cff28a37d23-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a72f7964-07ae-46a1-924d-6cff28a37d23" (UID: "a72f7964-07ae-46a1-924d-6cff28a37d23"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 20:05:15.177095 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:15.177026    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0561285a-4234-4a70-8b79-ccf03d825bbe-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0561285a-4234-4a70-8b79-ccf03d825bbe" (UID: "0561285a-4234-4a70-8b79-ccf03d825bbe"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 20:05:15.178896 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:15.178871    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0561285a-4234-4a70-8b79-ccf03d825bbe-kube-api-access-wc5cw" (OuterVolumeSpecName: "kube-api-access-wc5cw") pod "0561285a-4234-4a70-8b79-ccf03d825bbe" (UID: "0561285a-4234-4a70-8b79-ccf03d825bbe"). InnerVolumeSpecName "kube-api-access-wc5cw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:05:15.178896 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:15.178885    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a72f7964-07ae-46a1-924d-6cff28a37d23-kube-api-access-vmv2s" (OuterVolumeSpecName: "kube-api-access-vmv2s") pod "a72f7964-07ae-46a1-924d-6cff28a37d23" (UID: "a72f7964-07ae-46a1-924d-6cff28a37d23"). InnerVolumeSpecName "kube-api-access-vmv2s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:05:15.226193 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:15.226105    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:05:15.277848 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:15.277817    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a72f7964-07ae-46a1-924d-6cff28a37d23-plugins-config-volume\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:05:15.277848 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:15.277845    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0561285a-4234-4a70-8b79-ccf03d825bbe-plugins-config-volume\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:05:15.277848 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:15.277854    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vmv2s\" (UniqueName: \"kubernetes.io/projected/a72f7964-07ae-46a1-924d-6cff28a37d23-kube-api-access-vmv2s\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:05:15.278112 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:15.277863    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wc5cw\" (UniqueName: \"kubernetes.io/projected/0561285a-4234-4a70-8b79-ccf03d825bbe-kube-api-access-wc5cw\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:05:15.717772 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:15.717741    2558 generic.go:356] "Generic (PLEG): container finished" podID="0561285a-4234-4a70-8b79-ccf03d825bbe" containerID="b60a387536609858f89e7893cec7bab4a9b793fcd1de1d4cc77766cd27d87227" exitCode=0
Jul 27 20:05:15.717949 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:15.717794    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-79961d-65d6cb99cb-l9k84"
Jul 27 20:05:15.717949 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:15.717831    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-79961d-65d6cb99cb-l9k84" event={"ID":"0561285a-4234-4a70-8b79-ccf03d825bbe","Type":"ContainerDied","Data":"b60a387536609858f89e7893cec7bab4a9b793fcd1de1d4cc77766cd27d87227"}
Jul 27 20:05:15.717949 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:15.717881    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-79961d-65d6cb99cb-l9k84" event={"ID":"0561285a-4234-4a70-8b79-ccf03d825bbe","Type":"ContainerDied","Data":"c5591f8bd869f11d1d39acec540a74f6cc18a6fef5b7dacaaa679932cc9e5a86"}
Jul 27 20:05:15.717949 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:15.717909    2558 scope.go:117] "RemoveContainer" containerID="b60a387536609858f89e7893cec7bab4a9b793fcd1de1d4cc77766cd27d87227"
Jul 27 20:05:15.719231 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:15.719210    2558 generic.go:356] "Generic (PLEG): container finished" podID="a72f7964-07ae-46a1-924d-6cff28a37d23" containerID="82679c5592d01b95827fa1fbc82693f61ca970749793615c399e7f0221d6ee7f" exitCode=0
Jul 27 20:05:15.719320 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:15.719268    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-79961d-79f89db66-88g6l"
Jul 27 20:05:15.719320 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:15.719306    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-79961d-79f89db66-88g6l" event={"ID":"a72f7964-07ae-46a1-924d-6cff28a37d23","Type":"ContainerDied","Data":"82679c5592d01b95827fa1fbc82693f61ca970749793615c399e7f0221d6ee7f"}
Jul 27 20:05:15.719493 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:15.719356    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-79961d-79f89db66-88g6l" event={"ID":"a72f7964-07ae-46a1-924d-6cff28a37d23","Type":"ContainerDied","Data":"62f51c9d5c5075c585518b85372f5d58589a8a15196820984f4c459d000823e7"}
Jul 27 20:05:15.732864 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:15.732844    2558 scope.go:117] "RemoveContainer" containerID="b60a387536609858f89e7893cec7bab4a9b793fcd1de1d4cc77766cd27d87227"
Jul 27 20:05:15.733165 ip-10-0-129-235 kubenswrapper[2558]: E0727 20:05:15.733145    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b60a387536609858f89e7893cec7bab4a9b793fcd1de1d4cc77766cd27d87227\": container with ID starting with b60a387536609858f89e7893cec7bab4a9b793fcd1de1d4cc77766cd27d87227 not found: ID does not exist" containerID="b60a387536609858f89e7893cec7bab4a9b793fcd1de1d4cc77766cd27d87227"
Jul 27 20:05:15.733209 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:15.733175    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b60a387536609858f89e7893cec7bab4a9b793fcd1de1d4cc77766cd27d87227"} err="failed to get container status \"b60a387536609858f89e7893cec7bab4a9b793fcd1de1d4cc77766cd27d87227\": rpc error: code = NotFound desc = could not find container \"b60a387536609858f89e7893cec7bab4a9b793fcd1de1d4cc77766cd27d87227\": container with ID starting with b60a387536609858f89e7893cec7bab4a9b793fcd1de1d4cc77766cd27d87227 not found: ID does not exist"
Jul 27 20:05:15.733209 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:15.733192    2558 scope.go:117] "RemoveContainer" containerID="82679c5592d01b95827fa1fbc82693f61ca970749793615c399e7f0221d6ee7f"
Jul 27 20:05:15.743068 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:15.742945    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-79961d-65d6cb99cb-l9k84"]
Jul 27 20:05:15.743068 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:15.743055    2558 scope.go:117] "RemoveContainer" containerID="82679c5592d01b95827fa1fbc82693f61ca970749793615c399e7f0221d6ee7f"
Jul 27 20:05:15.743358 ip-10-0-129-235 kubenswrapper[2558]: E0727 20:05:15.743334    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82679c5592d01b95827fa1fbc82693f61ca970749793615c399e7f0221d6ee7f\": container with ID starting with 82679c5592d01b95827fa1fbc82693f61ca970749793615c399e7f0221d6ee7f not found: ID does not exist" containerID="82679c5592d01b95827fa1fbc82693f61ca970749793615c399e7f0221d6ee7f"
Jul 27 20:05:15.743449 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:15.743367    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82679c5592d01b95827fa1fbc82693f61ca970749793615c399e7f0221d6ee7f"} err="failed to get container status \"82679c5592d01b95827fa1fbc82693f61ca970749793615c399e7f0221d6ee7f\": rpc error: code = NotFound desc = could not find container \"82679c5592d01b95827fa1fbc82693f61ca970749793615c399e7f0221d6ee7f\": container with ID starting with 82679c5592d01b95827fa1fbc82693f61ca970749793615c399e7f0221d6ee7f not found: ID does not exist"
Jul 27 20:05:15.745431 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:15.745392    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-79961d-65d6cb99cb-l9k84"]
Jul 27 20:05:15.752016 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:15.751994    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-79961d-79f89db66-88g6l"]
Jul 27 20:05:15.755487 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:15.755465    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-79961d-79f89db66-88g6l"]
Jul 27 20:05:17.674228 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:17.674152    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0561285a-4234-4a70-8b79-ccf03d825bbe" path="/var/lib/kubelet/pods/0561285a-4234-4a70-8b79-ccf03d825bbe/volumes"
Jul 27 20:05:17.674612 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:17.674565    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a72f7964-07ae-46a1-924d-6cff28a37d23" path="/var/lib/kubelet/pods/a72f7964-07ae-46a1-924d-6cff28a37d23/volumes"
Jul 27 20:05:18.235352 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:18.235295    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-79961d/test-model-79961d-kserve-8cc7d675c-8vnmt" podUID="41aa7071-55f1-4531-9a13-9590e3d930a9" containerName="main" containerID="cri-o://b74863ab805d17153545fdeffe54dee202dfa49075c7a351e2318a1f68f300fe" gracePeriod=45
Jul 27 20:05:18.485943 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:18.485883    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-79961d/test-model-79961d-kserve-8cc7d675c-8vnmt"
Jul 27 20:05:18.608608 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:18.608580    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w8vhc\" (UniqueName: \"kubernetes.io/projected/41aa7071-55f1-4531-9a13-9590e3d930a9-kube-api-access-w8vhc\") pod \"41aa7071-55f1-4531-9a13-9590e3d930a9\" (UID: \"41aa7071-55f1-4531-9a13-9590e3d930a9\") "
Jul 27 20:05:18.608798 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:18.608620    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41aa7071-55f1-4531-9a13-9590e3d930a9-tmp-dir\") pod \"41aa7071-55f1-4531-9a13-9590e3d930a9\" (UID: \"41aa7071-55f1-4531-9a13-9590e3d930a9\") "
Jul 27 20:05:18.608798 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:18.608642    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41aa7071-55f1-4531-9a13-9590e3d930a9-dshm\") pod \"41aa7071-55f1-4531-9a13-9590e3d930a9\" (UID: \"41aa7071-55f1-4531-9a13-9590e3d930a9\") "
Jul 27 20:05:18.608798 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:18.608666    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41aa7071-55f1-4531-9a13-9590e3d930a9-tls-certs\") pod \"41aa7071-55f1-4531-9a13-9590e3d930a9\" (UID: \"41aa7071-55f1-4531-9a13-9590e3d930a9\") "
Jul 27 20:05:18.608798 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:18.608681    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41aa7071-55f1-4531-9a13-9590e3d930a9-model-cache\") pod \"41aa7071-55f1-4531-9a13-9590e3d930a9\" (UID: \"41aa7071-55f1-4531-9a13-9590e3d930a9\") "
Jul 27 20:05:18.608798 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:18.608793    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41aa7071-55f1-4531-9a13-9590e3d930a9-home\") pod \"41aa7071-55f1-4531-9a13-9590e3d930a9\" (UID: \"41aa7071-55f1-4531-9a13-9590e3d930a9\") "
Jul 27 20:05:18.609092 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:18.608947    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41aa7071-55f1-4531-9a13-9590e3d930a9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "41aa7071-55f1-4531-9a13-9590e3d930a9" (UID: "41aa7071-55f1-4531-9a13-9590e3d930a9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:05:18.609092 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:18.608962    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41aa7071-55f1-4531-9a13-9590e3d930a9-model-cache" (OuterVolumeSpecName: "model-cache") pod "41aa7071-55f1-4531-9a13-9590e3d930a9" (UID: "41aa7071-55f1-4531-9a13-9590e3d930a9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:05:18.609092 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:18.609053    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41aa7071-55f1-4531-9a13-9590e3d930a9-home" (OuterVolumeSpecName: "home") pod "41aa7071-55f1-4531-9a13-9590e3d930a9" (UID: "41aa7071-55f1-4531-9a13-9590e3d930a9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:05:18.609092 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:18.609082    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41aa7071-55f1-4531-9a13-9590e3d930a9-tmp-dir\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:05:18.609092 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:18.609094    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41aa7071-55f1-4531-9a13-9590e3d930a9-model-cache\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:05:18.611092 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:18.611068    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41aa7071-55f1-4531-9a13-9590e3d930a9-dshm" (OuterVolumeSpecName: "dshm") pod "41aa7071-55f1-4531-9a13-9590e3d930a9" (UID: "41aa7071-55f1-4531-9a13-9590e3d930a9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:05:18.611092 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:18.611080    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41aa7071-55f1-4531-9a13-9590e3d930a9-kube-api-access-w8vhc" (OuterVolumeSpecName: "kube-api-access-w8vhc") pod "41aa7071-55f1-4531-9a13-9590e3d930a9" (UID: "41aa7071-55f1-4531-9a13-9590e3d930a9"). InnerVolumeSpecName "kube-api-access-w8vhc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:05:18.611092 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:18.611078    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41aa7071-55f1-4531-9a13-9590e3d930a9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "41aa7071-55f1-4531-9a13-9590e3d930a9" (UID: "41aa7071-55f1-4531-9a13-9590e3d930a9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 20:05:18.709800 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:18.709758    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41aa7071-55f1-4531-9a13-9590e3d930a9-home\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:05:18.709800 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:18.709798    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w8vhc\" (UniqueName: \"kubernetes.io/projected/41aa7071-55f1-4531-9a13-9590e3d930a9-kube-api-access-w8vhc\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:05:18.709800 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:18.709809    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41aa7071-55f1-4531-9a13-9590e3d930a9-dshm\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:05:18.709800 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:18.709817    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41aa7071-55f1-4531-9a13-9590e3d930a9-tls-certs\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:05:18.739469 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:18.739369    2558 generic.go:356] "Generic (PLEG): container finished" podID="41aa7071-55f1-4531-9a13-9590e3d930a9" containerID="b74863ab805d17153545fdeffe54dee202dfa49075c7a351e2318a1f68f300fe" exitCode=0
Jul 27 20:05:18.739469 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:18.739441    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-79961d/test-model-79961d-kserve-8cc7d675c-8vnmt"
Jul 27 20:05:18.739469 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:18.739459    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-79961d/test-model-79961d-kserve-8cc7d675c-8vnmt" event={"ID":"41aa7071-55f1-4531-9a13-9590e3d930a9","Type":"ContainerDied","Data":"b74863ab805d17153545fdeffe54dee202dfa49075c7a351e2318a1f68f300fe"}
Jul 27 20:05:18.739688 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:18.739504    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-79961d/test-model-79961d-kserve-8cc7d675c-8vnmt" event={"ID":"41aa7071-55f1-4531-9a13-9590e3d930a9","Type":"ContainerDied","Data":"fa0eb6fd3b5f86241b1d0eedbf9100ec8c0a72a538960b2a5e8d2563e93a81a6"}
Jul 27 20:05:18.739688 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:18.739529    2558 scope.go:117] "RemoveContainer" containerID="b74863ab805d17153545fdeffe54dee202dfa49075c7a351e2318a1f68f300fe"
Jul 27 20:05:18.751963 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:18.751948    2558 scope.go:117] "RemoveContainer" containerID="b74863ab805d17153545fdeffe54dee202dfa49075c7a351e2318a1f68f300fe"
Jul 27 20:05:18.752428 ip-10-0-129-235 kubenswrapper[2558]: E0727 20:05:18.752381    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b74863ab805d17153545fdeffe54dee202dfa49075c7a351e2318a1f68f300fe\": container with ID starting with b74863ab805d17153545fdeffe54dee202dfa49075c7a351e2318a1f68f300fe not found: ID does not exist" containerID="b74863ab805d17153545fdeffe54dee202dfa49075c7a351e2318a1f68f300fe"
Jul 27 20:05:18.752541 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:18.752440    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b74863ab805d17153545fdeffe54dee202dfa49075c7a351e2318a1f68f300fe"} err="failed to get container status \"b74863ab805d17153545fdeffe54dee202dfa49075c7a351e2318a1f68f300fe\": rpc error: code = NotFound desc = could not find container \"b74863ab805d17153545fdeffe54dee202dfa49075c7a351e2318a1f68f300fe\": container with ID starting with b74863ab805d17153545fdeffe54dee202dfa49075c7a351e2318a1f68f300fe not found: ID does not exist"
Jul 27 20:05:18.762623 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:18.762602    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-79961d/test-model-79961d-kserve-8cc7d675c-8vnmt"]
Jul 27 20:05:18.764772 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:18.764751    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-79961d/test-model-79961d-kserve-8cc7d675c-8vnmt"]
Jul 27 20:05:19.108749 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:19.108660    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-08e715/test-model-08e715-kserve-84f9d5d5d9-c6hjp" podUID="3afe4f66-1cf4-4ec5-9c93-e27412ec6083" containerName="main" containerID="cri-o://552976168e3c3236792b5652d5611b2ef1c8156d58561ca7bbeee736c4213ed7" gracePeriod=45
Jul 27 20:05:19.362730 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:19.362710    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-08e715/test-model-08e715-kserve-84f9d5d5d9-c6hjp"
Jul 27 20:05:19.517632 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:19.517595    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3afe4f66-1cf4-4ec5-9c93-e27412ec6083-model-cache\") pod \"3afe4f66-1cf4-4ec5-9c93-e27412ec6083\" (UID: \"3afe4f66-1cf4-4ec5-9c93-e27412ec6083\") "
Jul 27 20:05:19.517823 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:19.517653    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3afe4f66-1cf4-4ec5-9c93-e27412ec6083-tls-certs\") pod \"3afe4f66-1cf4-4ec5-9c93-e27412ec6083\" (UID: \"3afe4f66-1cf4-4ec5-9c93-e27412ec6083\") "
Jul 27 20:05:19.517823 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:19.517694    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3afe4f66-1cf4-4ec5-9c93-e27412ec6083-home\") pod \"3afe4f66-1cf4-4ec5-9c93-e27412ec6083\" (UID: \"3afe4f66-1cf4-4ec5-9c93-e27412ec6083\") "
Jul 27 20:05:19.517823 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:19.517729    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v87xx\" (UniqueName: \"kubernetes.io/projected/3afe4f66-1cf4-4ec5-9c93-e27412ec6083-kube-api-access-v87xx\") pod \"3afe4f66-1cf4-4ec5-9c93-e27412ec6083\" (UID: \"3afe4f66-1cf4-4ec5-9c93-e27412ec6083\") "
Jul 27 20:05:19.517823 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:19.517756    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3afe4f66-1cf4-4ec5-9c93-e27412ec6083-tmp-dir\") pod \"3afe4f66-1cf4-4ec5-9c93-e27412ec6083\" (UID: \"3afe4f66-1cf4-4ec5-9c93-e27412ec6083\") "
Jul 27 20:05:19.517823 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:19.517794    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3afe4f66-1cf4-4ec5-9c93-e27412ec6083-dshm\") pod \"3afe4f66-1cf4-4ec5-9c93-e27412ec6083\" (UID: \"3afe4f66-1cf4-4ec5-9c93-e27412ec6083\") "
Jul 27 20:05:19.518100 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:19.517943    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3afe4f66-1cf4-4ec5-9c93-e27412ec6083-model-cache" (OuterVolumeSpecName: "model-cache") pod "3afe4f66-1cf4-4ec5-9c93-e27412ec6083" (UID: "3afe4f66-1cf4-4ec5-9c93-e27412ec6083"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:05:19.518100 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:19.518025    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3afe4f66-1cf4-4ec5-9c93-e27412ec6083-home" (OuterVolumeSpecName: "home") pod "3afe4f66-1cf4-4ec5-9c93-e27412ec6083" (UID: "3afe4f66-1cf4-4ec5-9c93-e27412ec6083"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:05:19.518100 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:19.518072    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3afe4f66-1cf4-4ec5-9c93-e27412ec6083-model-cache\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:05:19.518221 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:19.518107    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3afe4f66-1cf4-4ec5-9c93-e27412ec6083-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3afe4f66-1cf4-4ec5-9c93-e27412ec6083" (UID: "3afe4f66-1cf4-4ec5-9c93-e27412ec6083"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:05:19.520280 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:19.520253    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3afe4f66-1cf4-4ec5-9c93-e27412ec6083-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3afe4f66-1cf4-4ec5-9c93-e27412ec6083" (UID: "3afe4f66-1cf4-4ec5-9c93-e27412ec6083"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 20:05:19.520280 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:19.520274    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3afe4f66-1cf4-4ec5-9c93-e27412ec6083-kube-api-access-v87xx" (OuterVolumeSpecName: "kube-api-access-v87xx") pod "3afe4f66-1cf4-4ec5-9c93-e27412ec6083" (UID: "3afe4f66-1cf4-4ec5-9c93-e27412ec6083"). InnerVolumeSpecName "kube-api-access-v87xx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:05:19.520476 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:19.520279    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3afe4f66-1cf4-4ec5-9c93-e27412ec6083-dshm" (OuterVolumeSpecName: "dshm") pod "3afe4f66-1cf4-4ec5-9c93-e27412ec6083" (UID: "3afe4f66-1cf4-4ec5-9c93-e27412ec6083"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:05:19.618732 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:19.618654    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3afe4f66-1cf4-4ec5-9c93-e27412ec6083-home\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:05:19.618732 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:19.618682    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v87xx\" (UniqueName: \"kubernetes.io/projected/3afe4f66-1cf4-4ec5-9c93-e27412ec6083-kube-api-access-v87xx\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:05:19.618732 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:19.618693    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3afe4f66-1cf4-4ec5-9c93-e27412ec6083-tmp-dir\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:05:19.618732 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:19.618701    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3afe4f66-1cf4-4ec5-9c93-e27412ec6083-dshm\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:05:19.618732 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:19.618709    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3afe4f66-1cf4-4ec5-9c93-e27412ec6083-tls-certs\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:05:19.678779 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:19.678740    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41aa7071-55f1-4531-9a13-9590e3d930a9" path="/var/lib/kubelet/pods/41aa7071-55f1-4531-9a13-9590e3d930a9/volumes"
Jul 27 20:05:19.747799 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:19.747763    2558 generic.go:356] "Generic (PLEG): container finished" podID="3afe4f66-1cf4-4ec5-9c93-e27412ec6083" containerID="552976168e3c3236792b5652d5611b2ef1c8156d58561ca7bbeee736c4213ed7" exitCode=0
Jul 27 20:05:19.748181 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:19.747827    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-08e715/test-model-08e715-kserve-84f9d5d5d9-c6hjp"
Jul 27 20:05:19.748181 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:19.747845    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-08e715/test-model-08e715-kserve-84f9d5d5d9-c6hjp" event={"ID":"3afe4f66-1cf4-4ec5-9c93-e27412ec6083","Type":"ContainerDied","Data":"552976168e3c3236792b5652d5611b2ef1c8156d58561ca7bbeee736c4213ed7"}
Jul 27 20:05:19.748181 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:19.747885    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-08e715/test-model-08e715-kserve-84f9d5d5d9-c6hjp" event={"ID":"3afe4f66-1cf4-4ec5-9c93-e27412ec6083","Type":"ContainerDied","Data":"9fb4c085380f8c3c2177609b1a71829434065a9fc9730f1dec35add4af63bd39"}
Jul 27 20:05:19.748181 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:19.747902    2558 scope.go:117] "RemoveContainer" containerID="552976168e3c3236792b5652d5611b2ef1c8156d58561ca7bbeee736c4213ed7"
Jul 27 20:05:19.756876 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:19.756861    2558 scope.go:117] "RemoveContainer" containerID="552976168e3c3236792b5652d5611b2ef1c8156d58561ca7bbeee736c4213ed7"
Jul 27 20:05:19.757126 ip-10-0-129-235 kubenswrapper[2558]: E0727 20:05:19.757107    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"552976168e3c3236792b5652d5611b2ef1c8156d58561ca7bbeee736c4213ed7\": container with ID starting with 552976168e3c3236792b5652d5611b2ef1c8156d58561ca7bbeee736c4213ed7 not found: ID does not exist" containerID="552976168e3c3236792b5652d5611b2ef1c8156d58561ca7bbeee736c4213ed7"
Jul 27 20:05:19.757169 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:19.757133    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"552976168e3c3236792b5652d5611b2ef1c8156d58561ca7bbeee736c4213ed7"} err="failed to get container status \"552976168e3c3236792b5652d5611b2ef1c8156d58561ca7bbeee736c4213ed7\": rpc error: code = NotFound desc = could not find container \"552976168e3c3236792b5652d5611b2ef1c8156d58561ca7bbeee736c4213ed7\": container with ID starting with 552976168e3c3236792b5652d5611b2ef1c8156d58561ca7bbeee736c4213ed7 not found: ID does not exist"
Jul 27 20:05:19.764856 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:19.764832    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-08e715/test-model-08e715-kserve-84f9d5d5d9-c6hjp"]
Jul 27 20:05:19.768334 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:19.768310    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-08e715/test-model-08e715-kserve-84f9d5d5d9-c6hjp"]
Jul 27 20:05:21.174323 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:21.174291    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:05:21.250893 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:21.250862    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:05:21.469532 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:21.469443    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:05:21.675574 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:21.675536    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3afe4f66-1cf4-4ec5-9c93-e27412ec6083" path="/var/lib/kubelet/pods/3afe4f66-1cf4-4ec5-9c93-e27412ec6083/volumes"
Jul 27 20:05:21.678662 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:21.678627    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:05:32.379374 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:32.379339    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:05:32.380128 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:32.380104    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-08e715-85bd995d5b-j8fc8"]
Jul 27 20:05:32.380349 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:32.380314    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-08e715-85bd995d5b-j8fc8" podUID="b6215976-a00f-40dd-8f51-2115bdec63b6" containerName="payload-processing" containerID="cri-o://0dcdffd6afd5e0a743250a3e62ea0c490ad790d9da1f2db0bd04c9b23f89f362" gracePeriod=30
Jul 27 20:05:32.385220 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:32.385174    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-08e715-5c98c45688-4tst8"]
Jul 27 20:05:32.385446 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:32.385397    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-08e715-5c98c45688-4tst8" podUID="83c8bb74-e6e3-463c-9039-978c7afa64d1" containerName="payload-pre-processing" containerID="cri-o://47de7824f8988f35f1aa0a09c48fceb0c952a6a3fd9d3c36fd764d138532fee3" gracePeriod=30
Jul 27 20:05:32.478222 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:32.478188    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:05:32.580750 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:32.580715    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:05:32.667992 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:32.667965    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-08e715-85bd995d5b-j8fc8"
Jul 27 20:05:32.670807 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:32.670790    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-08e715-5c98c45688-4tst8"
Jul 27 20:05:32.725270 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:32.725237    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6215976-a00f-40dd-8f51-2115bdec63b6-plugins-config-volume\") pod \"b6215976-a00f-40dd-8f51-2115bdec63b6\" (UID: \"b6215976-a00f-40dd-8f51-2115bdec63b6\") "
Jul 27 20:05:32.725452 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:32.725335    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4rh2p\" (UniqueName: \"kubernetes.io/projected/83c8bb74-e6e3-463c-9039-978c7afa64d1-kube-api-access-4rh2p\") pod \"83c8bb74-e6e3-463c-9039-978c7afa64d1\" (UID: \"83c8bb74-e6e3-463c-9039-978c7afa64d1\") "
Jul 27 20:05:32.725452 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:32.725377    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sj86t\" (UniqueName: \"kubernetes.io/projected/b6215976-a00f-40dd-8f51-2115bdec63b6-kube-api-access-sj86t\") pod \"b6215976-a00f-40dd-8f51-2115bdec63b6\" (UID: \"b6215976-a00f-40dd-8f51-2115bdec63b6\") "
Jul 27 20:05:32.725452 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:32.725434    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/83c8bb74-e6e3-463c-9039-978c7afa64d1-plugins-config-volume\") pod \"83c8bb74-e6e3-463c-9039-978c7afa64d1\" (UID: \"83c8bb74-e6e3-463c-9039-978c7afa64d1\") "
Jul 27 20:05:32.725781 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:32.725641    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6215976-a00f-40dd-8f51-2115bdec63b6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b6215976-a00f-40dd-8f51-2115bdec63b6" (UID: "b6215976-a00f-40dd-8f51-2115bdec63b6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 20:05:32.725835 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:32.725809    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83c8bb74-e6e3-463c-9039-978c7afa64d1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "83c8bb74-e6e3-463c-9039-978c7afa64d1" (UID: "83c8bb74-e6e3-463c-9039-978c7afa64d1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 20:05:32.727655 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:32.727634    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83c8bb74-e6e3-463c-9039-978c7afa64d1-kube-api-access-4rh2p" (OuterVolumeSpecName: "kube-api-access-4rh2p") pod "83c8bb74-e6e3-463c-9039-978c7afa64d1" (UID: "83c8bb74-e6e3-463c-9039-978c7afa64d1"). InnerVolumeSpecName "kube-api-access-4rh2p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:05:32.727770 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:32.727722    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6215976-a00f-40dd-8f51-2115bdec63b6-kube-api-access-sj86t" (OuterVolumeSpecName: "kube-api-access-sj86t") pod "b6215976-a00f-40dd-8f51-2115bdec63b6" (UID: "b6215976-a00f-40dd-8f51-2115bdec63b6"). InnerVolumeSpecName "kube-api-access-sj86t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:05:32.818191 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:32.818152    2558 generic.go:356] "Generic (PLEG): container finished" podID="b6215976-a00f-40dd-8f51-2115bdec63b6" containerID="0dcdffd6afd5e0a743250a3e62ea0c490ad790d9da1f2db0bd04c9b23f89f362" exitCode=0
Jul 27 20:05:32.818345 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:32.818196    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-08e715-85bd995d5b-j8fc8"
Jul 27 20:05:32.818345 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:32.818232    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-08e715-85bd995d5b-j8fc8" event={"ID":"b6215976-a00f-40dd-8f51-2115bdec63b6","Type":"ContainerDied","Data":"0dcdffd6afd5e0a743250a3e62ea0c490ad790d9da1f2db0bd04c9b23f89f362"}
Jul 27 20:05:32.818345 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:32.818281    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-08e715-85bd995d5b-j8fc8" event={"ID":"b6215976-a00f-40dd-8f51-2115bdec63b6","Type":"ContainerDied","Data":"790e65c14c5676c16c58c2f0747bc8d8150fd051d22abd385ac295d7cd3e5c3f"}
Jul 27 20:05:32.818345 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:32.818302    2558 scope.go:117] "RemoveContainer" containerID="0dcdffd6afd5e0a743250a3e62ea0c490ad790d9da1f2db0bd04c9b23f89f362"
Jul 27 20:05:32.819777 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:32.819757    2558 generic.go:356] "Generic (PLEG): container finished" podID="83c8bb74-e6e3-463c-9039-978c7afa64d1" containerID="47de7824f8988f35f1aa0a09c48fceb0c952a6a3fd9d3c36fd764d138532fee3" exitCode=0
Jul 27 20:05:32.819878 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:32.819801    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-08e715-5c98c45688-4tst8" event={"ID":"83c8bb74-e6e3-463c-9039-978c7afa64d1","Type":"ContainerDied","Data":"47de7824f8988f35f1aa0a09c48fceb0c952a6a3fd9d3c36fd764d138532fee3"}
Jul 27 20:05:32.819878 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:32.819805    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-08e715-5c98c45688-4tst8"
Jul 27 20:05:32.819878 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:32.819838    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-08e715-5c98c45688-4tst8" event={"ID":"83c8bb74-e6e3-463c-9039-978c7afa64d1","Type":"ContainerDied","Data":"582626a6bd7bec9d013c4d8996f29d13c5b4a41e8d2f9b75aed31ee49241cfd9"}
Jul 27 20:05:32.826488 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:32.826463    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4rh2p\" (UniqueName: \"kubernetes.io/projected/83c8bb74-e6e3-463c-9039-978c7afa64d1-kube-api-access-4rh2p\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:05:32.826592 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:32.826492    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sj86t\" (UniqueName: \"kubernetes.io/projected/b6215976-a00f-40dd-8f51-2115bdec63b6-kube-api-access-sj86t\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:05:32.826592 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:32.826507    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/83c8bb74-e6e3-463c-9039-978c7afa64d1-plugins-config-volume\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:05:32.826592 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:32.826521    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6215976-a00f-40dd-8f51-2115bdec63b6-plugins-config-volume\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:05:32.828360 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:32.828342    2558 scope.go:117] "RemoveContainer" containerID="0dcdffd6afd5e0a743250a3e62ea0c490ad790d9da1f2db0bd04c9b23f89f362"
Jul 27 20:05:32.828699 ip-10-0-129-235 kubenswrapper[2558]: E0727 20:05:32.828673    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0dcdffd6afd5e0a743250a3e62ea0c490ad790d9da1f2db0bd04c9b23f89f362\": container with ID starting with 0dcdffd6afd5e0a743250a3e62ea0c490ad790d9da1f2db0bd04c9b23f89f362 not found: ID does not exist" containerID="0dcdffd6afd5e0a743250a3e62ea0c490ad790d9da1f2db0bd04c9b23f89f362"
Jul 27 20:05:32.828833 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:32.828712    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0dcdffd6afd5e0a743250a3e62ea0c490ad790d9da1f2db0bd04c9b23f89f362"} err="failed to get container status \"0dcdffd6afd5e0a743250a3e62ea0c490ad790d9da1f2db0bd04c9b23f89f362\": rpc error: code = NotFound desc = could not find container \"0dcdffd6afd5e0a743250a3e62ea0c490ad790d9da1f2db0bd04c9b23f89f362\": container with ID starting with 0dcdffd6afd5e0a743250a3e62ea0c490ad790d9da1f2db0bd04c9b23f89f362 not found: ID does not exist"
Jul 27 20:05:32.828833 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:32.828734    2558 scope.go:117] "RemoveContainer" containerID="47de7824f8988f35f1aa0a09c48fceb0c952a6a3fd9d3c36fd764d138532fee3"
Jul 27 20:05:32.839039 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:32.839021    2558 scope.go:117] "RemoveContainer" containerID="47de7824f8988f35f1aa0a09c48fceb0c952a6a3fd9d3c36fd764d138532fee3"
Jul 27 20:05:32.839303 ip-10-0-129-235 kubenswrapper[2558]: E0727 20:05:32.839282    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47de7824f8988f35f1aa0a09c48fceb0c952a6a3fd9d3c36fd764d138532fee3\": container with ID starting with 47de7824f8988f35f1aa0a09c48fceb0c952a6a3fd9d3c36fd764d138532fee3 not found: ID does not exist" containerID="47de7824f8988f35f1aa0a09c48fceb0c952a6a3fd9d3c36fd764d138532fee3"
Jul 27 20:05:32.839349 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:32.839311    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47de7824f8988f35f1aa0a09c48fceb0c952a6a3fd9d3c36fd764d138532fee3"} err="failed to get container status \"47de7824f8988f35f1aa0a09c48fceb0c952a6a3fd9d3c36fd764d138532fee3\": rpc error: code = NotFound desc = could not find container \"47de7824f8988f35f1aa0a09c48fceb0c952a6a3fd9d3c36fd764d138532fee3\": container with ID starting with 47de7824f8988f35f1aa0a09c48fceb0c952a6a3fd9d3c36fd764d138532fee3 not found: ID does not exist"
Jul 27 20:05:32.842465 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:32.842444    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-08e715-85bd995d5b-j8fc8"]
Jul 27 20:05:32.854988 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:32.854968    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-08e715-85bd995d5b-j8fc8"]
Jul 27 20:05:32.864587 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:32.864566    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-08e715-5c98c45688-4tst8"]
Jul 27 20:05:32.867966 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:32.867945    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-08e715-5c98c45688-4tst8"]
Jul 27 20:05:32.895929 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:32.895851    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:05:33.675052 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:33.675023    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83c8bb74-e6e3-463c-9039-978c7afa64d1" path="/var/lib/kubelet/pods/83c8bb74-e6e3-463c-9039-978c7afa64d1/volumes"
Jul 27 20:05:33.675475 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:33.675370    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6215976-a00f-40dd-8f51-2115bdec63b6" path="/var/lib/kubelet/pods/b6215976-a00f-40dd-8f51-2115bdec63b6/volumes"
Jul 27 20:05:39.035434 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:39.035378    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:05:39.243911 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:39.243881    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:05:39.358023 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:39.357941    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:05:39.510942 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:39.510900    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:05:39.844993 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:39.844952    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:05:40.183862 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:40.183816    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:05:40.522084 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:40.521992    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:05:43.467964 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:43.467919    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:05:43.561998 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:43.561969    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:05:43.667143 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:43.667103    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:05:44.144015 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:44.143982    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:05:44.240516 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:44.240469    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:05:44.282729 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:44.282698    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-3b36df-6bf557cb44-2n52g"]
Jul 27 20:05:44.283089 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:44.283076    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0561285a-4234-4a70-8b79-ccf03d825bbe" containerName="payload-pre-processing"
Jul 27 20:05:44.283134 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:44.283091    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="0561285a-4234-4a70-8b79-ccf03d825bbe" containerName="payload-pre-processing"
Jul 27 20:05:44.283134 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:44.283101    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3afe4f66-1cf4-4ec5-9c93-e27412ec6083" containerName="main"
Jul 27 20:05:44.283134 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:44.283106    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="3afe4f66-1cf4-4ec5-9c93-e27412ec6083" containerName="main"
Jul 27 20:05:44.283134 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:44.283113    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41aa7071-55f1-4531-9a13-9590e3d930a9" containerName="main"
Jul 27 20:05:44.283134 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:44.283118    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="41aa7071-55f1-4531-9a13-9590e3d930a9" containerName="main"
Jul 27 20:05:44.283134 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:44.283129    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a72f7964-07ae-46a1-924d-6cff28a37d23" containerName="payload-processing"
Jul 27 20:05:44.283317 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:44.283137    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="a72f7964-07ae-46a1-924d-6cff28a37d23" containerName="payload-processing"
Jul 27 20:05:44.283317 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:44.283151    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6215976-a00f-40dd-8f51-2115bdec63b6" containerName="payload-processing"
Jul 27 20:05:44.283317 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:44.283157    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6215976-a00f-40dd-8f51-2115bdec63b6" containerName="payload-processing"
Jul 27 20:05:44.283317 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:44.283164    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83c8bb74-e6e3-463c-9039-978c7afa64d1" containerName="payload-pre-processing"
Jul 27 20:05:44.283317 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:44.283169    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="83c8bb74-e6e3-463c-9039-978c7afa64d1" containerName="payload-pre-processing"
Jul 27 20:05:44.283317 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:44.283235    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="3afe4f66-1cf4-4ec5-9c93-e27412ec6083" containerName="main"
Jul 27 20:05:44.283317 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:44.283244    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="a72f7964-07ae-46a1-924d-6cff28a37d23" containerName="payload-processing"
Jul 27 20:05:44.283317 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:44.283250    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6215976-a00f-40dd-8f51-2115bdec63b6" containerName="payload-processing"
Jul 27 20:05:44.283317 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:44.283255    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="0561285a-4234-4a70-8b79-ccf03d825bbe" containerName="payload-pre-processing"
Jul 27 20:05:44.283317 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:44.283261    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="83c8bb74-e6e3-463c-9039-978c7afa64d1" containerName="payload-pre-processing"
Jul 27 20:05:44.283317 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:44.283267    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="41aa7071-55f1-4531-9a13-9590e3d930a9" containerName="main"
Jul 27 20:05:44.287971 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:44.287939    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-3b36df-6bf557cb44-2n52g"
Jul 27 20:05:44.290168 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:44.290149    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-3b36df\""
Jul 27 20:05:44.290525 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:44.290511    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-3b36df-dockercfg-qhd4b\""
Jul 27 20:05:44.294436 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:44.294394    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-3b36df-6bf557cb44-2n52g"]
Jul 27 20:05:44.319269 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:44.319242    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-3b36df-585f45c88d-fzttb"]
Jul 27 20:05:44.321197 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:44.321172    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c77f739-4e6c-45ed-8a40-bba25cbade0e-plugins-config-volume\") pod \"payload-processing-e2e-api-a-3b36df-6bf557cb44-2n52g\" (UID: \"9c77f739-4e6c-45ed-8a40-bba25cbade0e\") " pod="openshift-ingress/payload-processing-e2e-api-a-3b36df-6bf557cb44-2n52g"
Jul 27 20:05:44.321344 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:44.321245    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tprqc\" (UniqueName: \"kubernetes.io/projected/9c77f739-4e6c-45ed-8a40-bba25cbade0e-kube-api-access-tprqc\") pod \"payload-processing-e2e-api-a-3b36df-6bf557cb44-2n52g\" (UID: \"9c77f739-4e6c-45ed-8a40-bba25cbade0e\") " pod="openshift-ingress/payload-processing-e2e-api-a-3b36df-6bf557cb44-2n52g"
Jul 27 20:05:44.322906 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:44.322887    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-3b36df-585f45c88d-fzttb"
Jul 27 20:05:44.331382 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:44.331362    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-3b36df-585f45c88d-fzttb"]
Jul 27 20:05:44.421893 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:44.421803    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tprqc\" (UniqueName: \"kubernetes.io/projected/9c77f739-4e6c-45ed-8a40-bba25cbade0e-kube-api-access-tprqc\") pod \"payload-processing-e2e-api-a-3b36df-6bf557cb44-2n52g\" (UID: \"9c77f739-4e6c-45ed-8a40-bba25cbade0e\") " pod="openshift-ingress/payload-processing-e2e-api-a-3b36df-6bf557cb44-2n52g"
Jul 27 20:05:44.421893 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:44.421886    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5721763e-cb65-4054-bc22-fce240343e64-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-3b36df-585f45c88d-fzttb\" (UID: \"5721763e-cb65-4054-bc22-fce240343e64\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-3b36df-585f45c88d-fzttb"
Jul 27 20:05:44.422111 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:44.421921    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c77f739-4e6c-45ed-8a40-bba25cbade0e-plugins-config-volume\") pod \"payload-processing-e2e-api-a-3b36df-6bf557cb44-2n52g\" (UID: \"9c77f739-4e6c-45ed-8a40-bba25cbade0e\") " pod="openshift-ingress/payload-processing-e2e-api-a-3b36df-6bf557cb44-2n52g"
Jul 27 20:05:44.422111 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:44.421988    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xjpq\" (UniqueName: \"kubernetes.io/projected/5721763e-cb65-4054-bc22-fce240343e64-kube-api-access-7xjpq\") pod \"payload-pre-processing-e2e-api-a-3b36df-585f45c88d-fzttb\" (UID: \"5721763e-cb65-4054-bc22-fce240343e64\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-3b36df-585f45c88d-fzttb"
Jul 27 20:05:44.422824 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:44.422799    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c77f739-4e6c-45ed-8a40-bba25cbade0e-plugins-config-volume\") pod \"payload-processing-e2e-api-a-3b36df-6bf557cb44-2n52g\" (UID: \"9c77f739-4e6c-45ed-8a40-bba25cbade0e\") " pod="openshift-ingress/payload-processing-e2e-api-a-3b36df-6bf557cb44-2n52g"
Jul 27 20:05:44.429339 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:44.429315    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tprqc\" (UniqueName: \"kubernetes.io/projected/9c77f739-4e6c-45ed-8a40-bba25cbade0e-kube-api-access-tprqc\") pod \"payload-processing-e2e-api-a-3b36df-6bf557cb44-2n52g\" (UID: \"9c77f739-4e6c-45ed-8a40-bba25cbade0e\") " pod="openshift-ingress/payload-processing-e2e-api-a-3b36df-6bf557cb44-2n52g"
Jul 27 20:05:44.522941 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:44.522900    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5721763e-cb65-4054-bc22-fce240343e64-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-3b36df-585f45c88d-fzttb\" (UID: \"5721763e-cb65-4054-bc22-fce240343e64\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-3b36df-585f45c88d-fzttb"
Jul 27 20:05:44.523449 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:44.522945    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xjpq\" (UniqueName: \"kubernetes.io/projected/5721763e-cb65-4054-bc22-fce240343e64-kube-api-access-7xjpq\") pod \"payload-pre-processing-e2e-api-a-3b36df-585f45c88d-fzttb\" (UID: \"5721763e-cb65-4054-bc22-fce240343e64\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-3b36df-585f45c88d-fzttb"
Jul 27 20:05:44.523645 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:44.523624    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5721763e-cb65-4054-bc22-fce240343e64-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-3b36df-585f45c88d-fzttb\" (UID: \"5721763e-cb65-4054-bc22-fce240343e64\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-3b36df-585f45c88d-fzttb"
Jul 27 20:05:44.533550 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:44.533520    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xjpq\" (UniqueName: \"kubernetes.io/projected/5721763e-cb65-4054-bc22-fce240343e64-kube-api-access-7xjpq\") pod \"payload-pre-processing-e2e-api-a-3b36df-585f45c88d-fzttb\" (UID: \"5721763e-cb65-4054-bc22-fce240343e64\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-3b36df-585f45c88d-fzttb"
Jul 27 20:05:44.599395 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:44.599351    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-3b36df-6bf557cb44-2n52g"
Jul 27 20:05:44.633295 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:44.633265    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-3b36df-585f45c88d-fzttb"
Jul 27 20:05:44.751978 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:44.751951    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-3b36df-6bf557cb44-2n52g"]
Jul 27 20:05:44.754356 ip-10-0-129-235 kubenswrapper[2558]: W0727 20:05:44.754317    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c77f739_4e6c_45ed_8a40_bba25cbade0e.slice/crio-831793d48319b92cd7e8e304f08c50e0529303631c350f287d7b15ed4a5b7b71 WatchSource:0}: Error finding container 831793d48319b92cd7e8e304f08c50e0529303631c350f287d7b15ed4a5b7b71: Status 404 returned error can't find the container with id 831793d48319b92cd7e8e304f08c50e0529303631c350f287d7b15ed4a5b7b71
Jul 27 20:05:44.792213 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:44.792186    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-3b36df-585f45c88d-fzttb"]
Jul 27 20:05:44.794099 ip-10-0-129-235 kubenswrapper[2558]: W0727 20:05:44.794073    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5721763e_cb65_4054_bc22_fce240343e64.slice/crio-aa00204e2fae8f15b7ce314c09877f80b397f53e47a878748b2192843b41921d WatchSource:0}: Error finding container aa00204e2fae8f15b7ce314c09877f80b397f53e47a878748b2192843b41921d: Status 404 returned error can't find the container with id aa00204e2fae8f15b7ce314c09877f80b397f53e47a878748b2192843b41921d
Jul 27 20:05:44.889642 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:44.889616    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-3b36df-6bf557cb44-2n52g" event={"ID":"9c77f739-4e6c-45ed-8a40-bba25cbade0e","Type":"ContainerStarted","Data":"d77bd17cd42522e4dd710522b1423dbfe87810225876ea3617008895d4d3d11b"}
Jul 27 20:05:44.889778 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:44.889663    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-3b36df-6bf557cb44-2n52g" event={"ID":"9c77f739-4e6c-45ed-8a40-bba25cbade0e","Type":"ContainerStarted","Data":"831793d48319b92cd7e8e304f08c50e0529303631c350f287d7b15ed4a5b7b71"}
Jul 27 20:05:44.889778 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:44.889714    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-3b36df-6bf557cb44-2n52g"
Jul 27 20:05:44.891214 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:44.891196    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-3b36df-585f45c88d-fzttb" event={"ID":"5721763e-cb65-4054-bc22-fce240343e64","Type":"ContainerStarted","Data":"f24ee1e98377ae51fca37f6a091aafc6425efad4d779d6f2ed7d7d1dfc365b95"}
Jul 27 20:05:44.891309 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:44.891227    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-3b36df-585f45c88d-fzttb" event={"ID":"5721763e-cb65-4054-bc22-fce240343e64","Type":"ContainerStarted","Data":"aa00204e2fae8f15b7ce314c09877f80b397f53e47a878748b2192843b41921d"}
Jul 27 20:05:44.891346 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:44.891308    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-3b36df-585f45c88d-fzttb"
Jul 27 20:05:44.939826 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:44.939731    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-3b36df-6bf557cb44-2n52g" podStartSLOduration=0.939715761 podStartE2EDuration="939.715761ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:05:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:05:44.914968871 +0000 UTC m=+2357.787160856" watchObservedRunningTime="2026-07-27 20:05:44.939715761 +0000 UTC m=+2357.811907734"
Jul 27 20:05:44.940507 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:44.940462    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-3b36df-585f45c88d-fzttb" podStartSLOduration=0.940368881 podStartE2EDuration="940.368881ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:05:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:05:44.938828273 +0000 UTC m=+2357.811020260" watchObservedRunningTime="2026-07-27 20:05:44.940368881 +0000 UTC m=+2357.812560857"
Jul 27 20:05:47.975588 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:47.975548    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:05:48.425167 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:48.425133    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:05:48.717394 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:48.717299    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:05:48.940391 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:48.940345    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:05:51.962703 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:51.962658    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:05:52.043765 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:52.043715    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:05:52.116622 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:52.116577    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:05:52.481318 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:52.481283    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:05:52.646459 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:52.646396    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:05:52.717555 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:52.717370    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-d978f1-54fb5b9f48-lmrss"]
Jul 27 20:05:52.721134 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:52.721110    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d978f1-54fb5b9f48-lmrss"
Jul 27 20:05:52.723483 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:52.723458    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-d978f1\""
Jul 27 20:05:52.723647 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:52.723497    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-d978f1-dockercfg-hjmn2\""
Jul 27 20:05:52.730464 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:52.730440    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-d978f1-54fb5b9f48-lmrss"]
Jul 27 20:05:52.732342 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:52.732269    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:05:52.770550 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:52.770521    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-d978f1-658d59987b-wwm6q"]
Jul 27 20:05:52.774140 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:52.774124    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-d978f1-658d59987b-wwm6q"
Jul 27 20:05:52.783313 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:52.783288    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-d978f1-658d59987b-wwm6q"]
Jul 27 20:05:52.802221 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:52.802187    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hjbh\" (UniqueName: \"kubernetes.io/projected/871778fc-7792-49f6-b9d8-99736df6c541-kube-api-access-2hjbh\") pod \"payload-pre-processing-e2e-api-b-d978f1-54fb5b9f48-lmrss\" (UID: \"871778fc-7792-49f6-b9d8-99736df6c541\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-d978f1-54fb5b9f48-lmrss"
Jul 27 20:05:52.802356 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:52.802257    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/871778fc-7792-49f6-b9d8-99736df6c541-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-d978f1-54fb5b9f48-lmrss\" (UID: \"871778fc-7792-49f6-b9d8-99736df6c541\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-d978f1-54fb5b9f48-lmrss"
Jul 27 20:05:52.903731 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:52.903692    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/871778fc-7792-49f6-b9d8-99736df6c541-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-d978f1-54fb5b9f48-lmrss\" (UID: \"871778fc-7792-49f6-b9d8-99736df6c541\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-d978f1-54fb5b9f48-lmrss"
Jul 27 20:05:52.903918 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:52.903757    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ace9f826-4ca5-480b-88ba-833ea8391892-plugins-config-volume\") pod \"payload-processing-e2e-api-b-d978f1-658d59987b-wwm6q\" (UID: \"ace9f826-4ca5-480b-88ba-833ea8391892\") " pod="openshift-ingress/payload-processing-e2e-api-b-d978f1-658d59987b-wwm6q"
Jul 27 20:05:52.903918 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:52.903800    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wt6hz\" (UniqueName: \"kubernetes.io/projected/ace9f826-4ca5-480b-88ba-833ea8391892-kube-api-access-wt6hz\") pod \"payload-processing-e2e-api-b-d978f1-658d59987b-wwm6q\" (UID: \"ace9f826-4ca5-480b-88ba-833ea8391892\") " pod="openshift-ingress/payload-processing-e2e-api-b-d978f1-658d59987b-wwm6q"
Jul 27 20:05:52.904040 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:52.903938    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hjbh\" (UniqueName: \"kubernetes.io/projected/871778fc-7792-49f6-b9d8-99736df6c541-kube-api-access-2hjbh\") pod \"payload-pre-processing-e2e-api-b-d978f1-54fb5b9f48-lmrss\" (UID: \"871778fc-7792-49f6-b9d8-99736df6c541\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-d978f1-54fb5b9f48-lmrss"
Jul 27 20:05:52.904354 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:52.904334    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/871778fc-7792-49f6-b9d8-99736df6c541-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-d978f1-54fb5b9f48-lmrss\" (UID: \"871778fc-7792-49f6-b9d8-99736df6c541\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-d978f1-54fb5b9f48-lmrss"
Jul 27 20:05:52.912245 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:52.912223    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hjbh\" (UniqueName: \"kubernetes.io/projected/871778fc-7792-49f6-b9d8-99736df6c541-kube-api-access-2hjbh\") pod \"payload-pre-processing-e2e-api-b-d978f1-54fb5b9f48-lmrss\" (UID: \"871778fc-7792-49f6-b9d8-99736df6c541\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-d978f1-54fb5b9f48-lmrss"
Jul 27 20:05:53.004742 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:53.004664    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ace9f826-4ca5-480b-88ba-833ea8391892-plugins-config-volume\") pod \"payload-processing-e2e-api-b-d978f1-658d59987b-wwm6q\" (UID: \"ace9f826-4ca5-480b-88ba-833ea8391892\") " pod="openshift-ingress/payload-processing-e2e-api-b-d978f1-658d59987b-wwm6q"
Jul 27 20:05:53.004742 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:53.004708    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wt6hz\" (UniqueName: \"kubernetes.io/projected/ace9f826-4ca5-480b-88ba-833ea8391892-kube-api-access-wt6hz\") pod \"payload-processing-e2e-api-b-d978f1-658d59987b-wwm6q\" (UID: \"ace9f826-4ca5-480b-88ba-833ea8391892\") " pod="openshift-ingress/payload-processing-e2e-api-b-d978f1-658d59987b-wwm6q"
Jul 27 20:05:53.005361 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:53.005339    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ace9f826-4ca5-480b-88ba-833ea8391892-plugins-config-volume\") pod \"payload-processing-e2e-api-b-d978f1-658d59987b-wwm6q\" (UID: \"ace9f826-4ca5-480b-88ba-833ea8391892\") " pod="openshift-ingress/payload-processing-e2e-api-b-d978f1-658d59987b-wwm6q"
Jul 27 20:05:53.012303 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:53.012281    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wt6hz\" (UniqueName: \"kubernetes.io/projected/ace9f826-4ca5-480b-88ba-833ea8391892-kube-api-access-wt6hz\") pod \"payload-processing-e2e-api-b-d978f1-658d59987b-wwm6q\" (UID: \"ace9f826-4ca5-480b-88ba-833ea8391892\") " pod="openshift-ingress/payload-processing-e2e-api-b-d978f1-658d59987b-wwm6q"
Jul 27 20:05:53.033761 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:53.033736    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d978f1-54fb5b9f48-lmrss"
Jul 27 20:05:53.085007 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:53.084975    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-d978f1-658d59987b-wwm6q"
Jul 27 20:05:53.181946 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:53.181918    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-d978f1-54fb5b9f48-lmrss"]
Jul 27 20:05:53.184614 ip-10-0-129-235 kubenswrapper[2558]: W0727 20:05:53.184506    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod871778fc_7792_49f6_b9d8_99736df6c541.slice/crio-77e5d833d0d5c4ee0446b82ec6290c9b56889c44db90eb01220ee2ae8e4aa6bf WatchSource:0}: Error finding container 77e5d833d0d5c4ee0446b82ec6290c9b56889c44db90eb01220ee2ae8e4aa6bf: Status 404 returned error can't find the container with id 77e5d833d0d5c4ee0446b82ec6290c9b56889c44db90eb01220ee2ae8e4aa6bf
Jul 27 20:05:53.243838 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:53.243813    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-d978f1-658d59987b-wwm6q"]
Jul 27 20:05:53.246053 ip-10-0-129-235 kubenswrapper[2558]: W0727 20:05:53.246022    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podace9f826_4ca5_480b_88ba_833ea8391892.slice/crio-abd7a3e86d2477b6ad4033e4f6da50cf134c71195200b39fdf47c4c0c38390d3 WatchSource:0}: Error finding container abd7a3e86d2477b6ad4033e4f6da50cf134c71195200b39fdf47c4c0c38390d3: Status 404 returned error can't find the container with id abd7a3e86d2477b6ad4033e4f6da50cf134c71195200b39fdf47c4c0c38390d3
Jul 27 20:05:53.944768 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:53.944732    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-d978f1-658d59987b-wwm6q" event={"ID":"ace9f826-4ca5-480b-88ba-833ea8391892","Type":"ContainerStarted","Data":"54bfd764a575356f917f7ba08e68fb3ebbd0d637b8feac4c8e35765b61b1a6c5"}
Jul 27 20:05:53.944989 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:53.944793    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-d978f1-658d59987b-wwm6q" event={"ID":"ace9f826-4ca5-480b-88ba-833ea8391892","Type":"ContainerStarted","Data":"abd7a3e86d2477b6ad4033e4f6da50cf134c71195200b39fdf47c4c0c38390d3"}
Jul 27 20:05:53.944989 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:53.944823    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-d978f1-658d59987b-wwm6q"
Jul 27 20:05:53.946324 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:53.946289    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d978f1-54fb5b9f48-lmrss" event={"ID":"871778fc-7792-49f6-b9d8-99736df6c541","Type":"ContainerStarted","Data":"05a2a5e77cafd8f5d7e5a1c7e5804e49e3c3a9d5d76c8c53b46ffba556fabea5"}
Jul 27 20:05:53.946477 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:53.946342    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d978f1-54fb5b9f48-lmrss" event={"ID":"871778fc-7792-49f6-b9d8-99736df6c541","Type":"ContainerStarted","Data":"77e5d833d0d5c4ee0446b82ec6290c9b56889c44db90eb01220ee2ae8e4aa6bf"}
Jul 27 20:05:53.946477 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:53.946370    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d978f1-54fb5b9f48-lmrss"
Jul 27 20:05:53.965611 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:53.965564    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-d978f1-658d59987b-wwm6q" podStartSLOduration=1.965547236 podStartE2EDuration="1.965547236s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:05:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:05:53.963184895 +0000 UTC m=+2366.835376881" watchObservedRunningTime="2026-07-27 20:05:53.965547236 +0000 UTC m=+2366.837739211"
Jul 27 20:05:53.977806 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:53.977755    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d978f1-54fb5b9f48-lmrss" podStartSLOduration=1.977736836 podStartE2EDuration="1.977736836s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:05:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:05:53.977089276 +0000 UTC m=+2366.849281251" watchObservedRunningTime="2026-07-27 20:05:53.977736836 +0000 UTC m=+2366.849928811"
Jul 27 20:05:55.898213 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:55.898183    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-3b36df-6bf557cb44-2n52g"
Jul 27 20:05:55.898689 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:55.898470    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-3b36df-585f45c88d-fzttb"
Jul 27 20:05:56.485171 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:05:56.485139    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:06:04.954515 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:04.954481    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d978f1-54fb5b9f48-lmrss"
Jul 27 20:06:04.954959 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:04.954564    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-d978f1-658d59987b-wwm6q"
Jul 27 20:06:20.017373 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:20.017323    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:06:20.035461 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:20.035403    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-3b36df-6bf557cb44-2n52g"]
Jul 27 20:06:20.035786 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:20.035706    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-3b36df-6bf557cb44-2n52g" podUID="9c77f739-4e6c-45ed-8a40-bba25cbade0e" containerName="payload-processing" containerID="cri-o://d77bd17cd42522e4dd710522b1423dbfe87810225876ea3617008895d4d3d11b" gracePeriod=30
Jul 27 20:06:20.047429 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:20.047376    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-3b36df-585f45c88d-fzttb"]
Jul 27 20:06:20.047890 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:20.047835    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-3b36df-585f45c88d-fzttb" podUID="5721763e-cb65-4054-bc22-fce240343e64" containerName="payload-pre-processing" containerID="cri-o://f24ee1e98377ae51fca37f6a091aafc6425efad4d779d6f2ed7d7d1dfc365b95" gracePeriod=30
Jul 27 20:06:20.117489 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:20.117264    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:06:20.234132 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:20.234096    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:06:20.333029 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:20.333004    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-3b36df-6bf557cb44-2n52g"
Jul 27 20:06:20.336809 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:20.336779    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-3b36df-585f45c88d-fzttb"
Jul 27 20:06:20.355571 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:20.355539    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tprqc\" (UniqueName: \"kubernetes.io/projected/9c77f739-4e6c-45ed-8a40-bba25cbade0e-kube-api-access-tprqc\") pod \"9c77f739-4e6c-45ed-8a40-bba25cbade0e\" (UID: \"9c77f739-4e6c-45ed-8a40-bba25cbade0e\") "
Jul 27 20:06:20.355741 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:20.355584    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c77f739-4e6c-45ed-8a40-bba25cbade0e-plugins-config-volume\") pod \"9c77f739-4e6c-45ed-8a40-bba25cbade0e\" (UID: \"9c77f739-4e6c-45ed-8a40-bba25cbade0e\") "
Jul 27 20:06:20.355741 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:20.355611    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5721763e-cb65-4054-bc22-fce240343e64-plugins-config-volume\") pod \"5721763e-cb65-4054-bc22-fce240343e64\" (UID: \"5721763e-cb65-4054-bc22-fce240343e64\") "
Jul 27 20:06:20.355741 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:20.355709    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7xjpq\" (UniqueName: \"kubernetes.io/projected/5721763e-cb65-4054-bc22-fce240343e64-kube-api-access-7xjpq\") pod \"5721763e-cb65-4054-bc22-fce240343e64\" (UID: \"5721763e-cb65-4054-bc22-fce240343e64\") "
Jul 27 20:06:20.356012 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:20.355991    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c77f739-4e6c-45ed-8a40-bba25cbade0e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9c77f739-4e6c-45ed-8a40-bba25cbade0e" (UID: "9c77f739-4e6c-45ed-8a40-bba25cbade0e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 20:06:20.356084 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:20.356050    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5721763e-cb65-4054-bc22-fce240343e64-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5721763e-cb65-4054-bc22-fce240343e64" (UID: "5721763e-cb65-4054-bc22-fce240343e64"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 20:06:20.357858 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:20.357825    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c77f739-4e6c-45ed-8a40-bba25cbade0e-kube-api-access-tprqc" (OuterVolumeSpecName: "kube-api-access-tprqc") pod "9c77f739-4e6c-45ed-8a40-bba25cbade0e" (UID: "9c77f739-4e6c-45ed-8a40-bba25cbade0e"). InnerVolumeSpecName "kube-api-access-tprqc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:06:20.357998 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:20.357867    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5721763e-cb65-4054-bc22-fce240343e64-kube-api-access-7xjpq" (OuterVolumeSpecName: "kube-api-access-7xjpq") pod "5721763e-cb65-4054-bc22-fce240343e64" (UID: "5721763e-cb65-4054-bc22-fce240343e64"). InnerVolumeSpecName "kube-api-access-7xjpq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:06:20.456526 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:20.456488    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7xjpq\" (UniqueName: \"kubernetes.io/projected/5721763e-cb65-4054-bc22-fce240343e64-kube-api-access-7xjpq\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:06:20.456526 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:20.456521    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tprqc\" (UniqueName: \"kubernetes.io/projected/9c77f739-4e6c-45ed-8a40-bba25cbade0e-kube-api-access-tprqc\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:06:20.456526 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:20.456530    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c77f739-4e6c-45ed-8a40-bba25cbade0e-plugins-config-volume\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:06:20.456526 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:20.456540    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5721763e-cb65-4054-bc22-fce240343e64-plugins-config-volume\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:06:20.534909 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:20.534867    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:06:21.110990 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:21.110951    2558 generic.go:356] "Generic (PLEG): container finished" podID="5721763e-cb65-4054-bc22-fce240343e64" containerID="f24ee1e98377ae51fca37f6a091aafc6425efad4d779d6f2ed7d7d1dfc365b95" exitCode=0
Jul 27 20:06:21.111470 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:21.111001    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-3b36df-585f45c88d-fzttb"
Jul 27 20:06:21.111470 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:21.111036    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-3b36df-585f45c88d-fzttb" event={"ID":"5721763e-cb65-4054-bc22-fce240343e64","Type":"ContainerDied","Data":"f24ee1e98377ae51fca37f6a091aafc6425efad4d779d6f2ed7d7d1dfc365b95"}
Jul 27 20:06:21.111470 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:21.111088    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-3b36df-585f45c88d-fzttb" event={"ID":"5721763e-cb65-4054-bc22-fce240343e64","Type":"ContainerDied","Data":"aa00204e2fae8f15b7ce314c09877f80b397f53e47a878748b2192843b41921d"}
Jul 27 20:06:21.111470 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:21.111116    2558 scope.go:117] "RemoveContainer" containerID="f24ee1e98377ae51fca37f6a091aafc6425efad4d779d6f2ed7d7d1dfc365b95"
Jul 27 20:06:21.112561 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:21.112484    2558 generic.go:356] "Generic (PLEG): container finished" podID="9c77f739-4e6c-45ed-8a40-bba25cbade0e" containerID="d77bd17cd42522e4dd710522b1423dbfe87810225876ea3617008895d4d3d11b" exitCode=0
Jul 27 20:06:21.112561 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:21.112517    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-3b36df-6bf557cb44-2n52g" event={"ID":"9c77f739-4e6c-45ed-8a40-bba25cbade0e","Type":"ContainerDied","Data":"d77bd17cd42522e4dd710522b1423dbfe87810225876ea3617008895d4d3d11b"}
Jul 27 20:06:21.112561 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:21.112550    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-3b36df-6bf557cb44-2n52g"
Jul 27 20:06:21.112699 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:21.112582    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-3b36df-6bf557cb44-2n52g" event={"ID":"9c77f739-4e6c-45ed-8a40-bba25cbade0e","Type":"ContainerDied","Data":"831793d48319b92cd7e8e304f08c50e0529303631c350f287d7b15ed4a5b7b71"}
Jul 27 20:06:21.120979 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:21.120956    2558 scope.go:117] "RemoveContainer" containerID="f24ee1e98377ae51fca37f6a091aafc6425efad4d779d6f2ed7d7d1dfc365b95"
Jul 27 20:06:21.121362 ip-10-0-129-235 kubenswrapper[2558]: E0727 20:06:21.121329    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f24ee1e98377ae51fca37f6a091aafc6425efad4d779d6f2ed7d7d1dfc365b95\": container with ID starting with f24ee1e98377ae51fca37f6a091aafc6425efad4d779d6f2ed7d7d1dfc365b95 not found: ID does not exist" containerID="f24ee1e98377ae51fca37f6a091aafc6425efad4d779d6f2ed7d7d1dfc365b95"
Jul 27 20:06:21.121511 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:21.121375    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f24ee1e98377ae51fca37f6a091aafc6425efad4d779d6f2ed7d7d1dfc365b95"} err="failed to get container status \"f24ee1e98377ae51fca37f6a091aafc6425efad4d779d6f2ed7d7d1dfc365b95\": rpc error: code = NotFound desc = could not find container \"f24ee1e98377ae51fca37f6a091aafc6425efad4d779d6f2ed7d7d1dfc365b95\": container with ID starting with f24ee1e98377ae51fca37f6a091aafc6425efad4d779d6f2ed7d7d1dfc365b95 not found: ID does not exist"
Jul 27 20:06:21.121511 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:21.121400    2558 scope.go:117] "RemoveContainer" containerID="d77bd17cd42522e4dd710522b1423dbfe87810225876ea3617008895d4d3d11b"
Jul 27 20:06:21.131069 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:21.131043    2558 scope.go:117] "RemoveContainer" containerID="d77bd17cd42522e4dd710522b1423dbfe87810225876ea3617008895d4d3d11b"
Jul 27 20:06:21.131391 ip-10-0-129-235 kubenswrapper[2558]: E0727 20:06:21.131370    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d77bd17cd42522e4dd710522b1423dbfe87810225876ea3617008895d4d3d11b\": container with ID starting with d77bd17cd42522e4dd710522b1423dbfe87810225876ea3617008895d4d3d11b not found: ID does not exist" containerID="d77bd17cd42522e4dd710522b1423dbfe87810225876ea3617008895d4d3d11b"
Jul 27 20:06:21.131470 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:21.131402    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d77bd17cd42522e4dd710522b1423dbfe87810225876ea3617008895d4d3d11b"} err="failed to get container status \"d77bd17cd42522e4dd710522b1423dbfe87810225876ea3617008895d4d3d11b\": rpc error: code = NotFound desc = could not find container \"d77bd17cd42522e4dd710522b1423dbfe87810225876ea3617008895d4d3d11b\": container with ID starting with d77bd17cd42522e4dd710522b1423dbfe87810225876ea3617008895d4d3d11b not found: ID does not exist"
Jul 27 20:06:21.138261 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:21.138228    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-3b36df-585f45c88d-fzttb"]
Jul 27 20:06:21.140678 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:21.140651    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-3b36df-585f45c88d-fzttb"]
Jul 27 20:06:21.149467 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:21.149428    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-3b36df-6bf557cb44-2n52g"]
Jul 27 20:06:21.156936 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:21.156902    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-3b36df-6bf557cb44-2n52g"]
Jul 27 20:06:21.676141 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:21.676101    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5721763e-cb65-4054-bc22-fce240343e64" path="/var/lib/kubelet/pods/5721763e-cb65-4054-bc22-fce240343e64/volumes"
Jul 27 20:06:21.676515 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:21.676500    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c77f739-4e6c-45ed-8a40-bba25cbade0e" path="/var/lib/kubelet/pods/9c77f739-4e6c-45ed-8a40-bba25cbade0e/volumes"
Jul 27 20:06:26.556447 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:26.556384    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:06:26.626117 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:26.626084    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:06:26.817558 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:26.817469    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:06:27.082161 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:27.082071    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:06:27.765005 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:27.764976    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5dfc2_103bab23-bca3-472a-974b-7731f83c5bfb/ovn-acl-logging/0.log"
Jul 27 20:06:27.774765 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:27.774741    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5dfc2_103bab23-bca3-472a-974b-7731f83c5bfb/ovn-acl-logging/0.log"
Jul 27 20:06:42.583253 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:42.583213    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:06:42.608629 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:42.608591    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-d978f1-658d59987b-wwm6q"]
Jul 27 20:06:42.609498 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:42.608885    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-d978f1-658d59987b-wwm6q" podUID="ace9f826-4ca5-480b-88ba-833ea8391892" containerName="payload-processing" containerID="cri-o://54bfd764a575356f917f7ba08e68fb3ebbd0d637b8feac4c8e35765b61b1a6c5" gracePeriod=30
Jul 27 20:06:42.612893 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:42.612845    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-d978f1-54fb5b9f48-lmrss"]
Jul 27 20:06:42.613381 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:42.613327    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d978f1-54fb5b9f48-lmrss" podUID="871778fc-7792-49f6-b9d8-99736df6c541" containerName="payload-pre-processing" containerID="cri-o://05a2a5e77cafd8f5d7e5a1c7e5804e49e3c3a9d5d76c8c53b46ffba556fabea5" gracePeriod=30
Jul 27 20:06:42.683311 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:42.683284    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:06:42.786394 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:42.786358    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:06:42.875016 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:42.874981    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d978f1-54fb5b9f48-lmrss"
Jul 27 20:06:42.887085 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:42.887058    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-d978f1-658d59987b-wwm6q"
Jul 27 20:06:42.968670 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:42.968628    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2hjbh\" (UniqueName: \"kubernetes.io/projected/871778fc-7792-49f6-b9d8-99736df6c541-kube-api-access-2hjbh\") pod \"871778fc-7792-49f6-b9d8-99736df6c541\" (UID: \"871778fc-7792-49f6-b9d8-99736df6c541\") "
Jul 27 20:06:42.968882 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:42.968694    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/871778fc-7792-49f6-b9d8-99736df6c541-plugins-config-volume\") pod \"871778fc-7792-49f6-b9d8-99736df6c541\" (UID: \"871778fc-7792-49f6-b9d8-99736df6c541\") "
Jul 27 20:06:42.968882 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:42.968727    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wt6hz\" (UniqueName: \"kubernetes.io/projected/ace9f826-4ca5-480b-88ba-833ea8391892-kube-api-access-wt6hz\") pod \"ace9f826-4ca5-480b-88ba-833ea8391892\" (UID: \"ace9f826-4ca5-480b-88ba-833ea8391892\") "
Jul 27 20:06:42.968882 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:42.968768    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ace9f826-4ca5-480b-88ba-833ea8391892-plugins-config-volume\") pod \"ace9f826-4ca5-480b-88ba-833ea8391892\" (UID: \"ace9f826-4ca5-480b-88ba-833ea8391892\") "
Jul 27 20:06:42.969248 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:42.969221    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ace9f826-4ca5-480b-88ba-833ea8391892-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ace9f826-4ca5-480b-88ba-833ea8391892" (UID: "ace9f826-4ca5-480b-88ba-833ea8391892"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 20:06:42.969354 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:42.969224    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/871778fc-7792-49f6-b9d8-99736df6c541-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "871778fc-7792-49f6-b9d8-99736df6c541" (UID: "871778fc-7792-49f6-b9d8-99736df6c541"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 20:06:42.971170 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:42.971146    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ace9f826-4ca5-480b-88ba-833ea8391892-kube-api-access-wt6hz" (OuterVolumeSpecName: "kube-api-access-wt6hz") pod "ace9f826-4ca5-480b-88ba-833ea8391892" (UID: "ace9f826-4ca5-480b-88ba-833ea8391892"). InnerVolumeSpecName "kube-api-access-wt6hz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:06:42.971228 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:42.971153    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/871778fc-7792-49f6-b9d8-99736df6c541-kube-api-access-2hjbh" (OuterVolumeSpecName: "kube-api-access-2hjbh") pod "871778fc-7792-49f6-b9d8-99736df6c541" (UID: "871778fc-7792-49f6-b9d8-99736df6c541"). InnerVolumeSpecName "kube-api-access-2hjbh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:06:43.069853 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:43.069813    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2hjbh\" (UniqueName: \"kubernetes.io/projected/871778fc-7792-49f6-b9d8-99736df6c541-kube-api-access-2hjbh\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:06:43.069853 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:43.069846    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/871778fc-7792-49f6-b9d8-99736df6c541-plugins-config-volume\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:06:43.069853 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:43.069856    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wt6hz\" (UniqueName: \"kubernetes.io/projected/ace9f826-4ca5-480b-88ba-833ea8391892-kube-api-access-wt6hz\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:06:43.069853 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:43.069865    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ace9f826-4ca5-480b-88ba-833ea8391892-plugins-config-volume\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:06:43.117771 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:43.117731    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:06:43.241291 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:43.241252    2558 generic.go:356] "Generic (PLEG): container finished" podID="871778fc-7792-49f6-b9d8-99736df6c541" containerID="05a2a5e77cafd8f5d7e5a1c7e5804e49e3c3a9d5d76c8c53b46ffba556fabea5" exitCode=0
Jul 27 20:06:43.241534 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:43.241303    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d978f1-54fb5b9f48-lmrss"
Jul 27 20:06:43.241534 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:43.241346    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d978f1-54fb5b9f48-lmrss" event={"ID":"871778fc-7792-49f6-b9d8-99736df6c541","Type":"ContainerDied","Data":"05a2a5e77cafd8f5d7e5a1c7e5804e49e3c3a9d5d76c8c53b46ffba556fabea5"}
Jul 27 20:06:43.241534 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:43.241396    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d978f1-54fb5b9f48-lmrss" event={"ID":"871778fc-7792-49f6-b9d8-99736df6c541","Type":"ContainerDied","Data":"77e5d833d0d5c4ee0446b82ec6290c9b56889c44db90eb01220ee2ae8e4aa6bf"}
Jul 27 20:06:43.241534 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:43.241454    2558 scope.go:117] "RemoveContainer" containerID="05a2a5e77cafd8f5d7e5a1c7e5804e49e3c3a9d5d76c8c53b46ffba556fabea5"
Jul 27 20:06:43.242882 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:43.242858    2558 generic.go:356] "Generic (PLEG): container finished" podID="ace9f826-4ca5-480b-88ba-833ea8391892" containerID="54bfd764a575356f917f7ba08e68fb3ebbd0d637b8feac4c8e35765b61b1a6c5" exitCode=0
Jul 27 20:06:43.243024 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:43.242921    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-d978f1-658d59987b-wwm6q"
Jul 27 20:06:43.243024 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:43.242967    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-d978f1-658d59987b-wwm6q" event={"ID":"ace9f826-4ca5-480b-88ba-833ea8391892","Type":"ContainerDied","Data":"54bfd764a575356f917f7ba08e68fb3ebbd0d637b8feac4c8e35765b61b1a6c5"}
Jul 27 20:06:43.243114 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:43.243031    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-d978f1-658d59987b-wwm6q" event={"ID":"ace9f826-4ca5-480b-88ba-833ea8391892","Type":"ContainerDied","Data":"abd7a3e86d2477b6ad4033e4f6da50cf134c71195200b39fdf47c4c0c38390d3"}
Jul 27 20:06:43.251013 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:43.250993    2558 scope.go:117] "RemoveContainer" containerID="05a2a5e77cafd8f5d7e5a1c7e5804e49e3c3a9d5d76c8c53b46ffba556fabea5"
Jul 27 20:06:43.251355 ip-10-0-129-235 kubenswrapper[2558]: E0727 20:06:43.251335    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05a2a5e77cafd8f5d7e5a1c7e5804e49e3c3a9d5d76c8c53b46ffba556fabea5\": container with ID starting with 05a2a5e77cafd8f5d7e5a1c7e5804e49e3c3a9d5d76c8c53b46ffba556fabea5 not found: ID does not exist" containerID="05a2a5e77cafd8f5d7e5a1c7e5804e49e3c3a9d5d76c8c53b46ffba556fabea5"
Jul 27 20:06:43.251446 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:43.251365    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05a2a5e77cafd8f5d7e5a1c7e5804e49e3c3a9d5d76c8c53b46ffba556fabea5"} err="failed to get container status \"05a2a5e77cafd8f5d7e5a1c7e5804e49e3c3a9d5d76c8c53b46ffba556fabea5\": rpc error: code = NotFound desc = could not find container \"05a2a5e77cafd8f5d7e5a1c7e5804e49e3c3a9d5d76c8c53b46ffba556fabea5\": container with ID starting with 05a2a5e77cafd8f5d7e5a1c7e5804e49e3c3a9d5d76c8c53b46ffba556fabea5 not found: ID does not exist"
Jul 27 20:06:43.251446 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:43.251382    2558 scope.go:117] "RemoveContainer" containerID="54bfd764a575356f917f7ba08e68fb3ebbd0d637b8feac4c8e35765b61b1a6c5"
Jul 27 20:06:43.260478 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:43.260454    2558 scope.go:117] "RemoveContainer" containerID="54bfd764a575356f917f7ba08e68fb3ebbd0d637b8feac4c8e35765b61b1a6c5"
Jul 27 20:06:43.260838 ip-10-0-129-235 kubenswrapper[2558]: E0727 20:06:43.260814    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54bfd764a575356f917f7ba08e68fb3ebbd0d637b8feac4c8e35765b61b1a6c5\": container with ID starting with 54bfd764a575356f917f7ba08e68fb3ebbd0d637b8feac4c8e35765b61b1a6c5 not found: ID does not exist" containerID="54bfd764a575356f917f7ba08e68fb3ebbd0d637b8feac4c8e35765b61b1a6c5"
Jul 27 20:06:43.260937 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:43.260849    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54bfd764a575356f917f7ba08e68fb3ebbd0d637b8feac4c8e35765b61b1a6c5"} err="failed to get container status \"54bfd764a575356f917f7ba08e68fb3ebbd0d637b8feac4c8e35765b61b1a6c5\": rpc error: code = NotFound desc = could not find container \"54bfd764a575356f917f7ba08e68fb3ebbd0d637b8feac4c8e35765b61b1a6c5\": container with ID starting with 54bfd764a575356f917f7ba08e68fb3ebbd0d637b8feac4c8e35765b61b1a6c5 not found: ID does not exist"
Jul 27 20:06:43.268921 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:43.268889    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-d978f1-54fb5b9f48-lmrss"]
Jul 27 20:06:43.272264 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:43.272237    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-d978f1-54fb5b9f48-lmrss"]
Jul 27 20:06:43.282193 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:43.282160    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-d978f1-658d59987b-wwm6q"]
Jul 27 20:06:43.285489 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:43.285455    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-d978f1-658d59987b-wwm6q"]
Jul 27 20:06:43.674899 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:43.674865    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="871778fc-7792-49f6-b9d8-99736df6c541" path="/var/lib/kubelet/pods/871778fc-7792-49f6-b9d8-99736df6c541/volumes"
Jul 27 20:06:43.675280 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:43.675209    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ace9f826-4ca5-480b-88ba-833ea8391892" path="/var/lib/kubelet/pods/ace9f826-4ca5-480b-88ba-833ea8391892/volumes"
Jul 27 20:06:49.275429 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:49.275325    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:06:49.379821 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:49.379793    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:06:49.616155 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:49.616122    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:06:49.827900 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:49.827865    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:06:50.153030 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:50.152988    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:06:50.468224 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:50.468133    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:06:50.600314 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:50.600279    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:06:50.989152 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:50.989117    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:06:53.845068 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:53.845031    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:06:53.932194 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:53.932162    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:06:54.032899 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:54.032867    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:06:54.306562 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:54.306523    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:06:54.519718 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:54.519677    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:06:54.603441 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:54.603345    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:06:54.607834 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:54.607810    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-d3c546-68dbffb748-g5tfw"]
Jul 27 20:06:54.608164 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:54.608153    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c77f739-4e6c-45ed-8a40-bba25cbade0e" containerName="payload-processing"
Jul 27 20:06:54.608208 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:54.608166    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c77f739-4e6c-45ed-8a40-bba25cbade0e" containerName="payload-processing"
Jul 27 20:06:54.608208 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:54.608178    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="871778fc-7792-49f6-b9d8-99736df6c541" containerName="payload-pre-processing"
Jul 27 20:06:54.608208 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:54.608183    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="871778fc-7792-49f6-b9d8-99736df6c541" containerName="payload-pre-processing"
Jul 27 20:06:54.608208 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:54.608193    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5721763e-cb65-4054-bc22-fce240343e64" containerName="payload-pre-processing"
Jul 27 20:06:54.608208 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:54.608198    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="5721763e-cb65-4054-bc22-fce240343e64" containerName="payload-pre-processing"
Jul 27 20:06:54.608441 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:54.608210    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ace9f826-4ca5-480b-88ba-833ea8391892" containerName="payload-processing"
Jul 27 20:06:54.608441 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:54.608215    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="ace9f826-4ca5-480b-88ba-833ea8391892" containerName="payload-processing"
Jul 27 20:06:54.608441 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:54.608277    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="ace9f826-4ca5-480b-88ba-833ea8391892" containerName="payload-processing"
Jul 27 20:06:54.608441 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:54.608285    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c77f739-4e6c-45ed-8a40-bba25cbade0e" containerName="payload-processing"
Jul 27 20:06:54.608441 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:54.608293    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="5721763e-cb65-4054-bc22-fce240343e64" containerName="payload-pre-processing"
Jul 27 20:06:54.608441 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:54.608299    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="871778fc-7792-49f6-b9d8-99736df6c541" containerName="payload-pre-processing"
Jul 27 20:06:54.612913 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:54.612893    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d3c546-68dbffb748-g5tfw"
Jul 27 20:06:54.615456 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:54.615434    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-d3c546\""
Jul 27 20:06:54.615547 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:54.615458    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-d3c546-dockercfg-khpj9\""
Jul 27 20:06:54.621668 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:54.621644    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-d3c546-68dbffb748-g5tfw"]
Jul 27 20:06:54.641405 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:54.641373    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-d3c546-6b6f876dfd-tg88d"]
Jul 27 20:06:54.644808 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:54.644790    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-d3c546-6b6f876dfd-tg88d"
Jul 27 20:06:54.658392 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:54.657570    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-d3c546-6b6f876dfd-tg88d"]
Jul 27 20:06:54.778444 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:54.778371    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/548a6292-1d21-4a62-abb1-67bbc6b93ccf-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-d3c546-6b6f876dfd-tg88d\" (UID: \"548a6292-1d21-4a62-abb1-67bbc6b93ccf\") " pod="openshift-ingress/payload-processing-e2e-shared-a-d3c546-6b6f876dfd-tg88d"
Jul 27 20:06:54.778653 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:54.778462    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxbct\" (UniqueName: \"kubernetes.io/projected/cb09fff4-29c5-4ee5-909d-376451de8715-kube-api-access-rxbct\") pod \"payload-pre-processing-e2e-shared-a-d3c546-68dbffb748-g5tfw\" (UID: \"cb09fff4-29c5-4ee5-909d-376451de8715\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d3c546-68dbffb748-g5tfw"
Jul 27 20:06:54.778653 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:54.778502    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb09fff4-29c5-4ee5-909d-376451de8715-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-d3c546-68dbffb748-g5tfw\" (UID: \"cb09fff4-29c5-4ee5-909d-376451de8715\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d3c546-68dbffb748-g5tfw"
Jul 27 20:06:54.778653 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:54.778562    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-242vk\" (UniqueName: \"kubernetes.io/projected/548a6292-1d21-4a62-abb1-67bbc6b93ccf-kube-api-access-242vk\") pod \"payload-processing-e2e-shared-a-d3c546-6b6f876dfd-tg88d\" (UID: \"548a6292-1d21-4a62-abb1-67bbc6b93ccf\") " pod="openshift-ingress/payload-processing-e2e-shared-a-d3c546-6b6f876dfd-tg88d"
Jul 27 20:06:54.879465 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:54.879431    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/548a6292-1d21-4a62-abb1-67bbc6b93ccf-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-d3c546-6b6f876dfd-tg88d\" (UID: \"548a6292-1d21-4a62-abb1-67bbc6b93ccf\") " pod="openshift-ingress/payload-processing-e2e-shared-a-d3c546-6b6f876dfd-tg88d"
Jul 27 20:06:54.879960 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:54.879476    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxbct\" (UniqueName: \"kubernetes.io/projected/cb09fff4-29c5-4ee5-909d-376451de8715-kube-api-access-rxbct\") pod \"payload-pre-processing-e2e-shared-a-d3c546-68dbffb748-g5tfw\" (UID: \"cb09fff4-29c5-4ee5-909d-376451de8715\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d3c546-68dbffb748-g5tfw"
Jul 27 20:06:54.879960 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:54.879610    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb09fff4-29c5-4ee5-909d-376451de8715-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-d3c546-68dbffb748-g5tfw\" (UID: \"cb09fff4-29c5-4ee5-909d-376451de8715\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d3c546-68dbffb748-g5tfw"
Jul 27 20:06:54.879960 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:54.879675    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-242vk\" (UniqueName: \"kubernetes.io/projected/548a6292-1d21-4a62-abb1-67bbc6b93ccf-kube-api-access-242vk\") pod \"payload-processing-e2e-shared-a-d3c546-6b6f876dfd-tg88d\" (UID: \"548a6292-1d21-4a62-abb1-67bbc6b93ccf\") " pod="openshift-ingress/payload-processing-e2e-shared-a-d3c546-6b6f876dfd-tg88d"
Jul 27 20:06:54.880298 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:54.880275    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/548a6292-1d21-4a62-abb1-67bbc6b93ccf-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-d3c546-6b6f876dfd-tg88d\" (UID: \"548a6292-1d21-4a62-abb1-67bbc6b93ccf\") " pod="openshift-ingress/payload-processing-e2e-shared-a-d3c546-6b6f876dfd-tg88d"
Jul 27 20:06:54.880298 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:54.880289    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb09fff4-29c5-4ee5-909d-376451de8715-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-d3c546-68dbffb748-g5tfw\" (UID: \"cb09fff4-29c5-4ee5-909d-376451de8715\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d3c546-68dbffb748-g5tfw"
Jul 27 20:06:54.887457 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:54.887433    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-242vk\" (UniqueName: \"kubernetes.io/projected/548a6292-1d21-4a62-abb1-67bbc6b93ccf-kube-api-access-242vk\") pod \"payload-processing-e2e-shared-a-d3c546-6b6f876dfd-tg88d\" (UID: \"548a6292-1d21-4a62-abb1-67bbc6b93ccf\") " pod="openshift-ingress/payload-processing-e2e-shared-a-d3c546-6b6f876dfd-tg88d"
Jul 27 20:06:54.887573 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:54.887522    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxbct\" (UniqueName: \"kubernetes.io/projected/cb09fff4-29c5-4ee5-909d-376451de8715-kube-api-access-rxbct\") pod \"payload-pre-processing-e2e-shared-a-d3c546-68dbffb748-g5tfw\" (UID: \"cb09fff4-29c5-4ee5-909d-376451de8715\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d3c546-68dbffb748-g5tfw"
Jul 27 20:06:54.925741 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:54.925700    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d3c546-68dbffb748-g5tfw"
Jul 27 20:06:54.961201 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:54.961175    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-d3c546-6b6f876dfd-tg88d"
Jul 27 20:06:55.072299 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:55.072269    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-d3c546-68dbffb748-g5tfw"]
Jul 27 20:06:55.074541 ip-10-0-129-235 kubenswrapper[2558]: W0727 20:06:55.074509    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb09fff4_29c5_4ee5_909d_376451de8715.slice/crio-ef41533d741a596b1549d0a05b37e481f91cabe07d4578ffa0a7568a3054d512 WatchSource:0}: Error finding container ef41533d741a596b1549d0a05b37e481f91cabe07d4578ffa0a7568a3054d512: Status 404 returned error can't find the container with id ef41533d741a596b1549d0a05b37e481f91cabe07d4578ffa0a7568a3054d512
Jul 27 20:06:55.115618 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:55.115592    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-d3c546-6b6f876dfd-tg88d"]
Jul 27 20:06:55.117524 ip-10-0-129-235 kubenswrapper[2558]: W0727 20:06:55.117501    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod548a6292_1d21_4a62_abb1_67bbc6b93ccf.slice/crio-fe026e5f6d6c15180eb30c098602708fa8cda1fd1d06437716bc31908db39608 WatchSource:0}: Error finding container fe026e5f6d6c15180eb30c098602708fa8cda1fd1d06437716bc31908db39608: Status 404 returned error can't find the container with id fe026e5f6d6c15180eb30c098602708fa8cda1fd1d06437716bc31908db39608
Jul 27 20:06:55.309256 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:55.309220    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-d3c546-6b6f876dfd-tg88d" event={"ID":"548a6292-1d21-4a62-abb1-67bbc6b93ccf","Type":"ContainerStarted","Data":"2fce1cbf562354d9dfce70dd1db4d43dcc8926b1239000978f72d08ce000049d"}
Jul 27 20:06:55.309484 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:55.309277    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-d3c546-6b6f876dfd-tg88d" event={"ID":"548a6292-1d21-4a62-abb1-67bbc6b93ccf","Type":"ContainerStarted","Data":"fe026e5f6d6c15180eb30c098602708fa8cda1fd1d06437716bc31908db39608"}
Jul 27 20:06:55.309484 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:55.309327    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-d3c546-6b6f876dfd-tg88d"
Jul 27 20:06:55.310719 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:55.310696    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d3c546-68dbffb748-g5tfw" event={"ID":"cb09fff4-29c5-4ee5-909d-376451de8715","Type":"ContainerStarted","Data":"3a4c0c433767548e29f9a1fec0546f8cd87a6113c34a9343e15262b3445d9009"}
Jul 27 20:06:55.310841 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:55.310742    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d3c546-68dbffb748-g5tfw"
Jul 27 20:06:55.310841 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:55.310761    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d3c546-68dbffb748-g5tfw" event={"ID":"cb09fff4-29c5-4ee5-909d-376451de8715","Type":"ContainerStarted","Data":"ef41533d741a596b1549d0a05b37e481f91cabe07d4578ffa0a7568a3054d512"}
Jul 27 20:06:55.328145 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:55.328095    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-d3c546-6b6f876dfd-tg88d" podStartSLOduration=1.32808193 podStartE2EDuration="1.32808193s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:06:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:06:55.32457869 +0000 UTC m=+2428.196770665" watchObservedRunningTime="2026-07-27 20:06:55.32808193 +0000 UTC m=+2428.200273903"
Jul 27 20:06:55.340277 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:55.340219    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d3c546-68dbffb748-g5tfw" podStartSLOduration=1.340207045 podStartE2EDuration="1.340207045s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:06:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:06:55.339672578 +0000 UTC m=+2428.211864552" watchObservedRunningTime="2026-07-27 20:06:55.340207045 +0000 UTC m=+2428.212399070"
Jul 27 20:06:58.351448 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:58.351391    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:06:58.690872 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:58.690835    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:06:58.987521 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:58.987440    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:06:59.151644 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:06:59.151607    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:07:01.927066 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:01.927029    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:07:02.341054 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:02.340968    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:07:02.432188 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:02.432155    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:07:02.519071 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:02.519032    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:07:02.800454 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:02.800395    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:07:02.923437 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:02.923388    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:07:03.019942 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:03.019908    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:07:03.024827 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:03.024799    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-73ce16-975d66488-vs24j"]
Jul 27 20:07:03.028847 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:03.028832    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-73ce16-975d66488-vs24j"
Jul 27 20:07:03.031628 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:03.031607    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-73ce16\""
Jul 27 20:07:03.031732 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:03.031654    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-73ce16-dockercfg-nh25s\""
Jul 27 20:07:03.036643 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:03.036619    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-73ce16-975d66488-vs24j"]
Jul 27 20:07:03.052328 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:03.052258    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-73ce16-5979bbc9bb-lcrpb"]
Jul 27 20:07:03.056405 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:03.056383    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-73ce16-5979bbc9bb-lcrpb"
Jul 27 20:07:03.060504 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:03.060437    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2qkr\" (UniqueName: \"kubernetes.io/projected/166f3512-54e0-47ef-af08-ef653572c50c-kube-api-access-l2qkr\") pod \"payload-pre-processing-e2e-shared-b-73ce16-975d66488-vs24j\" (UID: \"166f3512-54e0-47ef-af08-ef653572c50c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-73ce16-975d66488-vs24j"
Jul 27 20:07:03.060504 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:03.060504    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/166f3512-54e0-47ef-af08-ef653572c50c-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-73ce16-975d66488-vs24j\" (UID: \"166f3512-54e0-47ef-af08-ef653572c50c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-73ce16-975d66488-vs24j"
Jul 27 20:07:03.063825 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:03.063798    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-73ce16-5979bbc9bb-lcrpb"]
Jul 27 20:07:03.161742 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:03.161714    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2qkr\" (UniqueName: \"kubernetes.io/projected/166f3512-54e0-47ef-af08-ef653572c50c-kube-api-access-l2qkr\") pod \"payload-pre-processing-e2e-shared-b-73ce16-975d66488-vs24j\" (UID: \"166f3512-54e0-47ef-af08-ef653572c50c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-73ce16-975d66488-vs24j"
Jul 27 20:07:03.161917 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:03.161750    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/166f3512-54e0-47ef-af08-ef653572c50c-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-73ce16-975d66488-vs24j\" (UID: \"166f3512-54e0-47ef-af08-ef653572c50c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-73ce16-975d66488-vs24j"
Jul 27 20:07:03.161917 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:03.161823    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/551bb311-0b2a-429f-8a4d-40d44d3685b8-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-73ce16-5979bbc9bb-lcrpb\" (UID: \"551bb311-0b2a-429f-8a4d-40d44d3685b8\") " pod="openshift-ingress/payload-processing-e2e-shared-b-73ce16-5979bbc9bb-lcrpb"
Jul 27 20:07:03.161917 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:03.161843    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rp2b\" (UniqueName: \"kubernetes.io/projected/551bb311-0b2a-429f-8a4d-40d44d3685b8-kube-api-access-5rp2b\") pod \"payload-processing-e2e-shared-b-73ce16-5979bbc9bb-lcrpb\" (UID: \"551bb311-0b2a-429f-8a4d-40d44d3685b8\") " pod="openshift-ingress/payload-processing-e2e-shared-b-73ce16-5979bbc9bb-lcrpb"
Jul 27 20:07:03.162480 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:03.162458    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/166f3512-54e0-47ef-af08-ef653572c50c-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-73ce16-975d66488-vs24j\" (UID: \"166f3512-54e0-47ef-af08-ef653572c50c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-73ce16-975d66488-vs24j"
Jul 27 20:07:03.169933 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:03.169911    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2qkr\" (UniqueName: \"kubernetes.io/projected/166f3512-54e0-47ef-af08-ef653572c50c-kube-api-access-l2qkr\") pod \"payload-pre-processing-e2e-shared-b-73ce16-975d66488-vs24j\" (UID: \"166f3512-54e0-47ef-af08-ef653572c50c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-73ce16-975d66488-vs24j"
Jul 27 20:07:03.262868 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:03.262824    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/551bb311-0b2a-429f-8a4d-40d44d3685b8-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-73ce16-5979bbc9bb-lcrpb\" (UID: \"551bb311-0b2a-429f-8a4d-40d44d3685b8\") " pod="openshift-ingress/payload-processing-e2e-shared-b-73ce16-5979bbc9bb-lcrpb"
Jul 27 20:07:03.262868 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:03.262869    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rp2b\" (UniqueName: \"kubernetes.io/projected/551bb311-0b2a-429f-8a4d-40d44d3685b8-kube-api-access-5rp2b\") pod \"payload-processing-e2e-shared-b-73ce16-5979bbc9bb-lcrpb\" (UID: \"551bb311-0b2a-429f-8a4d-40d44d3685b8\") " pod="openshift-ingress/payload-processing-e2e-shared-b-73ce16-5979bbc9bb-lcrpb"
Jul 27 20:07:03.263566 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:03.263547    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/551bb311-0b2a-429f-8a4d-40d44d3685b8-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-73ce16-5979bbc9bb-lcrpb\" (UID: \"551bb311-0b2a-429f-8a4d-40d44d3685b8\") " pod="openshift-ingress/payload-processing-e2e-shared-b-73ce16-5979bbc9bb-lcrpb"
Jul 27 20:07:03.271132 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:03.271110    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rp2b\" (UniqueName: \"kubernetes.io/projected/551bb311-0b2a-429f-8a4d-40d44d3685b8-kube-api-access-5rp2b\") pod \"payload-processing-e2e-shared-b-73ce16-5979bbc9bb-lcrpb\" (UID: \"551bb311-0b2a-429f-8a4d-40d44d3685b8\") " pod="openshift-ingress/payload-processing-e2e-shared-b-73ce16-5979bbc9bb-lcrpb"
Jul 27 20:07:03.341803 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:03.341720    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-73ce16-975d66488-vs24j"
Jul 27 20:07:03.367493 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:03.367468    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-73ce16-5979bbc9bb-lcrpb"
Jul 27 20:07:03.494210 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:03.493815    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-73ce16-975d66488-vs24j"]
Jul 27 20:07:03.496096 ip-10-0-129-235 kubenswrapper[2558]: W0727 20:07:03.496069    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod166f3512_54e0_47ef_af08_ef653572c50c.slice/crio-d4a493bd43a19b01f781f3063d8b4802b19ac9c114adb5f0fdc6dc6e09244e00 WatchSource:0}: Error finding container d4a493bd43a19b01f781f3063d8b4802b19ac9c114adb5f0fdc6dc6e09244e00: Status 404 returned error can't find the container with id d4a493bd43a19b01f781f3063d8b4802b19ac9c114adb5f0fdc6dc6e09244e00
Jul 27 20:07:03.522625 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:03.522594    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-73ce16-5979bbc9bb-lcrpb"]
Jul 27 20:07:03.524182 ip-10-0-129-235 kubenswrapper[2558]: W0727 20:07:03.524152    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod551bb311_0b2a_429f_8a4d_40d44d3685b8.slice/crio-1b30e23343223755cbe03f3f744821acfbd20b13c2e6fcdfce01e8bd5369d266 WatchSource:0}: Error finding container 1b30e23343223755cbe03f3f744821acfbd20b13c2e6fcdfce01e8bd5369d266: Status 404 returned error can't find the container with id 1b30e23343223755cbe03f3f744821acfbd20b13c2e6fcdfce01e8bd5369d266
Jul 27 20:07:04.369192 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:04.369142    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-73ce16-5979bbc9bb-lcrpb" event={"ID":"551bb311-0b2a-429f-8a4d-40d44d3685b8","Type":"ContainerStarted","Data":"3a6387a7b524438fe5cadfc3afcd2483af0467ea7a691be29225d3fc7224aa35"}
Jul 27 20:07:04.369681 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:04.369211    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-73ce16-5979bbc9bb-lcrpb"
Jul 27 20:07:04.369681 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:04.369239    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-73ce16-5979bbc9bb-lcrpb" event={"ID":"551bb311-0b2a-429f-8a4d-40d44d3685b8","Type":"ContainerStarted","Data":"1b30e23343223755cbe03f3f744821acfbd20b13c2e6fcdfce01e8bd5369d266"}
Jul 27 20:07:04.370754 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:04.370730    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-73ce16-975d66488-vs24j" event={"ID":"166f3512-54e0-47ef-af08-ef653572c50c","Type":"ContainerStarted","Data":"4d01d169120929633ce52f725026715fbded86f331ed577129827260448a459c"}
Jul 27 20:07:04.370842 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:04.370779    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-73ce16-975d66488-vs24j" event={"ID":"166f3512-54e0-47ef-af08-ef653572c50c","Type":"ContainerStarted","Data":"d4a493bd43a19b01f781f3063d8b4802b19ac9c114adb5f0fdc6dc6e09244e00"}
Jul 27 20:07:04.370842 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:04.370802    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-73ce16-975d66488-vs24j"
Jul 27 20:07:04.384567 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:04.384517    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-73ce16-5979bbc9bb-lcrpb" podStartSLOduration=1.384497555 podStartE2EDuration="1.384497555s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:07:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:07:04.384100921 +0000 UTC m=+2437.256292896" watchObservedRunningTime="2026-07-27 20:07:04.384497555 +0000 UTC m=+2437.256689510"
Jul 27 20:07:04.400923 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:04.400876    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-73ce16-975d66488-vs24j" podStartSLOduration=1.400863173 podStartE2EDuration="1.400863173s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:07:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:07:04.398674323 +0000 UTC m=+2437.270866325" watchObservedRunningTime="2026-07-27 20:07:04.400863173 +0000 UTC m=+2437.273055147"
Jul 27 20:07:06.318368 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:06.318331    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-d3c546-6b6f876dfd-tg88d"
Jul 27 20:07:06.318825 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:06.318401    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d3c546-68dbffb748-g5tfw"
Jul 27 20:07:06.831642 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:06.831604    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:07:15.378619 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:15.378580    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-73ce16-5979bbc9bb-lcrpb"
Jul 27 20:07:15.379046 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:15.378667    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-73ce16-975d66488-vs24j"
Jul 27 20:07:18.720183 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:18.720148    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-d3c546/auth-test-model-d3c546-kserve-5b96cfbb79-p4p9f"]
Jul 27 20:07:18.723969 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:18.723949    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d3c546/auth-test-model-d3c546-kserve-5b96cfbb79-p4p9f"
Jul 27 20:07:18.726393 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:18.726373    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-d3c546\"/\"openshift-service-ca.crt\""
Jul 27 20:07:18.726539 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:18.726375    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-d3c546\"/\"kube-root-ca.crt\""
Jul 27 20:07:18.727607 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:18.727583    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-d3c546\"/\"auth-test-model-d3c546-kserve-self-signed-certs\""
Jul 27 20:07:18.727737 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:18.727595    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-d3c546\"/\"default-dockercfg-hnjj2\""
Jul 27 20:07:18.734765 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:18.732834    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-d3c546/auth-test-model-d3c546-kserve-5b96cfbb79-p4p9f"]
Jul 27 20:07:18.750147 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:18.750117    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:07:18.811429 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:18.811375    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea744b6c-4042-497c-a296-a378d05fbd52-tls-certs\") pod \"auth-test-model-d3c546-kserve-5b96cfbb79-p4p9f\" (UID: \"ea744b6c-4042-497c-a296-a378d05fbd52\") " pod="ai-tenant-e2e-shared-a-d3c546/auth-test-model-d3c546-kserve-5b96cfbb79-p4p9f"
Jul 27 20:07:18.811585 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:18.811432    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea744b6c-4042-497c-a296-a378d05fbd52-dshm\") pod \"auth-test-model-d3c546-kserve-5b96cfbb79-p4p9f\" (UID: \"ea744b6c-4042-497c-a296-a378d05fbd52\") " pod="ai-tenant-e2e-shared-a-d3c546/auth-test-model-d3c546-kserve-5b96cfbb79-p4p9f"
Jul 27 20:07:18.811585 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:18.811533    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea744b6c-4042-497c-a296-a378d05fbd52-tmp-dir\") pod \"auth-test-model-d3c546-kserve-5b96cfbb79-p4p9f\" (UID: \"ea744b6c-4042-497c-a296-a378d05fbd52\") " pod="ai-tenant-e2e-shared-a-d3c546/auth-test-model-d3c546-kserve-5b96cfbb79-p4p9f"
Jul 27 20:07:18.811585 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:18.811562    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea744b6c-4042-497c-a296-a378d05fbd52-home\") pod \"auth-test-model-d3c546-kserve-5b96cfbb79-p4p9f\" (UID: \"ea744b6c-4042-497c-a296-a378d05fbd52\") " pod="ai-tenant-e2e-shared-a-d3c546/auth-test-model-d3c546-kserve-5b96cfbb79-p4p9f"
Jul 27 20:07:18.811585 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:18.811582    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea744b6c-4042-497c-a296-a378d05fbd52-model-cache\") pod \"auth-test-model-d3c546-kserve-5b96cfbb79-p4p9f\" (UID: \"ea744b6c-4042-497c-a296-a378d05fbd52\") " pod="ai-tenant-e2e-shared-a-d3c546/auth-test-model-d3c546-kserve-5b96cfbb79-p4p9f"
Jul 27 20:07:18.811822 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:18.811614    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfrlr\" (UniqueName: \"kubernetes.io/projected/ea744b6c-4042-497c-a296-a378d05fbd52-kube-api-access-hfrlr\") pod \"auth-test-model-d3c546-kserve-5b96cfbb79-p4p9f\" (UID: \"ea744b6c-4042-497c-a296-a378d05fbd52\") " pod="ai-tenant-e2e-shared-a-d3c546/auth-test-model-d3c546-kserve-5b96cfbb79-p4p9f"
Jul 27 20:07:18.912270 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:18.912242    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea744b6c-4042-497c-a296-a378d05fbd52-tmp-dir\") pod \"auth-test-model-d3c546-kserve-5b96cfbb79-p4p9f\" (UID: \"ea744b6c-4042-497c-a296-a378d05fbd52\") " pod="ai-tenant-e2e-shared-a-d3c546/auth-test-model-d3c546-kserve-5b96cfbb79-p4p9f"
Jul 27 20:07:18.912480 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:18.912276    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea744b6c-4042-497c-a296-a378d05fbd52-home\") pod \"auth-test-model-d3c546-kserve-5b96cfbb79-p4p9f\" (UID: \"ea744b6c-4042-497c-a296-a378d05fbd52\") " pod="ai-tenant-e2e-shared-a-d3c546/auth-test-model-d3c546-kserve-5b96cfbb79-p4p9f"
Jul 27 20:07:18.912480 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:18.912296    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea744b6c-4042-497c-a296-a378d05fbd52-model-cache\") pod \"auth-test-model-d3c546-kserve-5b96cfbb79-p4p9f\" (UID: \"ea744b6c-4042-497c-a296-a378d05fbd52\") " pod="ai-tenant-e2e-shared-a-d3c546/auth-test-model-d3c546-kserve-5b96cfbb79-p4p9f"
Jul 27 20:07:18.912480 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:18.912318    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfrlr\" (UniqueName: \"kubernetes.io/projected/ea744b6c-4042-497c-a296-a378d05fbd52-kube-api-access-hfrlr\") pod \"auth-test-model-d3c546-kserve-5b96cfbb79-p4p9f\" (UID: \"ea744b6c-4042-497c-a296-a378d05fbd52\") " pod="ai-tenant-e2e-shared-a-d3c546/auth-test-model-d3c546-kserve-5b96cfbb79-p4p9f"
Jul 27 20:07:18.912480 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:18.912435    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea744b6c-4042-497c-a296-a378d05fbd52-tls-certs\") pod \"auth-test-model-d3c546-kserve-5b96cfbb79-p4p9f\" (UID: \"ea744b6c-4042-497c-a296-a378d05fbd52\") " pod="ai-tenant-e2e-shared-a-d3c546/auth-test-model-d3c546-kserve-5b96cfbb79-p4p9f"
Jul 27 20:07:18.912480 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:18.912459    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea744b6c-4042-497c-a296-a378d05fbd52-dshm\") pod \"auth-test-model-d3c546-kserve-5b96cfbb79-p4p9f\" (UID: \"ea744b6c-4042-497c-a296-a378d05fbd52\") " pod="ai-tenant-e2e-shared-a-d3c546/auth-test-model-d3c546-kserve-5b96cfbb79-p4p9f"
Jul 27 20:07:18.912758 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:18.912707    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea744b6c-4042-497c-a296-a378d05fbd52-tmp-dir\") pod \"auth-test-model-d3c546-kserve-5b96cfbb79-p4p9f\" (UID: \"ea744b6c-4042-497c-a296-a378d05fbd52\") " pod="ai-tenant-e2e-shared-a-d3c546/auth-test-model-d3c546-kserve-5b96cfbb79-p4p9f"
Jul 27 20:07:18.912813 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:18.912770    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea744b6c-4042-497c-a296-a378d05fbd52-model-cache\") pod \"auth-test-model-d3c546-kserve-5b96cfbb79-p4p9f\" (UID: \"ea744b6c-4042-497c-a296-a378d05fbd52\") " pod="ai-tenant-e2e-shared-a-d3c546/auth-test-model-d3c546-kserve-5b96cfbb79-p4p9f"
Jul 27 20:07:18.912870 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:18.912809    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea744b6c-4042-497c-a296-a378d05fbd52-home\") pod \"auth-test-model-d3c546-kserve-5b96cfbb79-p4p9f\" (UID: \"ea744b6c-4042-497c-a296-a378d05fbd52\") " pod="ai-tenant-e2e-shared-a-d3c546/auth-test-model-d3c546-kserve-5b96cfbb79-p4p9f"
Jul 27 20:07:18.914885 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:18.914861    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea744b6c-4042-497c-a296-a378d05fbd52-dshm\") pod \"auth-test-model-d3c546-kserve-5b96cfbb79-p4p9f\" (UID: \"ea744b6c-4042-497c-a296-a378d05fbd52\") " pod="ai-tenant-e2e-shared-a-d3c546/auth-test-model-d3c546-kserve-5b96cfbb79-p4p9f"
Jul 27 20:07:18.915303 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:18.915284    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea744b6c-4042-497c-a296-a378d05fbd52-tls-certs\") pod \"auth-test-model-d3c546-kserve-5b96cfbb79-p4p9f\" (UID: \"ea744b6c-4042-497c-a296-a378d05fbd52\") " pod="ai-tenant-e2e-shared-a-d3c546/auth-test-model-d3c546-kserve-5b96cfbb79-p4p9f"
Jul 27 20:07:18.920340 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:18.920321    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfrlr\" (UniqueName: \"kubernetes.io/projected/ea744b6c-4042-497c-a296-a378d05fbd52-kube-api-access-hfrlr\") pod \"auth-test-model-d3c546-kserve-5b96cfbb79-p4p9f\" (UID: \"ea744b6c-4042-497c-a296-a378d05fbd52\") " pod="ai-tenant-e2e-shared-a-d3c546/auth-test-model-d3c546-kserve-5b96cfbb79-p4p9f"
Jul 27 20:07:19.037044 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:19.036958    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d3c546/auth-test-model-d3c546-kserve-5b96cfbb79-p4p9f"
Jul 27 20:07:19.175708 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:19.175667    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-d3c546/auth-test-model-d3c546-kserve-5b96cfbb79-p4p9f"]
Jul 27 20:07:19.176983 ip-10-0-129-235 kubenswrapper[2558]: W0727 20:07:19.176947    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podea744b6c_4042_497c_a296_a378d05fbd52.slice/crio-d28573c9db881efe6f3f664f53e599c370a55bcd7ee4a084a7e74e569a65ac86 WatchSource:0}: Error finding container d28573c9db881efe6f3f664f53e599c370a55bcd7ee4a084a7e74e569a65ac86: Status 404 returned error can't find the container with id d28573c9db881efe6f3f664f53e599c370a55bcd7ee4a084a7e74e569a65ac86
Jul 27 20:07:19.458928 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:19.458890    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d3c546/auth-test-model-d3c546-kserve-5b96cfbb79-p4p9f" event={"ID":"ea744b6c-4042-497c-a296-a378d05fbd52","Type":"ContainerStarted","Data":"fc8c42e7d64e53f8b85f1805ffcb2651103cb5c44919caf547663083dcf7cba6"}
Jul 27 20:07:19.458928 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:19.458936    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d3c546/auth-test-model-d3c546-kserve-5b96cfbb79-p4p9f" event={"ID":"ea744b6c-4042-497c-a296-a378d05fbd52","Type":"ContainerStarted","Data":"d28573c9db881efe6f3f664f53e599c370a55bcd7ee4a084a7e74e569a65ac86"}
Jul 27 20:07:19.478590 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:19.478547    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-d3c546/auth-test-model-d3c546-kserve-5b96cfbb79-p4p9f" podStartSLOduration=1.4785317789999999 podStartE2EDuration="1.478531779s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:07:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:07:19.476463286 +0000 UTC m=+2452.348655261" watchObservedRunningTime="2026-07-27 20:07:19.478531779 +0000 UTC m=+2452.350723753"
Jul 27 20:07:27.958999 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:27.958970    2558 scope.go:117] "RemoveContainer" containerID="98b42b1167fc718a0663226dc040df3ed89ad863185f74930475cf7fb3dfeae7"
Jul 27 20:07:29.037966 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:29.037928    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-d3c546/auth-test-model-d3c546-kserve-5b96cfbb79-p4p9f"
Jul 27 20:07:29.038357 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:29.037982    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-d3c546/auth-test-model-d3c546-kserve-5b96cfbb79-p4p9f"
Jul 27 20:07:29.050620 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:29.050593    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-d3c546/auth-test-model-d3c546-kserve-5b96cfbb79-p4p9f"
Jul 27 20:07:29.528109 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:29.528076    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-d3c546/auth-test-model-d3c546-kserve-5b96cfbb79-p4p9f"
Jul 27 20:07:39.724945 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:39.724907    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-73ce16/auth-test-model-73ce16-kserve-7dd9748687-dgjkh"]
Jul 27 20:07:39.728913 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:39.728888    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-73ce16/auth-test-model-73ce16-kserve-7dd9748687-dgjkh"
Jul 27 20:07:39.731406 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:39.731180    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-73ce16\"/\"openshift-service-ca.crt\""
Jul 27 20:07:39.731406 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:39.731321    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-73ce16\"/\"kube-root-ca.crt\""
Jul 27 20:07:39.732156 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:39.732135    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-73ce16\"/\"auth-test-model-73ce16-kserve-self-signed-certs\""
Jul 27 20:07:39.732318 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:39.732296    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-73ce16\"/\"default-dockercfg-2jg5m\""
Jul 27 20:07:39.739819 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:39.739792    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-73ce16/auth-test-model-73ce16-kserve-7dd9748687-dgjkh"]
Jul 27 20:07:39.762157 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:39.762126    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:07:39.806709 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:39.806678    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3181b849-aed2-407a-9b4a-0ca25c78dd00-model-cache\") pod \"auth-test-model-73ce16-kserve-7dd9748687-dgjkh\" (UID: \"3181b849-aed2-407a-9b4a-0ca25c78dd00\") " pod="ai-tenant-e2e-shared-b-73ce16/auth-test-model-73ce16-kserve-7dd9748687-dgjkh"
Jul 27 20:07:39.806709 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:39.806707    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3181b849-aed2-407a-9b4a-0ca25c78dd00-tls-certs\") pod \"auth-test-model-73ce16-kserve-7dd9748687-dgjkh\" (UID: \"3181b849-aed2-407a-9b4a-0ca25c78dd00\") " pod="ai-tenant-e2e-shared-b-73ce16/auth-test-model-73ce16-kserve-7dd9748687-dgjkh"
Jul 27 20:07:39.806915 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:39.806727    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3181b849-aed2-407a-9b4a-0ca25c78dd00-home\") pod \"auth-test-model-73ce16-kserve-7dd9748687-dgjkh\" (UID: \"3181b849-aed2-407a-9b4a-0ca25c78dd00\") " pod="ai-tenant-e2e-shared-b-73ce16/auth-test-model-73ce16-kserve-7dd9748687-dgjkh"
Jul 27 20:07:39.806915 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:39.806843    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3181b849-aed2-407a-9b4a-0ca25c78dd00-dshm\") pod \"auth-test-model-73ce16-kserve-7dd9748687-dgjkh\" (UID: \"3181b849-aed2-407a-9b4a-0ca25c78dd00\") " pod="ai-tenant-e2e-shared-b-73ce16/auth-test-model-73ce16-kserve-7dd9748687-dgjkh"
Jul 27 20:07:39.806915 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:39.806877    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3181b849-aed2-407a-9b4a-0ca25c78dd00-tmp-dir\") pod \"auth-test-model-73ce16-kserve-7dd9748687-dgjkh\" (UID: \"3181b849-aed2-407a-9b4a-0ca25c78dd00\") " pod="ai-tenant-e2e-shared-b-73ce16/auth-test-model-73ce16-kserve-7dd9748687-dgjkh"
Jul 27 20:07:39.807032 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:39.806935    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8cxjl\" (UniqueName: \"kubernetes.io/projected/3181b849-aed2-407a-9b4a-0ca25c78dd00-kube-api-access-8cxjl\") pod \"auth-test-model-73ce16-kserve-7dd9748687-dgjkh\" (UID: \"3181b849-aed2-407a-9b4a-0ca25c78dd00\") " pod="ai-tenant-e2e-shared-b-73ce16/auth-test-model-73ce16-kserve-7dd9748687-dgjkh"
Jul 27 20:07:39.907359 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:39.907319    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3181b849-aed2-407a-9b4a-0ca25c78dd00-dshm\") pod \"auth-test-model-73ce16-kserve-7dd9748687-dgjkh\" (UID: \"3181b849-aed2-407a-9b4a-0ca25c78dd00\") " pod="ai-tenant-e2e-shared-b-73ce16/auth-test-model-73ce16-kserve-7dd9748687-dgjkh"
Jul 27 20:07:39.907359 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:39.907363    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3181b849-aed2-407a-9b4a-0ca25c78dd00-tmp-dir\") pod \"auth-test-model-73ce16-kserve-7dd9748687-dgjkh\" (UID: \"3181b849-aed2-407a-9b4a-0ca25c78dd00\") " pod="ai-tenant-e2e-shared-b-73ce16/auth-test-model-73ce16-kserve-7dd9748687-dgjkh"
Jul 27 20:07:39.907645 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:39.907395    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cxjl\" (UniqueName: \"kubernetes.io/projected/3181b849-aed2-407a-9b4a-0ca25c78dd00-kube-api-access-8cxjl\") pod \"auth-test-model-73ce16-kserve-7dd9748687-dgjkh\" (UID: \"3181b849-aed2-407a-9b4a-0ca25c78dd00\") " pod="ai-tenant-e2e-shared-b-73ce16/auth-test-model-73ce16-kserve-7dd9748687-dgjkh"
Jul 27 20:07:39.907645 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:39.907464    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3181b849-aed2-407a-9b4a-0ca25c78dd00-model-cache\") pod \"auth-test-model-73ce16-kserve-7dd9748687-dgjkh\" (UID: \"3181b849-aed2-407a-9b4a-0ca25c78dd00\") " pod="ai-tenant-e2e-shared-b-73ce16/auth-test-model-73ce16-kserve-7dd9748687-dgjkh"
Jul 27 20:07:39.907645 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:39.907480    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3181b849-aed2-407a-9b4a-0ca25c78dd00-tls-certs\") pod \"auth-test-model-73ce16-kserve-7dd9748687-dgjkh\" (UID: \"3181b849-aed2-407a-9b4a-0ca25c78dd00\") " pod="ai-tenant-e2e-shared-b-73ce16/auth-test-model-73ce16-kserve-7dd9748687-dgjkh"
Jul 27 20:07:39.907645 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:39.907501    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3181b849-aed2-407a-9b4a-0ca25c78dd00-home\") pod \"auth-test-model-73ce16-kserve-7dd9748687-dgjkh\" (UID: \"3181b849-aed2-407a-9b4a-0ca25c78dd00\") " pod="ai-tenant-e2e-shared-b-73ce16/auth-test-model-73ce16-kserve-7dd9748687-dgjkh"
Jul 27 20:07:39.907877 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:39.907839    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3181b849-aed2-407a-9b4a-0ca25c78dd00-tmp-dir\") pod \"auth-test-model-73ce16-kserve-7dd9748687-dgjkh\" (UID: \"3181b849-aed2-407a-9b4a-0ca25c78dd00\") " pod="ai-tenant-e2e-shared-b-73ce16/auth-test-model-73ce16-kserve-7dd9748687-dgjkh"
Jul 27 20:07:39.907994 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:39.907881    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3181b849-aed2-407a-9b4a-0ca25c78dd00-model-cache\") pod \"auth-test-model-73ce16-kserve-7dd9748687-dgjkh\" (UID: \"3181b849-aed2-407a-9b4a-0ca25c78dd00\") " pod="ai-tenant-e2e-shared-b-73ce16/auth-test-model-73ce16-kserve-7dd9748687-dgjkh"
Jul 27 20:07:39.907994 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:39.907935    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3181b849-aed2-407a-9b4a-0ca25c78dd00-home\") pod \"auth-test-model-73ce16-kserve-7dd9748687-dgjkh\" (UID: \"3181b849-aed2-407a-9b4a-0ca25c78dd00\") " pod="ai-tenant-e2e-shared-b-73ce16/auth-test-model-73ce16-kserve-7dd9748687-dgjkh"
Jul 27 20:07:39.909928 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:39.909895    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3181b849-aed2-407a-9b4a-0ca25c78dd00-dshm\") pod \"auth-test-model-73ce16-kserve-7dd9748687-dgjkh\" (UID: \"3181b849-aed2-407a-9b4a-0ca25c78dd00\") " pod="ai-tenant-e2e-shared-b-73ce16/auth-test-model-73ce16-kserve-7dd9748687-dgjkh"
Jul 27 20:07:39.910066 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:39.910047    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3181b849-aed2-407a-9b4a-0ca25c78dd00-tls-certs\") pod \"auth-test-model-73ce16-kserve-7dd9748687-dgjkh\" (UID: \"3181b849-aed2-407a-9b4a-0ca25c78dd00\") " pod="ai-tenant-e2e-shared-b-73ce16/auth-test-model-73ce16-kserve-7dd9748687-dgjkh"
Jul 27 20:07:39.915052 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:39.915034    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8cxjl\" (UniqueName: \"kubernetes.io/projected/3181b849-aed2-407a-9b4a-0ca25c78dd00-kube-api-access-8cxjl\") pod \"auth-test-model-73ce16-kserve-7dd9748687-dgjkh\" (UID: \"3181b849-aed2-407a-9b4a-0ca25c78dd00\") " pod="ai-tenant-e2e-shared-b-73ce16/auth-test-model-73ce16-kserve-7dd9748687-dgjkh"
Jul 27 20:07:40.040920 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:40.040836    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-73ce16/auth-test-model-73ce16-kserve-7dd9748687-dgjkh"
Jul 27 20:07:40.176462 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:40.176434    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-73ce16/auth-test-model-73ce16-kserve-7dd9748687-dgjkh"]
Jul 27 20:07:40.178724 ip-10-0-129-235 kubenswrapper[2558]: W0727 20:07:40.178695    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3181b849_aed2_407a_9b4a_0ca25c78dd00.slice/crio-e71dc1db24a2d582ecc02a373336e9d793c35592d91f0b7c4163d20ea5097338 WatchSource:0}: Error finding container e71dc1db24a2d582ecc02a373336e9d793c35592d91f0b7c4163d20ea5097338: Status 404 returned error can't find the container with id e71dc1db24a2d582ecc02a373336e9d793c35592d91f0b7c4163d20ea5097338
Jul 27 20:07:40.580358 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:40.580317    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-73ce16/auth-test-model-73ce16-kserve-7dd9748687-dgjkh" event={"ID":"3181b849-aed2-407a-9b4a-0ca25c78dd00","Type":"ContainerStarted","Data":"83d6a6c09ecd2e39dbae39531f3cc4aaf8cde59e8ab00b0442f048e0cbd0bfa1"}
Jul 27 20:07:40.580552 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:40.580370    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-73ce16/auth-test-model-73ce16-kserve-7dd9748687-dgjkh" event={"ID":"3181b849-aed2-407a-9b4a-0ca25c78dd00","Type":"ContainerStarted","Data":"e71dc1db24a2d582ecc02a373336e9d793c35592d91f0b7c4163d20ea5097338"}
Jul 27 20:07:40.597379 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:40.597333    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-73ce16/auth-test-model-73ce16-kserve-7dd9748687-dgjkh" podStartSLOduration=1.59731344 podStartE2EDuration="1.59731344s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:07:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:07:40.596165858 +0000 UTC m=+2473.468357833" watchObservedRunningTime="2026-07-27 20:07:40.59731344 +0000 UTC m=+2473.469505414"
Jul 27 20:07:50.041395 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:50.041356    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-73ce16/auth-test-model-73ce16-kserve-7dd9748687-dgjkh"
Jul 27 20:07:50.041395 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:50.041399    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-73ce16/auth-test-model-73ce16-kserve-7dd9748687-dgjkh"
Jul 27 20:07:50.054254 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:50.054227    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-73ce16/auth-test-model-73ce16-kserve-7dd9748687-dgjkh"
Jul 27 20:07:50.647449 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:07:50.647398    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-73ce16/auth-test-model-73ce16-kserve-7dd9748687-dgjkh"
Jul 27 20:09:07.458749 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:09:07.458715    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:09:07.556772 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:09:07.556736    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:10:20.939522 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:10:20.939475    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:11:27.802566 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:11:27.802489    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5dfc2_103bab23-bca3-472a-974b-7731f83c5bfb/ovn-acl-logging/0.log"
Jul 27 20:11:27.812959 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:11:27.812930    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5dfc2_103bab23-bca3-472a-974b-7731f83c5bfb/ovn-acl-logging/0.log"
Jul 27 20:11:55.219119 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:11:55.219076    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-d3c546/auth-test-model-d3c546-kserve-5b96cfbb79-p4p9f"]
Jul 27 20:11:55.279262 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:11:55.279235    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:11:56.040876 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:11:56.040841    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-73ce16/auth-test-model-73ce16-kserve-7dd9748687-dgjkh"]
Jul 27 20:11:56.064761 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:11:56.064734    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:12:08.949137 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:08.949095    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:12:10.242093 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:10.242054    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-d3c546/auth-test-model-d3c546-kserve-5b96cfbb79-p4p9f" podUID="ea744b6c-4042-497c-a296-a378d05fbd52" containerName="main" containerID="cri-o://fc8c42e7d64e53f8b85f1805ffcb2651103cb5c44919caf547663083dcf7cba6" gracePeriod=45
Jul 27 20:12:10.490084 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:10.490060    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d3c546/auth-test-model-d3c546-kserve-5b96cfbb79-p4p9f"
Jul 27 20:12:10.579328 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:10.579247    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea744b6c-4042-497c-a296-a378d05fbd52-tmp-dir\") pod \"ea744b6c-4042-497c-a296-a378d05fbd52\" (UID: \"ea744b6c-4042-497c-a296-a378d05fbd52\") "
Jul 27 20:12:10.579328 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:10.579292    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea744b6c-4042-497c-a296-a378d05fbd52-home\") pod \"ea744b6c-4042-497c-a296-a378d05fbd52\" (UID: \"ea744b6c-4042-497c-a296-a378d05fbd52\") "
Jul 27 20:12:10.579328 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:10.579321    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea744b6c-4042-497c-a296-a378d05fbd52-tls-certs\") pod \"ea744b6c-4042-497c-a296-a378d05fbd52\" (UID: \"ea744b6c-4042-497c-a296-a378d05fbd52\") "
Jul 27 20:12:10.579652 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:10.579344    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea744b6c-4042-497c-a296-a378d05fbd52-dshm\") pod \"ea744b6c-4042-497c-a296-a378d05fbd52\" (UID: \"ea744b6c-4042-497c-a296-a378d05fbd52\") "
Jul 27 20:12:10.579652 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:10.579380    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hfrlr\" (UniqueName: \"kubernetes.io/projected/ea744b6c-4042-497c-a296-a378d05fbd52-kube-api-access-hfrlr\") pod \"ea744b6c-4042-497c-a296-a378d05fbd52\" (UID: \"ea744b6c-4042-497c-a296-a378d05fbd52\") "
Jul 27 20:12:10.579652 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:10.579437    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea744b6c-4042-497c-a296-a378d05fbd52-model-cache\") pod \"ea744b6c-4042-497c-a296-a378d05fbd52\" (UID: \"ea744b6c-4042-497c-a296-a378d05fbd52\") "
Jul 27 20:12:10.579652 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:10.579531    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea744b6c-4042-497c-a296-a378d05fbd52-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ea744b6c-4042-497c-a296-a378d05fbd52" (UID: "ea744b6c-4042-497c-a296-a378d05fbd52"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:12:10.579652 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:10.579570    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea744b6c-4042-497c-a296-a378d05fbd52-home" (OuterVolumeSpecName: "home") pod "ea744b6c-4042-497c-a296-a378d05fbd52" (UID: "ea744b6c-4042-497c-a296-a378d05fbd52"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:12:10.579894 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:10.579783    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea744b6c-4042-497c-a296-a378d05fbd52-model-cache" (OuterVolumeSpecName: "model-cache") pod "ea744b6c-4042-497c-a296-a378d05fbd52" (UID: "ea744b6c-4042-497c-a296-a378d05fbd52"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:12:10.579894 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:10.579841    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea744b6c-4042-497c-a296-a378d05fbd52-tmp-dir\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:12:10.579894 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:10.579863    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea744b6c-4042-497c-a296-a378d05fbd52-home\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:12:10.581834 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:10.581812    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea744b6c-4042-497c-a296-a378d05fbd52-kube-api-access-hfrlr" (OuterVolumeSpecName: "kube-api-access-hfrlr") pod "ea744b6c-4042-497c-a296-a378d05fbd52" (UID: "ea744b6c-4042-497c-a296-a378d05fbd52"). InnerVolumeSpecName "kube-api-access-hfrlr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:12:10.581924 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:10.581831    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea744b6c-4042-497c-a296-a378d05fbd52-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ea744b6c-4042-497c-a296-a378d05fbd52" (UID: "ea744b6c-4042-497c-a296-a378d05fbd52"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 20:12:10.581924 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:10.581856    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea744b6c-4042-497c-a296-a378d05fbd52-dshm" (OuterVolumeSpecName: "dshm") pod "ea744b6c-4042-497c-a296-a378d05fbd52" (UID: "ea744b6c-4042-497c-a296-a378d05fbd52"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:12:10.634353 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:10.634316    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-d3c546/sub-test-model-d3c546-kserve-56d8f47565-c7wsz"]
Jul 27 20:12:10.634931 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:10.634914    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea744b6c-4042-497c-a296-a378d05fbd52" containerName="main"
Jul 27 20:12:10.634980 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:10.634935    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea744b6c-4042-497c-a296-a378d05fbd52" containerName="main"
Jul 27 20:12:10.635066 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:10.635054    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea744b6c-4042-497c-a296-a378d05fbd52" containerName="main"
Jul 27 20:12:10.638577 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:10.638557    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d3c546/sub-test-model-d3c546-kserve-56d8f47565-c7wsz"
Jul 27 20:12:10.640937 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:10.640919    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-d3c546\"/\"sub-test-model-d3c546-kserve-self-signed-certs\""
Jul 27 20:12:10.645327 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:10.645305    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-d3c546/sub-test-model-d3c546-kserve-56d8f47565-c7wsz"]
Jul 27 20:12:10.680856 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:10.680823    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea744b6c-4042-497c-a296-a378d05fbd52-tls-certs\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:12:10.680856 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:10.680853    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea744b6c-4042-497c-a296-a378d05fbd52-dshm\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:12:10.681014 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:10.680863    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hfrlr\" (UniqueName: \"kubernetes.io/projected/ea744b6c-4042-497c-a296-a378d05fbd52-kube-api-access-hfrlr\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:12:10.681014 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:10.680872    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea744b6c-4042-497c-a296-a378d05fbd52-model-cache\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:12:10.727343 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:10.727318    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:12:10.782356 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:10.782313    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/295219a7-889c-425c-bf8c-d6332ad25d93-home\") pod \"sub-test-model-d3c546-kserve-56d8f47565-c7wsz\" (UID: \"295219a7-889c-425c-bf8c-d6332ad25d93\") " pod="ai-tenant-e2e-shared-a-d3c546/sub-test-model-d3c546-kserve-56d8f47565-c7wsz"
Jul 27 20:12:10.782356 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:10.782358    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/295219a7-889c-425c-bf8c-d6332ad25d93-tmp-dir\") pod \"sub-test-model-d3c546-kserve-56d8f47565-c7wsz\" (UID: \"295219a7-889c-425c-bf8c-d6332ad25d93\") " pod="ai-tenant-e2e-shared-a-d3c546/sub-test-model-d3c546-kserve-56d8f47565-c7wsz"
Jul 27 20:12:10.782600 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:10.782490    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/295219a7-889c-425c-bf8c-d6332ad25d93-dshm\") pod \"sub-test-model-d3c546-kserve-56d8f47565-c7wsz\" (UID: \"295219a7-889c-425c-bf8c-d6332ad25d93\") " pod="ai-tenant-e2e-shared-a-d3c546/sub-test-model-d3c546-kserve-56d8f47565-c7wsz"
Jul 27 20:12:10.782600 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:10.782516    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhrnc\" (UniqueName: \"kubernetes.io/projected/295219a7-889c-425c-bf8c-d6332ad25d93-kube-api-access-bhrnc\") pod \"sub-test-model-d3c546-kserve-56d8f47565-c7wsz\" (UID: \"295219a7-889c-425c-bf8c-d6332ad25d93\") " pod="ai-tenant-e2e-shared-a-d3c546/sub-test-model-d3c546-kserve-56d8f47565-c7wsz"
Jul 27 20:12:10.782687 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:10.782604    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/295219a7-889c-425c-bf8c-d6332ad25d93-model-cache\") pod \"sub-test-model-d3c546-kserve-56d8f47565-c7wsz\" (UID: \"295219a7-889c-425c-bf8c-d6332ad25d93\") " pod="ai-tenant-e2e-shared-a-d3c546/sub-test-model-d3c546-kserve-56d8f47565-c7wsz"
Jul 27 20:12:10.782687 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:10.782627    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/295219a7-889c-425c-bf8c-d6332ad25d93-tls-certs\") pod \"sub-test-model-d3c546-kserve-56d8f47565-c7wsz\" (UID: \"295219a7-889c-425c-bf8c-d6332ad25d93\") " pod="ai-tenant-e2e-shared-a-d3c546/sub-test-model-d3c546-kserve-56d8f47565-c7wsz"
Jul 27 20:12:10.883911 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:10.883877    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/295219a7-889c-425c-bf8c-d6332ad25d93-home\") pod \"sub-test-model-d3c546-kserve-56d8f47565-c7wsz\" (UID: \"295219a7-889c-425c-bf8c-d6332ad25d93\") " pod="ai-tenant-e2e-shared-a-d3c546/sub-test-model-d3c546-kserve-56d8f47565-c7wsz"
Jul 27 20:12:10.883911 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:10.883913    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/295219a7-889c-425c-bf8c-d6332ad25d93-tmp-dir\") pod \"sub-test-model-d3c546-kserve-56d8f47565-c7wsz\" (UID: \"295219a7-889c-425c-bf8c-d6332ad25d93\") " pod="ai-tenant-e2e-shared-a-d3c546/sub-test-model-d3c546-kserve-56d8f47565-c7wsz"
Jul 27 20:12:10.884153 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:10.883965    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/295219a7-889c-425c-bf8c-d6332ad25d93-dshm\") pod \"sub-test-model-d3c546-kserve-56d8f47565-c7wsz\" (UID: \"295219a7-889c-425c-bf8c-d6332ad25d93\") " pod="ai-tenant-e2e-shared-a-d3c546/sub-test-model-d3c546-kserve-56d8f47565-c7wsz"
Jul 27 20:12:10.884153 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:10.883984    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhrnc\" (UniqueName: \"kubernetes.io/projected/295219a7-889c-425c-bf8c-d6332ad25d93-kube-api-access-bhrnc\") pod \"sub-test-model-d3c546-kserve-56d8f47565-c7wsz\" (UID: \"295219a7-889c-425c-bf8c-d6332ad25d93\") " pod="ai-tenant-e2e-shared-a-d3c546/sub-test-model-d3c546-kserve-56d8f47565-c7wsz"
Jul 27 20:12:10.884153 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:10.884029    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/295219a7-889c-425c-bf8c-d6332ad25d93-model-cache\") pod \"sub-test-model-d3c546-kserve-56d8f47565-c7wsz\" (UID: \"295219a7-889c-425c-bf8c-d6332ad25d93\") " pod="ai-tenant-e2e-shared-a-d3c546/sub-test-model-d3c546-kserve-56d8f47565-c7wsz"
Jul 27 20:12:10.884153 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:10.884052    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/295219a7-889c-425c-bf8c-d6332ad25d93-tls-certs\") pod \"sub-test-model-d3c546-kserve-56d8f47565-c7wsz\" (UID: \"295219a7-889c-425c-bf8c-d6332ad25d93\") " pod="ai-tenant-e2e-shared-a-d3c546/sub-test-model-d3c546-kserve-56d8f47565-c7wsz"
Jul 27 20:12:10.884385 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:10.884353    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/295219a7-889c-425c-bf8c-d6332ad25d93-home\") pod \"sub-test-model-d3c546-kserve-56d8f47565-c7wsz\" (UID: \"295219a7-889c-425c-bf8c-d6332ad25d93\") " pod="ai-tenant-e2e-shared-a-d3c546/sub-test-model-d3c546-kserve-56d8f47565-c7wsz"
Jul 27 20:12:10.884385 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:10.884369    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/295219a7-889c-425c-bf8c-d6332ad25d93-tmp-dir\") pod \"sub-test-model-d3c546-kserve-56d8f47565-c7wsz\" (UID: \"295219a7-889c-425c-bf8c-d6332ad25d93\") " pod="ai-tenant-e2e-shared-a-d3c546/sub-test-model-d3c546-kserve-56d8f47565-c7wsz"
Jul 27 20:12:10.884521 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:10.884450    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/295219a7-889c-425c-bf8c-d6332ad25d93-model-cache\") pod \"sub-test-model-d3c546-kserve-56d8f47565-c7wsz\" (UID: \"295219a7-889c-425c-bf8c-d6332ad25d93\") " pod="ai-tenant-e2e-shared-a-d3c546/sub-test-model-d3c546-kserve-56d8f47565-c7wsz"
Jul 27 20:12:10.886544 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:10.886528    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/295219a7-889c-425c-bf8c-d6332ad25d93-dshm\") pod \"sub-test-model-d3c546-kserve-56d8f47565-c7wsz\" (UID: \"295219a7-889c-425c-bf8c-d6332ad25d93\") " pod="ai-tenant-e2e-shared-a-d3c546/sub-test-model-d3c546-kserve-56d8f47565-c7wsz"
Jul 27 20:12:10.886799 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:10.886783    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/295219a7-889c-425c-bf8c-d6332ad25d93-tls-certs\") pod \"sub-test-model-d3c546-kserve-56d8f47565-c7wsz\" (UID: \"295219a7-889c-425c-bf8c-d6332ad25d93\") " pod="ai-tenant-e2e-shared-a-d3c546/sub-test-model-d3c546-kserve-56d8f47565-c7wsz"
Jul 27 20:12:10.892595 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:10.892574    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhrnc\" (UniqueName: \"kubernetes.io/projected/295219a7-889c-425c-bf8c-d6332ad25d93-kube-api-access-bhrnc\") pod \"sub-test-model-d3c546-kserve-56d8f47565-c7wsz\" (UID: \"295219a7-889c-425c-bf8c-d6332ad25d93\") " pod="ai-tenant-e2e-shared-a-d3c546/sub-test-model-d3c546-kserve-56d8f47565-c7wsz"
Jul 27 20:12:10.950262 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:10.950237    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d3c546/sub-test-model-d3c546-kserve-56d8f47565-c7wsz"
Jul 27 20:12:11.061765 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:11.061722    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-73ce16/auth-test-model-73ce16-kserve-7dd9748687-dgjkh" podUID="3181b849-aed2-407a-9b4a-0ca25c78dd00" containerName="main" containerID="cri-o://83d6a6c09ecd2e39dbae39531f3cc4aaf8cde59e8ab00b0442f048e0cbd0bfa1" gracePeriod=45
Jul 27 20:12:11.086997 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:11.086973    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-d3c546/sub-test-model-d3c546-kserve-56d8f47565-c7wsz"]
Jul 27 20:12:11.098608 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:11.098583    2558 generic.go:356] "Generic (PLEG): container finished" podID="ea744b6c-4042-497c-a296-a378d05fbd52" containerID="fc8c42e7d64e53f8b85f1805ffcb2651103cb5c44919caf547663083dcf7cba6" exitCode=0
Jul 27 20:12:11.098738 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:11.098612    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d3c546/auth-test-model-d3c546-kserve-5b96cfbb79-p4p9f" event={"ID":"ea744b6c-4042-497c-a296-a378d05fbd52","Type":"ContainerDied","Data":"fc8c42e7d64e53f8b85f1805ffcb2651103cb5c44919caf547663083dcf7cba6"}
Jul 27 20:12:11.098738 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:11.098639    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d3c546/auth-test-model-d3c546-kserve-5b96cfbb79-p4p9f"
Jul 27 20:12:11.098738 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:11.098654    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d3c546/auth-test-model-d3c546-kserve-5b96cfbb79-p4p9f" event={"ID":"ea744b6c-4042-497c-a296-a378d05fbd52","Type":"ContainerDied","Data":"d28573c9db881efe6f3f664f53e599c370a55bcd7ee4a084a7e74e569a65ac86"}
Jul 27 20:12:11.098738 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:11.098673    2558 scope.go:117] "RemoveContainer" containerID="fc8c42e7d64e53f8b85f1805ffcb2651103cb5c44919caf547663083dcf7cba6"
Jul 27 20:12:11.134810 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:11.134748    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-d3c546/auth-test-model-d3c546-kserve-5b96cfbb79-p4p9f"]
Jul 27 20:12:11.136389 ip-10-0-129-235 kubenswrapper[2558]: W0727 20:12:11.136366    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod295219a7_889c_425c_bf8c_d6332ad25d93.slice/crio-a23ce0609d4fa779f9e89238c6de9e21582e7b1f4b007856185393652742a11e WatchSource:0}: Error finding container a23ce0609d4fa779f9e89238c6de9e21582e7b1f4b007856185393652742a11e: Status 404 returned error can't find the container with id a23ce0609d4fa779f9e89238c6de9e21582e7b1f4b007856185393652742a11e
Jul 27 20:12:11.138639 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:11.138614    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-d3c546/auth-test-model-d3c546-kserve-5b96cfbb79-p4p9f"]
Jul 27 20:12:11.173968 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:11.173944    2558 scope.go:117] "RemoveContainer" containerID="fc8c42e7d64e53f8b85f1805ffcb2651103cb5c44919caf547663083dcf7cba6"
Jul 27 20:12:11.174261 ip-10-0-129-235 kubenswrapper[2558]: E0727 20:12:11.174240    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc8c42e7d64e53f8b85f1805ffcb2651103cb5c44919caf547663083dcf7cba6\": container with ID starting with fc8c42e7d64e53f8b85f1805ffcb2651103cb5c44919caf547663083dcf7cba6 not found: ID does not exist" containerID="fc8c42e7d64e53f8b85f1805ffcb2651103cb5c44919caf547663083dcf7cba6"
Jul 27 20:12:11.174323 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:11.174271    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc8c42e7d64e53f8b85f1805ffcb2651103cb5c44919caf547663083dcf7cba6"} err="failed to get container status \"fc8c42e7d64e53f8b85f1805ffcb2651103cb5c44919caf547663083dcf7cba6\": rpc error: code = NotFound desc = could not find container \"fc8c42e7d64e53f8b85f1805ffcb2651103cb5c44919caf547663083dcf7cba6\": container with ID starting with fc8c42e7d64e53f8b85f1805ffcb2651103cb5c44919caf547663083dcf7cba6 not found: ID does not exist"
Jul 27 20:12:11.326850 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:11.326825    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-73ce16/auth-test-model-73ce16-kserve-7dd9748687-dgjkh"
Jul 27 20:12:11.489514 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:11.489427    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8cxjl\" (UniqueName: \"kubernetes.io/projected/3181b849-aed2-407a-9b4a-0ca25c78dd00-kube-api-access-8cxjl\") pod \"3181b849-aed2-407a-9b4a-0ca25c78dd00\" (UID: \"3181b849-aed2-407a-9b4a-0ca25c78dd00\") "
Jul 27 20:12:11.489514 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:11.489464    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3181b849-aed2-407a-9b4a-0ca25c78dd00-model-cache\") pod \"3181b849-aed2-407a-9b4a-0ca25c78dd00\" (UID: \"3181b849-aed2-407a-9b4a-0ca25c78dd00\") "
Jul 27 20:12:11.489701 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:11.489536    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3181b849-aed2-407a-9b4a-0ca25c78dd00-dshm\") pod \"3181b849-aed2-407a-9b4a-0ca25c78dd00\" (UID: \"3181b849-aed2-407a-9b4a-0ca25c78dd00\") "
Jul 27 20:12:11.489701 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:11.489568    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3181b849-aed2-407a-9b4a-0ca25c78dd00-tmp-dir\") pod \"3181b849-aed2-407a-9b4a-0ca25c78dd00\" (UID: \"3181b849-aed2-407a-9b4a-0ca25c78dd00\") "
Jul 27 20:12:11.489701 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:11.489607    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3181b849-aed2-407a-9b4a-0ca25c78dd00-home\") pod \"3181b849-aed2-407a-9b4a-0ca25c78dd00\" (UID: \"3181b849-aed2-407a-9b4a-0ca25c78dd00\") "
Jul 27 20:12:11.489701 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:11.489639    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3181b849-aed2-407a-9b4a-0ca25c78dd00-tls-certs\") pod \"3181b849-aed2-407a-9b4a-0ca25c78dd00\" (UID: \"3181b849-aed2-407a-9b4a-0ca25c78dd00\") "
Jul 27 20:12:11.489894 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:11.489770    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3181b849-aed2-407a-9b4a-0ca25c78dd00-model-cache" (OuterVolumeSpecName: "model-cache") pod "3181b849-aed2-407a-9b4a-0ca25c78dd00" (UID: "3181b849-aed2-407a-9b4a-0ca25c78dd00"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:12:11.489894 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:11.489824    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3181b849-aed2-407a-9b4a-0ca25c78dd00-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3181b849-aed2-407a-9b4a-0ca25c78dd00" (UID: "3181b849-aed2-407a-9b4a-0ca25c78dd00"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:12:11.489894 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:11.489868    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3181b849-aed2-407a-9b4a-0ca25c78dd00-home" (OuterVolumeSpecName: "home") pod "3181b849-aed2-407a-9b4a-0ca25c78dd00" (UID: "3181b849-aed2-407a-9b4a-0ca25c78dd00"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:12:11.490068 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:11.489979    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3181b849-aed2-407a-9b4a-0ca25c78dd00-tmp-dir\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:12:11.490068 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:11.489996    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3181b849-aed2-407a-9b4a-0ca25c78dd00-home\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:12:11.490068 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:11.490009    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3181b849-aed2-407a-9b4a-0ca25c78dd00-model-cache\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:12:11.491898 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:11.491876    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3181b849-aed2-407a-9b4a-0ca25c78dd00-dshm" (OuterVolumeSpecName: "dshm") pod "3181b849-aed2-407a-9b4a-0ca25c78dd00" (UID: "3181b849-aed2-407a-9b4a-0ca25c78dd00"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:12:11.491983 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:11.491932    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3181b849-aed2-407a-9b4a-0ca25c78dd00-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3181b849-aed2-407a-9b4a-0ca25c78dd00" (UID: "3181b849-aed2-407a-9b4a-0ca25c78dd00"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 20:12:11.491983 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:11.491959    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3181b849-aed2-407a-9b4a-0ca25c78dd00-kube-api-access-8cxjl" (OuterVolumeSpecName: "kube-api-access-8cxjl") pod "3181b849-aed2-407a-9b4a-0ca25c78dd00" (UID: "3181b849-aed2-407a-9b4a-0ca25c78dd00"). InnerVolumeSpecName "kube-api-access-8cxjl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:12:11.590765 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:11.590730    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3181b849-aed2-407a-9b4a-0ca25c78dd00-dshm\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:12:11.590765 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:11.590760    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3181b849-aed2-407a-9b4a-0ca25c78dd00-tls-certs\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:12:11.590765 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:11.590771    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8cxjl\" (UniqueName: \"kubernetes.io/projected/3181b849-aed2-407a-9b4a-0ca25c78dd00-kube-api-access-8cxjl\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:12:11.678279 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:11.678246    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea744b6c-4042-497c-a296-a378d05fbd52" path="/var/lib/kubelet/pods/ea744b6c-4042-497c-a296-a378d05fbd52/volumes"
Jul 27 20:12:12.105631 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:12.105597    2558 generic.go:356] "Generic (PLEG): container finished" podID="3181b849-aed2-407a-9b4a-0ca25c78dd00" containerID="83d6a6c09ecd2e39dbae39531f3cc4aaf8cde59e8ab00b0442f048e0cbd0bfa1" exitCode=0
Jul 27 20:12:12.105814 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:12.105639    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-73ce16/auth-test-model-73ce16-kserve-7dd9748687-dgjkh"
Jul 27 20:12:12.105814 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:12.105687    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-73ce16/auth-test-model-73ce16-kserve-7dd9748687-dgjkh" event={"ID":"3181b849-aed2-407a-9b4a-0ca25c78dd00","Type":"ContainerDied","Data":"83d6a6c09ecd2e39dbae39531f3cc4aaf8cde59e8ab00b0442f048e0cbd0bfa1"}
Jul 27 20:12:12.105814 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:12.105733    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-73ce16/auth-test-model-73ce16-kserve-7dd9748687-dgjkh" event={"ID":"3181b849-aed2-407a-9b4a-0ca25c78dd00","Type":"ContainerDied","Data":"e71dc1db24a2d582ecc02a373336e9d793c35592d91f0b7c4163d20ea5097338"}
Jul 27 20:12:12.105814 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:12.105755    2558 scope.go:117] "RemoveContainer" containerID="83d6a6c09ecd2e39dbae39531f3cc4aaf8cde59e8ab00b0442f048e0cbd0bfa1"
Jul 27 20:12:12.108616 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:12.108592    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d3c546/sub-test-model-d3c546-kserve-56d8f47565-c7wsz" event={"ID":"295219a7-889c-425c-bf8c-d6332ad25d93","Type":"ContainerStarted","Data":"2ea5431816fa6ec1eeb9ebf0c5bb4ccfe2518cfe1f2f91f47811dcb875fd544c"}
Jul 27 20:12:12.108732 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:12.108625    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d3c546/sub-test-model-d3c546-kserve-56d8f47565-c7wsz" event={"ID":"295219a7-889c-425c-bf8c-d6332ad25d93","Type":"ContainerStarted","Data":"a23ce0609d4fa779f9e89238c6de9e21582e7b1f4b007856185393652742a11e"}
Jul 27 20:12:12.114986 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:12.114969    2558 scope.go:117] "RemoveContainer" containerID="83d6a6c09ecd2e39dbae39531f3cc4aaf8cde59e8ab00b0442f048e0cbd0bfa1"
Jul 27 20:12:12.115247 ip-10-0-129-235 kubenswrapper[2558]: E0727 20:12:12.115231    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83d6a6c09ecd2e39dbae39531f3cc4aaf8cde59e8ab00b0442f048e0cbd0bfa1\": container with ID starting with 83d6a6c09ecd2e39dbae39531f3cc4aaf8cde59e8ab00b0442f048e0cbd0bfa1 not found: ID does not exist" containerID="83d6a6c09ecd2e39dbae39531f3cc4aaf8cde59e8ab00b0442f048e0cbd0bfa1"
Jul 27 20:12:12.115292 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:12.115255    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83d6a6c09ecd2e39dbae39531f3cc4aaf8cde59e8ab00b0442f048e0cbd0bfa1"} err="failed to get container status \"83d6a6c09ecd2e39dbae39531f3cc4aaf8cde59e8ab00b0442f048e0cbd0bfa1\": rpc error: code = NotFound desc = could not find container \"83d6a6c09ecd2e39dbae39531f3cc4aaf8cde59e8ab00b0442f048e0cbd0bfa1\": container with ID starting with 83d6a6c09ecd2e39dbae39531f3cc4aaf8cde59e8ab00b0442f048e0cbd0bfa1 not found: ID does not exist"
Jul 27 20:12:12.122471 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:12.122449    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-73ce16/auth-test-model-73ce16-kserve-7dd9748687-dgjkh"]
Jul 27 20:12:12.127127 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:12.127101    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-73ce16/auth-test-model-73ce16-kserve-7dd9748687-dgjkh"]
Jul 27 20:12:12.139460 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:12.139397    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-d3c546/sub-test-model-d3c546-kserve-56d8f47565-c7wsz" podStartSLOduration=2.13938365 podStartE2EDuration="2.13938365s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:12:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:12:12.137992588 +0000 UTC m=+2745.010184573" watchObservedRunningTime="2026-07-27 20:12:12.13938365 +0000 UTC m=+2745.011575665"
Jul 27 20:12:13.674952 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:13.674917    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3181b849-aed2-407a-9b4a-0ca25c78dd00" path="/var/lib/kubelet/pods/3181b849-aed2-407a-9b4a-0ca25c78dd00/volumes"
Jul 27 20:12:20.951307 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:20.951270    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-d3c546/sub-test-model-d3c546-kserve-56d8f47565-c7wsz"
Jul 27 20:12:20.951728 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:20.951357    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-d3c546/sub-test-model-d3c546-kserve-56d8f47565-c7wsz"
Jul 27 20:12:20.964316 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:20.964293    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-d3c546/sub-test-model-d3c546-kserve-56d8f47565-c7wsz"
Jul 27 20:12:21.171830 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:12:21.171801    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-d3c546/sub-test-model-d3c546-kserve-56d8f47565-c7wsz"
Jul 27 20:13:05.963963 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:13:05.963930    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:13:14.066462 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:13:14.066428    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:13:14.166448 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:13:14.166393    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:14:23.510176 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:23.510086    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-d3c546/sub-test-model-d3c546-kserve-56d8f47565-c7wsz"]
Jul 27 20:14:23.530134 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:23.530103    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:14:24.323836 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:24.323804    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:14:28.728448 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:28.728390    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-d3c546/rate-test-model-d3c546-kserve-56476c7fd7-sds7w"]
Jul 27 20:14:28.729008 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:28.728988    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3181b849-aed2-407a-9b4a-0ca25c78dd00" containerName="main"
Jul 27 20:14:28.729095 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:28.729010    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="3181b849-aed2-407a-9b4a-0ca25c78dd00" containerName="main"
Jul 27 20:14:28.729164 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:28.729105    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="3181b849-aed2-407a-9b4a-0ca25c78dd00" containerName="main"
Jul 27 20:14:28.732129 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:28.732107    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d3c546/rate-test-model-d3c546-kserve-56476c7fd7-sds7w"
Jul 27 20:14:28.734223 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:28.734202    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-d3c546\"/\"rate-test-model-d3c546-kserve-self-signed-certs\""
Jul 27 20:14:28.743242 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:28.743217    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-d3c546/rate-test-model-d3c546-kserve-56476c7fd7-sds7w"]
Jul 27 20:14:28.777063 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:28.777042    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:14:28.864640 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:28.864610    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2wg7\" (UniqueName: \"kubernetes.io/projected/e4258f71-8a85-4b71-a5de-b842ef753747-kube-api-access-x2wg7\") pod \"rate-test-model-d3c546-kserve-56476c7fd7-sds7w\" (UID: \"e4258f71-8a85-4b71-a5de-b842ef753747\") " pod="ai-tenant-e2e-shared-a-d3c546/rate-test-model-d3c546-kserve-56476c7fd7-sds7w"
Jul 27 20:14:28.864786 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:28.864651    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4258f71-8a85-4b71-a5de-b842ef753747-tls-certs\") pod \"rate-test-model-d3c546-kserve-56476c7fd7-sds7w\" (UID: \"e4258f71-8a85-4b71-a5de-b842ef753747\") " pod="ai-tenant-e2e-shared-a-d3c546/rate-test-model-d3c546-kserve-56476c7fd7-sds7w"
Jul 27 20:14:28.864786 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:28.864716    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4258f71-8a85-4b71-a5de-b842ef753747-tmp-dir\") pod \"rate-test-model-d3c546-kserve-56476c7fd7-sds7w\" (UID: \"e4258f71-8a85-4b71-a5de-b842ef753747\") " pod="ai-tenant-e2e-shared-a-d3c546/rate-test-model-d3c546-kserve-56476c7fd7-sds7w"
Jul 27 20:14:28.864786 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:28.864777    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4258f71-8a85-4b71-a5de-b842ef753747-home\") pod \"rate-test-model-d3c546-kserve-56476c7fd7-sds7w\" (UID: \"e4258f71-8a85-4b71-a5de-b842ef753747\") " pod="ai-tenant-e2e-shared-a-d3c546/rate-test-model-d3c546-kserve-56476c7fd7-sds7w"
Jul 27 20:14:28.864925 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:28.864799    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4258f71-8a85-4b71-a5de-b842ef753747-dshm\") pod \"rate-test-model-d3c546-kserve-56476c7fd7-sds7w\" (UID: \"e4258f71-8a85-4b71-a5de-b842ef753747\") " pod="ai-tenant-e2e-shared-a-d3c546/rate-test-model-d3c546-kserve-56476c7fd7-sds7w"
Jul 27 20:14:28.864925 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:28.864883    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4258f71-8a85-4b71-a5de-b842ef753747-model-cache\") pod \"rate-test-model-d3c546-kserve-56476c7fd7-sds7w\" (UID: \"e4258f71-8a85-4b71-a5de-b842ef753747\") " pod="ai-tenant-e2e-shared-a-d3c546/rate-test-model-d3c546-kserve-56476c7fd7-sds7w"
Jul 27 20:14:28.965275 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:28.965241    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2wg7\" (UniqueName: \"kubernetes.io/projected/e4258f71-8a85-4b71-a5de-b842ef753747-kube-api-access-x2wg7\") pod \"rate-test-model-d3c546-kserve-56476c7fd7-sds7w\" (UID: \"e4258f71-8a85-4b71-a5de-b842ef753747\") " pod="ai-tenant-e2e-shared-a-d3c546/rate-test-model-d3c546-kserve-56476c7fd7-sds7w"
Jul 27 20:14:28.965555 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:28.965284    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4258f71-8a85-4b71-a5de-b842ef753747-tls-certs\") pod \"rate-test-model-d3c546-kserve-56476c7fd7-sds7w\" (UID: \"e4258f71-8a85-4b71-a5de-b842ef753747\") " pod="ai-tenant-e2e-shared-a-d3c546/rate-test-model-d3c546-kserve-56476c7fd7-sds7w"
Jul 27 20:14:28.965555 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:28.965433    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4258f71-8a85-4b71-a5de-b842ef753747-tmp-dir\") pod \"rate-test-model-d3c546-kserve-56476c7fd7-sds7w\" (UID: \"e4258f71-8a85-4b71-a5de-b842ef753747\") " pod="ai-tenant-e2e-shared-a-d3c546/rate-test-model-d3c546-kserve-56476c7fd7-sds7w"
Jul 27 20:14:28.965555 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:28.965537    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4258f71-8a85-4b71-a5de-b842ef753747-home\") pod \"rate-test-model-d3c546-kserve-56476c7fd7-sds7w\" (UID: \"e4258f71-8a85-4b71-a5de-b842ef753747\") " pod="ai-tenant-e2e-shared-a-d3c546/rate-test-model-d3c546-kserve-56476c7fd7-sds7w"
Jul 27 20:14:28.965707 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:28.965570    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4258f71-8a85-4b71-a5de-b842ef753747-dshm\") pod \"rate-test-model-d3c546-kserve-56476c7fd7-sds7w\" (UID: \"e4258f71-8a85-4b71-a5de-b842ef753747\") " pod="ai-tenant-e2e-shared-a-d3c546/rate-test-model-d3c546-kserve-56476c7fd7-sds7w"
Jul 27 20:14:28.965707 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:28.965609    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4258f71-8a85-4b71-a5de-b842ef753747-model-cache\") pod \"rate-test-model-d3c546-kserve-56476c7fd7-sds7w\" (UID: \"e4258f71-8a85-4b71-a5de-b842ef753747\") " pod="ai-tenant-e2e-shared-a-d3c546/rate-test-model-d3c546-kserve-56476c7fd7-sds7w"
Jul 27 20:14:28.965818 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:28.965772    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4258f71-8a85-4b71-a5de-b842ef753747-tmp-dir\") pod \"rate-test-model-d3c546-kserve-56476c7fd7-sds7w\" (UID: \"e4258f71-8a85-4b71-a5de-b842ef753747\") " pod="ai-tenant-e2e-shared-a-d3c546/rate-test-model-d3c546-kserve-56476c7fd7-sds7w"
Jul 27 20:14:28.965880 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:28.965830    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4258f71-8a85-4b71-a5de-b842ef753747-home\") pod \"rate-test-model-d3c546-kserve-56476c7fd7-sds7w\" (UID: \"e4258f71-8a85-4b71-a5de-b842ef753747\") " pod="ai-tenant-e2e-shared-a-d3c546/rate-test-model-d3c546-kserve-56476c7fd7-sds7w"
Jul 27 20:14:28.965989 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:28.965963    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4258f71-8a85-4b71-a5de-b842ef753747-model-cache\") pod \"rate-test-model-d3c546-kserve-56476c7fd7-sds7w\" (UID: \"e4258f71-8a85-4b71-a5de-b842ef753747\") " pod="ai-tenant-e2e-shared-a-d3c546/rate-test-model-d3c546-kserve-56476c7fd7-sds7w"
Jul 27 20:14:28.967778 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:28.967755    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4258f71-8a85-4b71-a5de-b842ef753747-dshm\") pod \"rate-test-model-d3c546-kserve-56476c7fd7-sds7w\" (UID: \"e4258f71-8a85-4b71-a5de-b842ef753747\") " pod="ai-tenant-e2e-shared-a-d3c546/rate-test-model-d3c546-kserve-56476c7fd7-sds7w"
Jul 27 20:14:28.968079 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:28.968059    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4258f71-8a85-4b71-a5de-b842ef753747-tls-certs\") pod \"rate-test-model-d3c546-kserve-56476c7fd7-sds7w\" (UID: \"e4258f71-8a85-4b71-a5de-b842ef753747\") " pod="ai-tenant-e2e-shared-a-d3c546/rate-test-model-d3c546-kserve-56476c7fd7-sds7w"
Jul 27 20:14:28.973882 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:28.973860    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2wg7\" (UniqueName: \"kubernetes.io/projected/e4258f71-8a85-4b71-a5de-b842ef753747-kube-api-access-x2wg7\") pod \"rate-test-model-d3c546-kserve-56476c7fd7-sds7w\" (UID: \"e4258f71-8a85-4b71-a5de-b842ef753747\") " pod="ai-tenant-e2e-shared-a-d3c546/rate-test-model-d3c546-kserve-56476c7fd7-sds7w"
Jul 27 20:14:29.042057 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:29.041989    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d3c546/rate-test-model-d3c546-kserve-56476c7fd7-sds7w"
Jul 27 20:14:29.180298 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:29.180270    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-d3c546/rate-test-model-d3c546-kserve-56476c7fd7-sds7w"]
Jul 27 20:14:29.181899 ip-10-0-129-235 kubenswrapper[2558]: W0727 20:14:29.181871    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode4258f71_8a85_4b71_a5de_b842ef753747.slice/crio-02d89b1a48a59e009a9f9e1cbd33bb72e94cdea33b500aa2d163edea5bee502e WatchSource:0}: Error finding container 02d89b1a48a59e009a9f9e1cbd33bb72e94cdea33b500aa2d163edea5bee502e: Status 404 returned error can't find the container with id 02d89b1a48a59e009a9f9e1cbd33bb72e94cdea33b500aa2d163edea5bee502e
Jul 27 20:14:29.857195 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:29.857164    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d3c546/rate-test-model-d3c546-kserve-56476c7fd7-sds7w" event={"ID":"e4258f71-8a85-4b71-a5de-b842ef753747","Type":"ContainerStarted","Data":"cfb8e8bc48d7846aef6b9bdd652efd65d6a377d7019793626b382dae4592b106"}
Jul 27 20:14:29.857620 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:29.857207    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d3c546/rate-test-model-d3c546-kserve-56476c7fd7-sds7w" event={"ID":"e4258f71-8a85-4b71-a5de-b842ef753747","Type":"ContainerStarted","Data":"02d89b1a48a59e009a9f9e1cbd33bb72e94cdea33b500aa2d163edea5bee502e"}
Jul 27 20:14:29.874771 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:29.874724    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-d3c546/rate-test-model-d3c546-kserve-56476c7fd7-sds7w" podStartSLOduration=1.874710956 podStartE2EDuration="1.874710956s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:14:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:14:29.873234478 +0000 UTC m=+2882.745426452" watchObservedRunningTime="2026-07-27 20:14:29.874710956 +0000 UTC m=+2882.746902930"
Jul 27 20:14:38.529997 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:38.529933    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-d3c546/sub-test-model-d3c546-kserve-56d8f47565-c7wsz" podUID="295219a7-889c-425c-bf8c-d6332ad25d93" containerName="main" containerID="cri-o://2ea5431816fa6ec1eeb9ebf0c5bb4ccfe2518cfe1f2f91f47811dcb875fd544c" gracePeriod=45
Jul 27 20:14:38.772492 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:38.772462    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d3c546/sub-test-model-d3c546-kserve-56d8f47565-c7wsz"
Jul 27 20:14:38.853574 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:38.853487    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/295219a7-889c-425c-bf8c-d6332ad25d93-home\") pod \"295219a7-889c-425c-bf8c-d6332ad25d93\" (UID: \"295219a7-889c-425c-bf8c-d6332ad25d93\") "
Jul 27 20:14:38.853574 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:38.853521    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/295219a7-889c-425c-bf8c-d6332ad25d93-tmp-dir\") pod \"295219a7-889c-425c-bf8c-d6332ad25d93\" (UID: \"295219a7-889c-425c-bf8c-d6332ad25d93\") "
Jul 27 20:14:38.853574 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:38.853571    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bhrnc\" (UniqueName: \"kubernetes.io/projected/295219a7-889c-425c-bf8c-d6332ad25d93-kube-api-access-bhrnc\") pod \"295219a7-889c-425c-bf8c-d6332ad25d93\" (UID: \"295219a7-889c-425c-bf8c-d6332ad25d93\") "
Jul 27 20:14:38.853866 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:38.853601    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/295219a7-889c-425c-bf8c-d6332ad25d93-tls-certs\") pod \"295219a7-889c-425c-bf8c-d6332ad25d93\" (UID: \"295219a7-889c-425c-bf8c-d6332ad25d93\") "
Jul 27 20:14:38.853866 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:38.853678    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/295219a7-889c-425c-bf8c-d6332ad25d93-model-cache\") pod \"295219a7-889c-425c-bf8c-d6332ad25d93\" (UID: \"295219a7-889c-425c-bf8c-d6332ad25d93\") "
Jul 27 20:14:38.853866 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:38.853742    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/295219a7-889c-425c-bf8c-d6332ad25d93-dshm\") pod \"295219a7-889c-425c-bf8c-d6332ad25d93\" (UID: \"295219a7-889c-425c-bf8c-d6332ad25d93\") "
Jul 27 20:14:38.853866 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:38.853770    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/295219a7-889c-425c-bf8c-d6332ad25d93-home" (OuterVolumeSpecName: "home") pod "295219a7-889c-425c-bf8c-d6332ad25d93" (UID: "295219a7-889c-425c-bf8c-d6332ad25d93"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:14:38.853866 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:38.853857    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/295219a7-889c-425c-bf8c-d6332ad25d93-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "295219a7-889c-425c-bf8c-d6332ad25d93" (UID: "295219a7-889c-425c-bf8c-d6332ad25d93"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:14:38.854091 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:38.853972    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/295219a7-889c-425c-bf8c-d6332ad25d93-model-cache" (OuterVolumeSpecName: "model-cache") pod "295219a7-889c-425c-bf8c-d6332ad25d93" (UID: "295219a7-889c-425c-bf8c-d6332ad25d93"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:14:38.854134 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:38.854097    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/295219a7-889c-425c-bf8c-d6332ad25d93-model-cache\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:14:38.854134 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:38.854120    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/295219a7-889c-425c-bf8c-d6332ad25d93-home\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:14:38.854226 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:38.854160    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/295219a7-889c-425c-bf8c-d6332ad25d93-tmp-dir\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:14:38.855920 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:38.855895    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/295219a7-889c-425c-bf8c-d6332ad25d93-dshm" (OuterVolumeSpecName: "dshm") pod "295219a7-889c-425c-bf8c-d6332ad25d93" (UID: "295219a7-889c-425c-bf8c-d6332ad25d93"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:14:38.856029 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:38.855927    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/295219a7-889c-425c-bf8c-d6332ad25d93-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "295219a7-889c-425c-bf8c-d6332ad25d93" (UID: "295219a7-889c-425c-bf8c-d6332ad25d93"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 20:14:38.856029 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:38.856003    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/295219a7-889c-425c-bf8c-d6332ad25d93-kube-api-access-bhrnc" (OuterVolumeSpecName: "kube-api-access-bhrnc") pod "295219a7-889c-425c-bf8c-d6332ad25d93" (UID: "295219a7-889c-425c-bf8c-d6332ad25d93"). InnerVolumeSpecName "kube-api-access-bhrnc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:14:38.916493 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:38.916460    2558 generic.go:356] "Generic (PLEG): container finished" podID="295219a7-889c-425c-bf8c-d6332ad25d93" containerID="2ea5431816fa6ec1eeb9ebf0c5bb4ccfe2518cfe1f2f91f47811dcb875fd544c" exitCode=0
Jul 27 20:14:38.916493 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:38.916504    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d3c546/sub-test-model-d3c546-kserve-56d8f47565-c7wsz"
Jul 27 20:14:38.916737 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:38.916550    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d3c546/sub-test-model-d3c546-kserve-56d8f47565-c7wsz" event={"ID":"295219a7-889c-425c-bf8c-d6332ad25d93","Type":"ContainerDied","Data":"2ea5431816fa6ec1eeb9ebf0c5bb4ccfe2518cfe1f2f91f47811dcb875fd544c"}
Jul 27 20:14:38.916737 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:38.916599    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d3c546/sub-test-model-d3c546-kserve-56d8f47565-c7wsz" event={"ID":"295219a7-889c-425c-bf8c-d6332ad25d93","Type":"ContainerDied","Data":"a23ce0609d4fa779f9e89238c6de9e21582e7b1f4b007856185393652742a11e"}
Jul 27 20:14:38.916737 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:38.916630    2558 scope.go:117] "RemoveContainer" containerID="2ea5431816fa6ec1eeb9ebf0c5bb4ccfe2518cfe1f2f91f47811dcb875fd544c"
Jul 27 20:14:38.926402 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:38.926384    2558 scope.go:117] "RemoveContainer" containerID="2ea5431816fa6ec1eeb9ebf0c5bb4ccfe2518cfe1f2f91f47811dcb875fd544c"
Jul 27 20:14:38.926722 ip-10-0-129-235 kubenswrapper[2558]: E0727 20:14:38.926701    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ea5431816fa6ec1eeb9ebf0c5bb4ccfe2518cfe1f2f91f47811dcb875fd544c\": container with ID starting with 2ea5431816fa6ec1eeb9ebf0c5bb4ccfe2518cfe1f2f91f47811dcb875fd544c not found: ID does not exist" containerID="2ea5431816fa6ec1eeb9ebf0c5bb4ccfe2518cfe1f2f91f47811dcb875fd544c"
Jul 27 20:14:38.926787 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:38.926729    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ea5431816fa6ec1eeb9ebf0c5bb4ccfe2518cfe1f2f91f47811dcb875fd544c"} err="failed to get container status \"2ea5431816fa6ec1eeb9ebf0c5bb4ccfe2518cfe1f2f91f47811dcb875fd544c\": rpc error: code = NotFound desc = could not find container \"2ea5431816fa6ec1eeb9ebf0c5bb4ccfe2518cfe1f2f91f47811dcb875fd544c\": container with ID starting with 2ea5431816fa6ec1eeb9ebf0c5bb4ccfe2518cfe1f2f91f47811dcb875fd544c not found: ID does not exist"
Jul 27 20:14:38.938005 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:38.937972    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-d3c546/sub-test-model-d3c546-kserve-56d8f47565-c7wsz"]
Jul 27 20:14:38.941597 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:38.941575    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-d3c546/sub-test-model-d3c546-kserve-56d8f47565-c7wsz"]
Jul 27 20:14:38.955337 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:38.955314    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bhrnc\" (UniqueName: \"kubernetes.io/projected/295219a7-889c-425c-bf8c-d6332ad25d93-kube-api-access-bhrnc\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:14:38.955337 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:38.955337    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/295219a7-889c-425c-bf8c-d6332ad25d93-tls-certs\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:14:38.955479 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:38.955347    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/295219a7-889c-425c-bf8c-d6332ad25d93-dshm\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:14:39.042902 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:39.042868    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-d3c546/rate-test-model-d3c546-kserve-56476c7fd7-sds7w"
Jul 27 20:14:39.043099 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:39.042915    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-d3c546/rate-test-model-d3c546-kserve-56476c7fd7-sds7w"
Jul 27 20:14:39.056190 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:39.056162    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-d3c546/rate-test-model-d3c546-kserve-56476c7fd7-sds7w"
Jul 27 20:14:39.674497 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:39.674468    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="295219a7-889c-425c-bf8c-d6332ad25d93" path="/var/lib/kubelet/pods/295219a7-889c-425c-bf8c-d6332ad25d93/volumes"
Jul 27 20:14:39.936597 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:39.936502    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-d3c546/rate-test-model-d3c546-kserve-56476c7fd7-sds7w"
Jul 27 20:14:48.946887 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:14:48.946856    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:15:20.031118 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:15:20.031078    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:15:57.000230 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:15:57.000150    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-d3c546/rate-test-model-d3c546-kserve-56476c7fd7-sds7w"]
Jul 27 20:15:57.017485 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:15:57.017392    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:15:58.426752 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:15:58.426704    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:15:58.738877 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:15:58.738798    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7483-140606343273520"]
Jul 27 20:15:58.739203 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:15:58.739191    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="295219a7-889c-425c-bf8c-d6332ad25d93" containerName="main"
Jul 27 20:15:58.739255 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:15:58.739205    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="295219a7-889c-425c-bf8c-d6332ad25d93" containerName="main"
Jul 27 20:15:58.739293 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:15:58.739288    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="295219a7-889c-425c-bf8c-d6332ad25d93" containerName="main"
Jul 27 20:15:58.742352 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:15:58.742336    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7483-140606343273520"
Jul 27 20:15:58.744475 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:15:58.744458    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-khr5c\""
Jul 27 20:15:58.749153 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:15:58.749128    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7483-140606343273520"]
Jul 27 20:15:58.880992 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:15:58.880948    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qc7lh\" (UniqueName: \"kubernetes.io/projected/fb5a28a4-60a5-4c11-a65a-f87545a8d070-kube-api-access-qc7lh\") pod \"test-curl-7483-140606343273520\" (UID: \"fb5a28a4-60a5-4c11-a65a-f87545a8d070\") " pod="opendatahub/test-curl-7483-140606343273520"
Jul 27 20:15:58.981849 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:15:58.981814    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qc7lh\" (UniqueName: \"kubernetes.io/projected/fb5a28a4-60a5-4c11-a65a-f87545a8d070-kube-api-access-qc7lh\") pod \"test-curl-7483-140606343273520\" (UID: \"fb5a28a4-60a5-4c11-a65a-f87545a8d070\") " pod="opendatahub/test-curl-7483-140606343273520"
Jul 27 20:15:58.989703 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:15:58.989632    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qc7lh\" (UniqueName: \"kubernetes.io/projected/fb5a28a4-60a5-4c11-a65a-f87545a8d070-kube-api-access-qc7lh\") pod \"test-curl-7483-140606343273520\" (UID: \"fb5a28a4-60a5-4c11-a65a-f87545a8d070\") " pod="opendatahub/test-curl-7483-140606343273520"
Jul 27 20:15:59.052651 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:15:59.052613    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7483-140606343273520"
Jul 27 20:15:59.190309 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:15:59.190281    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7483-140606343273520"]
Jul 27 20:15:59.192298 ip-10-0-129-235 kubenswrapper[2558]: W0727 20:15:59.192273    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfb5a28a4_60a5_4c11_a65a_f87545a8d070.slice/crio-daf9b66f65854f3da827bbbb5d9117ebd3a0f94b68b6028e902c14169a6fc9c6 WatchSource:0}: Error finding container daf9b66f65854f3da827bbbb5d9117ebd3a0f94b68b6028e902c14169a6fc9c6: Status 404 returned error can't find the container with id daf9b66f65854f3da827bbbb5d9117ebd3a0f94b68b6028e902c14169a6fc9c6
Jul 27 20:15:59.193653 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:15:59.193635    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 20:15:59.383342 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:15:59.383303    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7483-140606343273520" event={"ID":"fb5a28a4-60a5-4c11-a65a-f87545a8d070","Type":"ContainerStarted","Data":"daf9b66f65854f3da827bbbb5d9117ebd3a0f94b68b6028e902c14169a6fc9c6"}
Jul 27 20:16:00.390746 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:00.390707    2558 generic.go:356] "Generic (PLEG): container finished" podID="fb5a28a4-60a5-4c11-a65a-f87545a8d070" containerID="f3325f4dd50851a8131a0fc4cedcc048a1ec2d47b1233710ee8f8a4197c1be2f" exitCode=0
Jul 27 20:16:00.391159 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:00.390796    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7483-140606343273520" event={"ID":"fb5a28a4-60a5-4c11-a65a-f87545a8d070","Type":"ContainerDied","Data":"f3325f4dd50851a8131a0fc4cedcc048a1ec2d47b1233710ee8f8a4197c1be2f"}
Jul 27 20:16:01.542025 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:01.541994    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7483-140606343273520"
Jul 27 20:16:01.561231 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:01.561192    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7483-140606343273520_fb5a28a4-60a5-4c11-a65a-f87545a8d070/test-curl-7483-140606343273520/0.log"
Jul 27 20:16:01.583985 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:01.583945    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7483-140606343273520"]
Jul 27 20:16:01.587442 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:01.587398    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7483-140606343273520"]
Jul 27 20:16:01.700431 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:01.700316    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7483-140606343270960"]
Jul 27 20:16:01.700752 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:01.700736    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb5a28a4-60a5-4c11-a65a-f87545a8d070" containerName="test-curl-7483-140606343273520"
Jul 27 20:16:01.700798 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:01.700756    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb5a28a4-60a5-4c11-a65a-f87545a8d070" containerName="test-curl-7483-140606343273520"
Jul 27 20:16:01.700855 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:01.700845    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb5a28a4-60a5-4c11-a65a-f87545a8d070" containerName="test-curl-7483-140606343273520"
Jul 27 20:16:01.704024 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:01.704003    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7483-140606343270960"
Jul 27 20:16:01.708591 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:01.708562    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7483-140606343270960"]
Jul 27 20:16:01.708891 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:01.708866    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qc7lh\" (UniqueName: \"kubernetes.io/projected/fb5a28a4-60a5-4c11-a65a-f87545a8d070-kube-api-access-qc7lh\") pod \"fb5a28a4-60a5-4c11-a65a-f87545a8d070\" (UID: \"fb5a28a4-60a5-4c11-a65a-f87545a8d070\") "
Jul 27 20:16:01.712088 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:01.712049    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb5a28a4-60a5-4c11-a65a-f87545a8d070-kube-api-access-qc7lh" (OuterVolumeSpecName: "kube-api-access-qc7lh") pod "fb5a28a4-60a5-4c11-a65a-f87545a8d070" (UID: "fb5a28a4-60a5-4c11-a65a-f87545a8d070"). InnerVolumeSpecName "kube-api-access-qc7lh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:16:01.810608 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:01.810561    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7qxj\" (UniqueName: \"kubernetes.io/projected/ca7b5cb0-1e3a-4916-9765-f42a8948a89a-kube-api-access-v7qxj\") pod \"test-curl-7483-140606343270960\" (UID: \"ca7b5cb0-1e3a-4916-9765-f42a8948a89a\") " pod="opendatahub/test-curl-7483-140606343270960"
Jul 27 20:16:01.810813 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:01.810733    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qc7lh\" (UniqueName: \"kubernetes.io/projected/fb5a28a4-60a5-4c11-a65a-f87545a8d070-kube-api-access-qc7lh\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:16:01.911639 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:01.911597    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7qxj\" (UniqueName: \"kubernetes.io/projected/ca7b5cb0-1e3a-4916-9765-f42a8948a89a-kube-api-access-v7qxj\") pod \"test-curl-7483-140606343270960\" (UID: \"ca7b5cb0-1e3a-4916-9765-f42a8948a89a\") " pod="opendatahub/test-curl-7483-140606343270960"
Jul 27 20:16:01.921224 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:01.921192    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7qxj\" (UniqueName: \"kubernetes.io/projected/ca7b5cb0-1e3a-4916-9765-f42a8948a89a-kube-api-access-v7qxj\") pod \"test-curl-7483-140606343270960\" (UID: \"ca7b5cb0-1e3a-4916-9765-f42a8948a89a\") " pod="opendatahub/test-curl-7483-140606343270960"
Jul 27 20:16:02.025504 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:02.025393    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7483-140606343270960"
Jul 27 20:16:02.171524 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:02.171489    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7483-140606343270960"]
Jul 27 20:16:02.173001 ip-10-0-129-235 kubenswrapper[2558]: W0727 20:16:02.172971    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podca7b5cb0_1e3a_4916_9765_f42a8948a89a.slice/crio-49c7296860f46dc223dd77add0ed93c24c29700fb7836d7fdf9ee516c1f2ceda WatchSource:0}: Error finding container 49c7296860f46dc223dd77add0ed93c24c29700fb7836d7fdf9ee516c1f2ceda: Status 404 returned error can't find the container with id 49c7296860f46dc223dd77add0ed93c24c29700fb7836d7fdf9ee516c1f2ceda
Jul 27 20:16:02.404341 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:02.404305    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7483-140606343273520"
Jul 27 20:16:02.404587 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:02.404379    2558 scope.go:117] "RemoveContainer" containerID="f3325f4dd50851a8131a0fc4cedcc048a1ec2d47b1233710ee8f8a4197c1be2f"
Jul 27 20:16:02.405750 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:02.405721    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7483-140606343270960" event={"ID":"ca7b5cb0-1e3a-4916-9765-f42a8948a89a","Type":"ContainerStarted","Data":"49c7296860f46dc223dd77add0ed93c24c29700fb7836d7fdf9ee516c1f2ceda"}
Jul 27 20:16:03.413262 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:03.413226    2558 generic.go:356] "Generic (PLEG): container finished" podID="ca7b5cb0-1e3a-4916-9765-f42a8948a89a" containerID="48aa988f01f86f4e3360e4b0426541b2c5df08ca5f6b992dbac0890b3bfb1c9e" exitCode=0
Jul 27 20:16:03.413747 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:03.413315    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7483-140606343270960" event={"ID":"ca7b5cb0-1e3a-4916-9765-f42a8948a89a","Type":"ContainerDied","Data":"48aa988f01f86f4e3360e4b0426541b2c5df08ca5f6b992dbac0890b3bfb1c9e"}
Jul 27 20:16:03.674604 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:03.674514    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb5a28a4-60a5-4c11-a65a-f87545a8d070" path="/var/lib/kubelet/pods/fb5a28a4-60a5-4c11-a65a-f87545a8d070/volumes"
Jul 27 20:16:04.562169 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:04.562141    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7483-140606343270960"
Jul 27 20:16:04.580719 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:04.580690    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7483-140606343270960_ca7b5cb0-1e3a-4916-9765-f42a8948a89a/test-curl-7483-140606343270960/0.log"
Jul 27 20:16:04.602948 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:04.602918    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7483-140606343270960"]
Jul 27 20:16:04.606682 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:04.606659    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7483-140606343270960"]
Jul 27 20:16:04.634494 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:04.634470    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v7qxj\" (UniqueName: \"kubernetes.io/projected/ca7b5cb0-1e3a-4916-9765-f42a8948a89a-kube-api-access-v7qxj\") pod \"ca7b5cb0-1e3a-4916-9765-f42a8948a89a\" (UID: \"ca7b5cb0-1e3a-4916-9765-f42a8948a89a\") "
Jul 27 20:16:04.636690 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:04.636667    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca7b5cb0-1e3a-4916-9765-f42a8948a89a-kube-api-access-v7qxj" (OuterVolumeSpecName: "kube-api-access-v7qxj") pod "ca7b5cb0-1e3a-4916-9765-f42a8948a89a" (UID: "ca7b5cb0-1e3a-4916-9765-f42a8948a89a"). InnerVolumeSpecName "kube-api-access-v7qxj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:16:04.735354 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:04.735276    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v7qxj\" (UniqueName: \"kubernetes.io/projected/ca7b5cb0-1e3a-4916-9765-f42a8948a89a-kube-api-access-v7qxj\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:16:04.760266 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:04.760238    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7483-140606338824064"]
Jul 27 20:16:04.760707 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:04.760689    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca7b5cb0-1e3a-4916-9765-f42a8948a89a" containerName="test-curl-7483-140606343270960"
Jul 27 20:16:04.760791 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:04.760712    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca7b5cb0-1e3a-4916-9765-f42a8948a89a" containerName="test-curl-7483-140606343270960"
Jul 27 20:16:04.760826 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:04.760814    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca7b5cb0-1e3a-4916-9765-f42a8948a89a" containerName="test-curl-7483-140606343270960"
Jul 27 20:16:04.763873 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:04.763857    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7483-140606338824064"
Jul 27 20:16:04.769896 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:04.769873    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7483-140606338824064"]
Jul 27 20:16:04.836173 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:04.836139    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qppgn\" (UniqueName: \"kubernetes.io/projected/2e22dd6b-005b-4b58-a8d2-59959096a492-kube-api-access-qppgn\") pod \"test-curl-7483-140606338824064\" (UID: \"2e22dd6b-005b-4b58-a8d2-59959096a492\") " pod="opendatahub/test-curl-7483-140606338824064"
Jul 27 20:16:04.937445 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:04.937397    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qppgn\" (UniqueName: \"kubernetes.io/projected/2e22dd6b-005b-4b58-a8d2-59959096a492-kube-api-access-qppgn\") pod \"test-curl-7483-140606338824064\" (UID: \"2e22dd6b-005b-4b58-a8d2-59959096a492\") " pod="opendatahub/test-curl-7483-140606338824064"
Jul 27 20:16:04.945127 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:04.945106    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qppgn\" (UniqueName: \"kubernetes.io/projected/2e22dd6b-005b-4b58-a8d2-59959096a492-kube-api-access-qppgn\") pod \"test-curl-7483-140606338824064\" (UID: \"2e22dd6b-005b-4b58-a8d2-59959096a492\") " pod="opendatahub/test-curl-7483-140606338824064"
Jul 27 20:16:05.074331 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:05.074248    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7483-140606338824064"
Jul 27 20:16:05.206138 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:05.206095    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7483-140606338824064"]
Jul 27 20:16:05.207824 ip-10-0-129-235 kubenswrapper[2558]: W0727 20:16:05.207791    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2e22dd6b_005b_4b58_a8d2_59959096a492.slice/crio-bdc7a50661e9d706190253beabe8f0e196905c640eb59df2100b48f752a709de WatchSource:0}: Error finding container bdc7a50661e9d706190253beabe8f0e196905c640eb59df2100b48f752a709de: Status 404 returned error can't find the container with id bdc7a50661e9d706190253beabe8f0e196905c640eb59df2100b48f752a709de
Jul 27 20:16:05.427808 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:05.427769    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7483-140606338824064" event={"ID":"2e22dd6b-005b-4b58-a8d2-59959096a492","Type":"ContainerStarted","Data":"bdc7a50661e9d706190253beabe8f0e196905c640eb59df2100b48f752a709de"}
Jul 27 20:16:05.429125 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:05.429092    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="49c7296860f46dc223dd77add0ed93c24c29700fb7836d7fdf9ee516c1f2ceda"
Jul 27 20:16:05.429125 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:05.429108    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7483-140606343270960"
Jul 27 20:16:05.440035 ip-10-0-129-235 kubenswrapper[2558]: E0727 20:16:05.440002    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7483-140606343270960\" is forbidden: User \"system:node:ip-10-0-129-235.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-235.ec2.internal' and this object" podUID="ca7b5cb0-1e3a-4916-9765-f42a8948a89a" pod="opendatahub/test-curl-7483-140606343270960"
Jul 27 20:16:05.674919 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:05.674881    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca7b5cb0-1e3a-4916-9765-f42a8948a89a" path="/var/lib/kubelet/pods/ca7b5cb0-1e3a-4916-9765-f42a8948a89a/volumes"
Jul 27 20:16:06.435745 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:06.435709    2558 generic.go:356] "Generic (PLEG): container finished" podID="2e22dd6b-005b-4b58-a8d2-59959096a492" containerID="47529c3d479664e76037692c4847d494557be82e5726db53ff9470ad0df7bc80" exitCode=0
Jul 27 20:16:06.435946 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:06.435801    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7483-140606338824064" event={"ID":"2e22dd6b-005b-4b58-a8d2-59959096a492","Type":"ContainerDied","Data":"47529c3d479664e76037692c4847d494557be82e5726db53ff9470ad0df7bc80"}
Jul 27 20:16:07.580173 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:07.580143    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7483-140606338824064"
Jul 27 20:16:07.599820 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:07.599784    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7483-140606338824064_2e22dd6b-005b-4b58-a8d2-59959096a492/test-curl-7483-140606338824064/0.log"
Jul 27 20:16:07.625158 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:07.625111    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7483-140606338824064"]
Jul 27 20:16:07.627602 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:07.627570    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7483-140606338824064"]
Jul 27 20:16:07.660804 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:07.660763    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qppgn\" (UniqueName: \"kubernetes.io/projected/2e22dd6b-005b-4b58-a8d2-59959096a492-kube-api-access-qppgn\") pod \"2e22dd6b-005b-4b58-a8d2-59959096a492\" (UID: \"2e22dd6b-005b-4b58-a8d2-59959096a492\") "
Jul 27 20:16:07.663121 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:07.663091    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e22dd6b-005b-4b58-a8d2-59959096a492-kube-api-access-qppgn" (OuterVolumeSpecName: "kube-api-access-qppgn") pod "2e22dd6b-005b-4b58-a8d2-59959096a492" (UID: "2e22dd6b-005b-4b58-a8d2-59959096a492"). InnerVolumeSpecName "kube-api-access-qppgn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:16:07.675146 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:07.675113    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e22dd6b-005b-4b58-a8d2-59959096a492" path="/var/lib/kubelet/pods/2e22dd6b-005b-4b58-a8d2-59959096a492/volumes"
Jul 27 20:16:07.746063 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:07.745975    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7483-140606338824064"]
Jul 27 20:16:07.746381 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:07.746367    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e22dd6b-005b-4b58-a8d2-59959096a492" containerName="test-curl-7483-140606338824064"
Jul 27 20:16:07.746442 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:07.746382    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e22dd6b-005b-4b58-a8d2-59959096a492" containerName="test-curl-7483-140606338824064"
Jul 27 20:16:07.746493 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:07.746482    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e22dd6b-005b-4b58-a8d2-59959096a492" containerName="test-curl-7483-140606338824064"
Jul 27 20:16:07.749657 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:07.749636    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7483-140606338824064"
Jul 27 20:16:07.755909 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:07.755879    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7483-140606338824064"]
Jul 27 20:16:07.762474 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:07.762401    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vdkf\" (UniqueName: \"kubernetes.io/projected/8b21377b-64b4-428a-831f-537af297ef66-kube-api-access-7vdkf\") pod \"test-curl-7483-140606338824064\" (UID: \"8b21377b-64b4-428a-831f-537af297ef66\") " pod="opendatahub/test-curl-7483-140606338824064"
Jul 27 20:16:07.762658 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:07.762626    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qppgn\" (UniqueName: \"kubernetes.io/projected/2e22dd6b-005b-4b58-a8d2-59959096a492-kube-api-access-qppgn\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:16:07.863452 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:07.863374    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vdkf\" (UniqueName: \"kubernetes.io/projected/8b21377b-64b4-428a-831f-537af297ef66-kube-api-access-7vdkf\") pod \"test-curl-7483-140606338824064\" (UID: \"8b21377b-64b4-428a-831f-537af297ef66\") " pod="opendatahub/test-curl-7483-140606338824064"
Jul 27 20:16:07.871664 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:07.871637    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vdkf\" (UniqueName: \"kubernetes.io/projected/8b21377b-64b4-428a-831f-537af297ef66-kube-api-access-7vdkf\") pod \"test-curl-7483-140606338824064\" (UID: \"8b21377b-64b4-428a-831f-537af297ef66\") " pod="opendatahub/test-curl-7483-140606338824064"
Jul 27 20:16:08.061562 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:08.061467    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7483-140606338824064"
Jul 27 20:16:08.201890 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:08.201860    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7483-140606338824064"]
Jul 27 20:16:08.203727 ip-10-0-129-235 kubenswrapper[2558]: W0727 20:16:08.203691    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8b21377b_64b4_428a_831f_537af297ef66.slice/crio-2b0f32dbb49503a83e5c2acfe9ed51f109866f186b48553718849af125335d57 WatchSource:0}: Error finding container 2b0f32dbb49503a83e5c2acfe9ed51f109866f186b48553718849af125335d57: Status 404 returned error can't find the container with id 2b0f32dbb49503a83e5c2acfe9ed51f109866f186b48553718849af125335d57
Jul 27 20:16:08.447733 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:08.447703    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7483-140606338824064"
Jul 27 20:16:08.447936 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:08.447768    2558 scope.go:117] "RemoveContainer" containerID="47529c3d479664e76037692c4847d494557be82e5726db53ff9470ad0df7bc80"
Jul 27 20:16:08.449044 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:08.448916    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7483-140606338824064" event={"ID":"8b21377b-64b4-428a-831f-537af297ef66","Type":"ContainerStarted","Data":"2b0f32dbb49503a83e5c2acfe9ed51f109866f186b48553718849af125335d57"}
Jul 27 20:16:09.457235 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:09.457196    2558 generic.go:356] "Generic (PLEG): container finished" podID="8b21377b-64b4-428a-831f-537af297ef66" containerID="2e5087a046e69b6169c9ebc8d5d6127eb38a5fec1ee4ab91675a986a276abf35" exitCode=0
Jul 27 20:16:09.457674 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:09.457286    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7483-140606338824064" event={"ID":"8b21377b-64b4-428a-831f-537af297ef66","Type":"ContainerDied","Data":"2e5087a046e69b6169c9ebc8d5d6127eb38a5fec1ee4ab91675a986a276abf35"}
Jul 27 20:16:10.608868 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:10.608843    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7483-140606338824064"
Jul 27 20:16:10.627567 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:10.627543    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7483-140606338824064_8b21377b-64b4-428a-831f-537af297ef66/test-curl-7483-140606338824064/0.log"
Jul 27 20:16:10.652154 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:10.652115    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7483-140606338824064"]
Jul 27 20:16:10.655403 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:10.655374    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7483-140606338824064"]
Jul 27 20:16:10.687688 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:10.687649    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7vdkf\" (UniqueName: \"kubernetes.io/projected/8b21377b-64b4-428a-831f-537af297ef66-kube-api-access-7vdkf\") pod \"8b21377b-64b4-428a-831f-537af297ef66\" (UID: \"8b21377b-64b4-428a-831f-537af297ef66\") "
Jul 27 20:16:10.689937 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:10.689903    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b21377b-64b4-428a-831f-537af297ef66-kube-api-access-7vdkf" (OuterVolumeSpecName: "kube-api-access-7vdkf") pod "8b21377b-64b4-428a-831f-537af297ef66" (UID: "8b21377b-64b4-428a-831f-537af297ef66"). InnerVolumeSpecName "kube-api-access-7vdkf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:16:10.767459 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:10.767364    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7483-140606343276128"]
Jul 27 20:16:10.767770 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:10.767757    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b21377b-64b4-428a-831f-537af297ef66" containerName="test-curl-7483-140606338824064"
Jul 27 20:16:10.767821 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:10.767773    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b21377b-64b4-428a-831f-537af297ef66" containerName="test-curl-7483-140606338824064"
Jul 27 20:16:10.767855 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:10.767843    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b21377b-64b4-428a-831f-537af297ef66" containerName="test-curl-7483-140606338824064"
Jul 27 20:16:10.770876 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:10.770859    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7483-140606343276128"
Jul 27 20:16:10.775401 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:10.775373    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7483-140606343276128"]
Jul 27 20:16:10.789028 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:10.788988    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hj9p\" (UniqueName: \"kubernetes.io/projected/efdf9715-5663-421b-8322-298c8431b048-kube-api-access-2hj9p\") pod \"test-curl-7483-140606343276128\" (UID: \"efdf9715-5663-421b-8322-298c8431b048\") " pod="opendatahub/test-curl-7483-140606343276128"
Jul 27 20:16:10.789199 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:10.789176    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7vdkf\" (UniqueName: \"kubernetes.io/projected/8b21377b-64b4-428a-831f-537af297ef66-kube-api-access-7vdkf\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:16:10.890390 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:10.890336    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hj9p\" (UniqueName: \"kubernetes.io/projected/efdf9715-5663-421b-8322-298c8431b048-kube-api-access-2hj9p\") pod \"test-curl-7483-140606343276128\" (UID: \"efdf9715-5663-421b-8322-298c8431b048\") " pod="opendatahub/test-curl-7483-140606343276128"
Jul 27 20:16:10.898279 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:10.898251    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hj9p\" (UniqueName: \"kubernetes.io/projected/efdf9715-5663-421b-8322-298c8431b048-kube-api-access-2hj9p\") pod \"test-curl-7483-140606343276128\" (UID: \"efdf9715-5663-421b-8322-298c8431b048\") " pod="opendatahub/test-curl-7483-140606343276128"
Jul 27 20:16:11.083285 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:11.083190    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7483-140606343276128"
Jul 27 20:16:11.231787 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:11.231747    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7483-140606343276128"]
Jul 27 20:16:11.235435 ip-10-0-129-235 kubenswrapper[2558]: W0727 20:16:11.235385    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podefdf9715_5663_421b_8322_298c8431b048.slice/crio-a5ff69169075d84d820c01d94a2106466876ca3b8b19e2a54e7ef7889d890092 WatchSource:0}: Error finding container a5ff69169075d84d820c01d94a2106466876ca3b8b19e2a54e7ef7889d890092: Status 404 returned error can't find the container with id a5ff69169075d84d820c01d94a2106466876ca3b8b19e2a54e7ef7889d890092
Jul 27 20:16:11.469940 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:11.469907    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7483-140606338824064"
Jul 27 20:16:11.469940 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:11.469929    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2b0f32dbb49503a83e5c2acfe9ed51f109866f186b48553718849af125335d57"
Jul 27 20:16:11.471118 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:11.471094    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7483-140606343276128" event={"ID":"efdf9715-5663-421b-8322-298c8431b048","Type":"ContainerStarted","Data":"a5ff69169075d84d820c01d94a2106466876ca3b8b19e2a54e7ef7889d890092"}
Jul 27 20:16:11.480682 ip-10-0-129-235 kubenswrapper[2558]: E0727 20:16:11.480655    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7483-140606338824064\" is forbidden: User \"system:node:ip-10-0-129-235.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-235.ec2.internal' and this object" podUID="8b21377b-64b4-428a-831f-537af297ef66" pod="opendatahub/test-curl-7483-140606338824064"
Jul 27 20:16:11.675405 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:11.675371    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b21377b-64b4-428a-831f-537af297ef66" path="/var/lib/kubelet/pods/8b21377b-64b4-428a-831f-537af297ef66/volumes"
Jul 27 20:16:12.019893 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:12.019847    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-d3c546/rate-test-model-d3c546-kserve-56476c7fd7-sds7w" podUID="e4258f71-8a85-4b71-a5de-b842ef753747" containerName="main" containerID="cri-o://cfb8e8bc48d7846aef6b9bdd652efd65d6a377d7019793626b382dae4592b106" gracePeriod=45
Jul 27 20:16:12.308701 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:12.308671    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d3c546/rate-test-model-d3c546-kserve-56476c7fd7-sds7w"
Jul 27 20:16:12.404388 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:12.404346    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4258f71-8a85-4b71-a5de-b842ef753747-tmp-dir\") pod \"e4258f71-8a85-4b71-a5de-b842ef753747\" (UID: \"e4258f71-8a85-4b71-a5de-b842ef753747\") "
Jul 27 20:16:12.404388 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:12.404392    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4258f71-8a85-4b71-a5de-b842ef753747-tls-certs\") pod \"e4258f71-8a85-4b71-a5de-b842ef753747\" (UID: \"e4258f71-8a85-4b71-a5de-b842ef753747\") "
Jul 27 20:16:12.404689 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:12.404435    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4258f71-8a85-4b71-a5de-b842ef753747-home\") pod \"e4258f71-8a85-4b71-a5de-b842ef753747\" (UID: \"e4258f71-8a85-4b71-a5de-b842ef753747\") "
Jul 27 20:16:12.404689 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:12.404482    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4258f71-8a85-4b71-a5de-b842ef753747-model-cache\") pod \"e4258f71-8a85-4b71-a5de-b842ef753747\" (UID: \"e4258f71-8a85-4b71-a5de-b842ef753747\") "
Jul 27 20:16:12.404689 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:12.404556    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x2wg7\" (UniqueName: \"kubernetes.io/projected/e4258f71-8a85-4b71-a5de-b842ef753747-kube-api-access-x2wg7\") pod \"e4258f71-8a85-4b71-a5de-b842ef753747\" (UID: \"e4258f71-8a85-4b71-a5de-b842ef753747\") "
Jul 27 20:16:12.404689 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:12.404594    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4258f71-8a85-4b71-a5de-b842ef753747-dshm\") pod \"e4258f71-8a85-4b71-a5de-b842ef753747\" (UID: \"e4258f71-8a85-4b71-a5de-b842ef753747\") "
Jul 27 20:16:12.404886 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:12.404714    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4258f71-8a85-4b71-a5de-b842ef753747-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e4258f71-8a85-4b71-a5de-b842ef753747" (UID: "e4258f71-8a85-4b71-a5de-b842ef753747"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:16:12.404886 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:12.404723    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4258f71-8a85-4b71-a5de-b842ef753747-model-cache" (OuterVolumeSpecName: "model-cache") pod "e4258f71-8a85-4b71-a5de-b842ef753747" (UID: "e4258f71-8a85-4b71-a5de-b842ef753747"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:16:12.404886 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:12.404740    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4258f71-8a85-4b71-a5de-b842ef753747-home" (OuterVolumeSpecName: "home") pod "e4258f71-8a85-4b71-a5de-b842ef753747" (UID: "e4258f71-8a85-4b71-a5de-b842ef753747"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:16:12.405028 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:12.404943    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4258f71-8a85-4b71-a5de-b842ef753747-tmp-dir\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:16:12.405028 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:12.404964    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4258f71-8a85-4b71-a5de-b842ef753747-home\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:16:12.405028 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:12.404980    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4258f71-8a85-4b71-a5de-b842ef753747-model-cache\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:16:12.406710 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:12.406687    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e4258f71-8a85-4b71-a5de-b842ef753747-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e4258f71-8a85-4b71-a5de-b842ef753747" (UID: "e4258f71-8a85-4b71-a5de-b842ef753747"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 20:16:12.406790 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:12.406732    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4258f71-8a85-4b71-a5de-b842ef753747-dshm" (OuterVolumeSpecName: "dshm") pod "e4258f71-8a85-4b71-a5de-b842ef753747" (UID: "e4258f71-8a85-4b71-a5de-b842ef753747"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:16:12.406830 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:12.406795    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4258f71-8a85-4b71-a5de-b842ef753747-kube-api-access-x2wg7" (OuterVolumeSpecName: "kube-api-access-x2wg7") pod "e4258f71-8a85-4b71-a5de-b842ef753747" (UID: "e4258f71-8a85-4b71-a5de-b842ef753747"). InnerVolumeSpecName "kube-api-access-x2wg7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:16:12.478422 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:12.478386    2558 generic.go:356] "Generic (PLEG): container finished" podID="efdf9715-5663-421b-8322-298c8431b048" containerID="0a2ba886fe0a6c06036376e34cb83c1ff4b47f47c668d77284d181ee9a91e874" exitCode=0
Jul 27 20:16:12.478591 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:12.478474    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7483-140606343276128" event={"ID":"efdf9715-5663-421b-8322-298c8431b048","Type":"ContainerDied","Data":"0a2ba886fe0a6c06036376e34cb83c1ff4b47f47c668d77284d181ee9a91e874"}
Jul 27 20:16:12.479747 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:12.479725    2558 generic.go:356] "Generic (PLEG): container finished" podID="e4258f71-8a85-4b71-a5de-b842ef753747" containerID="cfb8e8bc48d7846aef6b9bdd652efd65d6a377d7019793626b382dae4592b106" exitCode=0
Jul 27 20:16:12.479841 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:12.479759    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d3c546/rate-test-model-d3c546-kserve-56476c7fd7-sds7w"
Jul 27 20:16:12.479841 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:12.479807    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d3c546/rate-test-model-d3c546-kserve-56476c7fd7-sds7w" event={"ID":"e4258f71-8a85-4b71-a5de-b842ef753747","Type":"ContainerDied","Data":"cfb8e8bc48d7846aef6b9bdd652efd65d6a377d7019793626b382dae4592b106"}
Jul 27 20:16:12.479916 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:12.479847    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d3c546/rate-test-model-d3c546-kserve-56476c7fd7-sds7w" event={"ID":"e4258f71-8a85-4b71-a5de-b842ef753747","Type":"ContainerDied","Data":"02d89b1a48a59e009a9f9e1cbd33bb72e94cdea33b500aa2d163edea5bee502e"}
Jul 27 20:16:12.479916 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:12.479867    2558 scope.go:117] "RemoveContainer" containerID="cfb8e8bc48d7846aef6b9bdd652efd65d6a377d7019793626b382dae4592b106"
Jul 27 20:16:12.489382 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:12.489367    2558 scope.go:117] "RemoveContainer" containerID="cfb8e8bc48d7846aef6b9bdd652efd65d6a377d7019793626b382dae4592b106"
Jul 27 20:16:12.489655 ip-10-0-129-235 kubenswrapper[2558]: E0727 20:16:12.489640    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cfb8e8bc48d7846aef6b9bdd652efd65d6a377d7019793626b382dae4592b106\": container with ID starting with cfb8e8bc48d7846aef6b9bdd652efd65d6a377d7019793626b382dae4592b106 not found: ID does not exist" containerID="cfb8e8bc48d7846aef6b9bdd652efd65d6a377d7019793626b382dae4592b106"
Jul 27 20:16:12.489731 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:12.489661    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cfb8e8bc48d7846aef6b9bdd652efd65d6a377d7019793626b382dae4592b106"} err="failed to get container status \"cfb8e8bc48d7846aef6b9bdd652efd65d6a377d7019793626b382dae4592b106\": rpc error: code = NotFound desc = could not find container \"cfb8e8bc48d7846aef6b9bdd652efd65d6a377d7019793626b382dae4592b106\": container with ID starting with cfb8e8bc48d7846aef6b9bdd652efd65d6a377d7019793626b382dae4592b106 not found: ID does not exist"
Jul 27 20:16:12.503619 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:12.503597    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-d3c546/rate-test-model-d3c546-kserve-56476c7fd7-sds7w"]
Jul 27 20:16:12.506246 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:12.506221    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4258f71-8a85-4b71-a5de-b842ef753747-dshm\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:16:12.506335 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:12.506250    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4258f71-8a85-4b71-a5de-b842ef753747-tls-certs\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:16:12.506335 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:12.506265    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x2wg7\" (UniqueName: \"kubernetes.io/projected/e4258f71-8a85-4b71-a5de-b842ef753747-kube-api-access-x2wg7\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:16:12.507432 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:12.507399    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-d3c546/rate-test-model-d3c546-kserve-56476c7fd7-sds7w"]
Jul 27 20:16:13.627956 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:13.627929    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7483-140606343276128"
Jul 27 20:16:13.646828 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:13.646803    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7483-140606343276128_efdf9715-5663-421b-8322-298c8431b048/test-curl-7483-140606343276128/0.log"
Jul 27 20:16:13.678919 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:13.678887    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4258f71-8a85-4b71-a5de-b842ef753747" path="/var/lib/kubelet/pods/e4258f71-8a85-4b71-a5de-b842ef753747/volumes"
Jul 27 20:16:13.680788 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:13.680768    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7483-140606343276128"]
Jul 27 20:16:13.680905 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:13.680801    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7483-140606343276128"]
Jul 27 20:16:13.715881 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:13.715858    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2hj9p\" (UniqueName: \"kubernetes.io/projected/efdf9715-5663-421b-8322-298c8431b048-kube-api-access-2hj9p\") pod \"efdf9715-5663-421b-8322-298c8431b048\" (UID: \"efdf9715-5663-421b-8322-298c8431b048\") "
Jul 27 20:16:13.718134 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:13.718112    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efdf9715-5663-421b-8322-298c8431b048-kube-api-access-2hj9p" (OuterVolumeSpecName: "kube-api-access-2hj9p") pod "efdf9715-5663-421b-8322-298c8431b048" (UID: "efdf9715-5663-421b-8322-298c8431b048"). InnerVolumeSpecName "kube-api-access-2hj9p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:16:13.788401 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:13.788341    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7483-140606343276128"]
Jul 27 20:16:13.788774 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:13.788759    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efdf9715-5663-421b-8322-298c8431b048" containerName="test-curl-7483-140606343276128"
Jul 27 20:16:13.788823 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:13.788776    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="efdf9715-5663-421b-8322-298c8431b048" containerName="test-curl-7483-140606343276128"
Jul 27 20:16:13.788823 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:13.788788    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4258f71-8a85-4b71-a5de-b842ef753747" containerName="main"
Jul 27 20:16:13.788823 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:13.788793    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4258f71-8a85-4b71-a5de-b842ef753747" containerName="main"
Jul 27 20:16:13.788911 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:13.788848    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="efdf9715-5663-421b-8322-298c8431b048" containerName="test-curl-7483-140606343276128"
Jul 27 20:16:13.788911 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:13.788857    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4258f71-8a85-4b71-a5de-b842ef753747" containerName="main"
Jul 27 20:16:13.791986 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:13.791972    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7483-140606343276128"
Jul 27 20:16:13.797924 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:13.797894    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7483-140606343276128"]
Jul 27 20:16:13.816708 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:13.816685    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rx84f\" (UniqueName: \"kubernetes.io/projected/b64f0a02-8cf1-42b8-8a9d-a0b3ce8645d0-kube-api-access-rx84f\") pod \"test-curl-7483-140606343276128\" (UID: \"b64f0a02-8cf1-42b8-8a9d-a0b3ce8645d0\") " pod="opendatahub/test-curl-7483-140606343276128"
Jul 27 20:16:13.816826 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:13.816808    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2hj9p\" (UniqueName: \"kubernetes.io/projected/efdf9715-5663-421b-8322-298c8431b048-kube-api-access-2hj9p\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:16:13.917539 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:13.917502    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rx84f\" (UniqueName: \"kubernetes.io/projected/b64f0a02-8cf1-42b8-8a9d-a0b3ce8645d0-kube-api-access-rx84f\") pod \"test-curl-7483-140606343276128\" (UID: \"b64f0a02-8cf1-42b8-8a9d-a0b3ce8645d0\") " pod="opendatahub/test-curl-7483-140606343276128"
Jul 27 20:16:13.925202 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:13.925175    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rx84f\" (UniqueName: \"kubernetes.io/projected/b64f0a02-8cf1-42b8-8a9d-a0b3ce8645d0-kube-api-access-rx84f\") pod \"test-curl-7483-140606343276128\" (UID: \"b64f0a02-8cf1-42b8-8a9d-a0b3ce8645d0\") " pod="opendatahub/test-curl-7483-140606343276128"
Jul 27 20:16:14.102643 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:14.102569    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7483-140606343276128"
Jul 27 20:16:14.240604 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:14.240579    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7483-140606343276128"]
Jul 27 20:16:14.242248 ip-10-0-129-235 kubenswrapper[2558]: W0727 20:16:14.242221    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb64f0a02_8cf1_42b8_8a9d_a0b3ce8645d0.slice/crio-e5e0fa3110ff608104e29548839e61a8a21c18bececb7e3e5cd17195c60d3f97 WatchSource:0}: Error finding container e5e0fa3110ff608104e29548839e61a8a21c18bececb7e3e5cd17195c60d3f97: Status 404 returned error can't find the container with id e5e0fa3110ff608104e29548839e61a8a21c18bececb7e3e5cd17195c60d3f97
Jul 27 20:16:14.493222 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:14.493195    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7483-140606343276128"
Jul 27 20:16:14.493397 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:14.493195    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a5ff69169075d84d820c01d94a2106466876ca3b8b19e2a54e7ef7889d890092"
Jul 27 20:16:14.494439 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:14.494396    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7483-140606343276128" event={"ID":"b64f0a02-8cf1-42b8-8a9d-a0b3ce8645d0","Type":"ContainerStarted","Data":"e5e0fa3110ff608104e29548839e61a8a21c18bececb7e3e5cd17195c60d3f97"}
Jul 27 20:16:14.504188 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:14.504155    2558 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7483-140606343276128" oldPodUID="efdf9715-5663-421b-8322-298c8431b048" podUID="b64f0a02-8cf1-42b8-8a9d-a0b3ce8645d0"
Jul 27 20:16:15.509005 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:15.508960    2558 generic.go:356] "Generic (PLEG): container finished" podID="b64f0a02-8cf1-42b8-8a9d-a0b3ce8645d0" containerID="2e6eadccc513608e25f922ec85e9d1931a9296a98b15fdb9cae742759965c13f" exitCode=0
Jul 27 20:16:15.509483 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:15.509001    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7483-140606343276128" event={"ID":"b64f0a02-8cf1-42b8-8a9d-a0b3ce8645d0","Type":"ContainerDied","Data":"2e6eadccc513608e25f922ec85e9d1931a9296a98b15fdb9cae742759965c13f"}
Jul 27 20:16:15.675203 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:15.675163    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="efdf9715-5663-421b-8322-298c8431b048" path="/var/lib/kubelet/pods/efdf9715-5663-421b-8322-298c8431b048/volumes"
Jul 27 20:16:16.662132 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:16.662102    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7483-140606343276128"
Jul 27 20:16:16.681689 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:16.681653    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7483-140606343276128_b64f0a02-8cf1-42b8-8a9d-a0b3ce8645d0/test-curl-7483-140606343276128/0.log"
Jul 27 20:16:16.705644 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:16.705599    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7483-140606343276128"]
Jul 27 20:16:16.708458 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:16.708402    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7483-140606343276128"]
Jul 27 20:16:16.742286 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:16.742258    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rx84f\" (UniqueName: \"kubernetes.io/projected/b64f0a02-8cf1-42b8-8a9d-a0b3ce8645d0-kube-api-access-rx84f\") pod \"b64f0a02-8cf1-42b8-8a9d-a0b3ce8645d0\" (UID: \"b64f0a02-8cf1-42b8-8a9d-a0b3ce8645d0\") "
Jul 27 20:16:16.744697 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:16.744662    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b64f0a02-8cf1-42b8-8a9d-a0b3ce8645d0-kube-api-access-rx84f" (OuterVolumeSpecName: "kube-api-access-rx84f") pod "b64f0a02-8cf1-42b8-8a9d-a0b3ce8645d0" (UID: "b64f0a02-8cf1-42b8-8a9d-a0b3ce8645d0"). InnerVolumeSpecName "kube-api-access-rx84f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:16:16.843362 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:16.843273    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rx84f\" (UniqueName: \"kubernetes.io/projected/b64f0a02-8cf1-42b8-8a9d-a0b3ce8645d0-kube-api-access-rx84f\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:16:17.189056 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:17.189022    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:16:17.480885 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:17.480780    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:16:17.524376 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:17.524337    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7483-140606343276128"
Jul 27 20:16:17.524376 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:17.524366    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e5e0fa3110ff608104e29548839e61a8a21c18bececb7e3e5cd17195c60d3f97"
Jul 27 20:16:17.534447 ip-10-0-129-235 kubenswrapper[2558]: E0727 20:16:17.534387    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7483-140606343276128\" is forbidden: User \"system:node:ip-10-0-129-235.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-235.ec2.internal' and this object" podUID="b64f0a02-8cf1-42b8-8a9d-a0b3ce8645d0" pod="opendatahub/test-curl-7483-140606343276128"
Jul 27 20:16:17.646730 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:17.646694    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:16:17.673568 ip-10-0-129-235 kubenswrapper[2558]: E0727 20:16:17.673522    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7483-140606343276128\" is forbidden: User \"system:node:ip-10-0-129-235.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-235.ec2.internal' and this object" podUID="b64f0a02-8cf1-42b8-8a9d-a0b3ce8645d0" pod="opendatahub/test-curl-7483-140606343276128"
Jul 27 20:16:17.675175 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:17.675152    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b64f0a02-8cf1-42b8-8a9d-a0b3ce8645d0" path="/var/lib/kubelet/pods/b64f0a02-8cf1-42b8-8a9d-a0b3ce8645d0/volumes"
Jul 27 20:16:20.192700 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:20.192665    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:16:20.288129 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:20.288095    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:16:20.345364 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:20.345328    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:16:20.446281 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:20.446169    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:16:20.968376 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:20.968337    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:16:21.073053 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:21.073018    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:16:21.081768 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:21.081725    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-018406-5fc4cbcbd8-5z95h"]
Jul 27 20:16:21.082143 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:21.082129    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b64f0a02-8cf1-42b8-8a9d-a0b3ce8645d0" containerName="test-curl-7483-140606343276128"
Jul 27 20:16:21.082188 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:21.082145    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="b64f0a02-8cf1-42b8-8a9d-a0b3ce8645d0" containerName="test-curl-7483-140606343276128"
Jul 27 20:16:21.082229 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:21.082209    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="b64f0a02-8cf1-42b8-8a9d-a0b3ce8645d0" containerName="test-curl-7483-140606343276128"
Jul 27 20:16:21.091260 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:21.091160    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-018406-5fc4cbcbd8-5z95h"
Jul 27 20:16:21.093953 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:21.093924    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-018406-dockercfg-9rwds\""
Jul 27 20:16:21.094169 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:21.093943    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-018406\""
Jul 27 20:16:21.095801 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:21.095772    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-018406-5fc4cbcbd8-5z95h"]
Jul 27 20:16:21.112897 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:21.112863    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-018406-75754c7976-84hjb"]
Jul 27 20:16:21.116371 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:21.116353    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-018406-75754c7976-84hjb"
Jul 27 20:16:21.126106 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:21.126079    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-018406-75754c7976-84hjb"]
Jul 27 20:16:21.186637 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:21.186599    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pz242\" (UniqueName: \"kubernetes.io/projected/f80492ac-8e54-4cd9-b0ab-8733548af2cc-kube-api-access-pz242\") pod \"payload-pre-processing-e2e-ipp-a-018406-5fc4cbcbd8-5z95h\" (UID: \"f80492ac-8e54-4cd9-b0ab-8733548af2cc\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-018406-5fc4cbcbd8-5z95h"
Jul 27 20:16:21.186828 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:21.186700    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/faf8bee3-858c-45e6-bbc8-9ffcd5538b1b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-018406-75754c7976-84hjb\" (UID: \"faf8bee3-858c-45e6-bbc8-9ffcd5538b1b\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-018406-75754c7976-84hjb"
Jul 27 20:16:21.186828 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:21.186754    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f80492ac-8e54-4cd9-b0ab-8733548af2cc-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-018406-5fc4cbcbd8-5z95h\" (UID: \"f80492ac-8e54-4cd9-b0ab-8733548af2cc\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-018406-5fc4cbcbd8-5z95h"
Jul 27 20:16:21.186828 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:21.186777    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmqn2\" (UniqueName: \"kubernetes.io/projected/faf8bee3-858c-45e6-bbc8-9ffcd5538b1b-kube-api-access-wmqn2\") pod \"payload-processing-e2e-ipp-a-018406-75754c7976-84hjb\" (UID: \"faf8bee3-858c-45e6-bbc8-9ffcd5538b1b\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-018406-75754c7976-84hjb"
Jul 27 20:16:21.287664 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:21.287547    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pz242\" (UniqueName: \"kubernetes.io/projected/f80492ac-8e54-4cd9-b0ab-8733548af2cc-kube-api-access-pz242\") pod \"payload-pre-processing-e2e-ipp-a-018406-5fc4cbcbd8-5z95h\" (UID: \"f80492ac-8e54-4cd9-b0ab-8733548af2cc\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-018406-5fc4cbcbd8-5z95h"
Jul 27 20:16:21.287664 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:21.287610    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/faf8bee3-858c-45e6-bbc8-9ffcd5538b1b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-018406-75754c7976-84hjb\" (UID: \"faf8bee3-858c-45e6-bbc8-9ffcd5538b1b\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-018406-75754c7976-84hjb"
Jul 27 20:16:21.287664 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:21.287654    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f80492ac-8e54-4cd9-b0ab-8733548af2cc-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-018406-5fc4cbcbd8-5z95h\" (UID: \"f80492ac-8e54-4cd9-b0ab-8733548af2cc\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-018406-5fc4cbcbd8-5z95h"
Jul 27 20:16:21.288215 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:21.287766    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmqn2\" (UniqueName: \"kubernetes.io/projected/faf8bee3-858c-45e6-bbc8-9ffcd5538b1b-kube-api-access-wmqn2\") pod \"payload-processing-e2e-ipp-a-018406-75754c7976-84hjb\" (UID: \"faf8bee3-858c-45e6-bbc8-9ffcd5538b1b\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-018406-75754c7976-84hjb"
Jul 27 20:16:21.288541 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:21.288511    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/faf8bee3-858c-45e6-bbc8-9ffcd5538b1b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-018406-75754c7976-84hjb\" (UID: \"faf8bee3-858c-45e6-bbc8-9ffcd5538b1b\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-018406-75754c7976-84hjb"
Jul 27 20:16:21.288661 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:21.288636    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f80492ac-8e54-4cd9-b0ab-8733548af2cc-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-018406-5fc4cbcbd8-5z95h\" (UID: \"f80492ac-8e54-4cd9-b0ab-8733548af2cc\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-018406-5fc4cbcbd8-5z95h"
Jul 27 20:16:21.295901 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:21.295871    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pz242\" (UniqueName: \"kubernetes.io/projected/f80492ac-8e54-4cd9-b0ab-8733548af2cc-kube-api-access-pz242\") pod \"payload-pre-processing-e2e-ipp-a-018406-5fc4cbcbd8-5z95h\" (UID: \"f80492ac-8e54-4cd9-b0ab-8733548af2cc\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-018406-5fc4cbcbd8-5z95h"
Jul 27 20:16:21.296010 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:21.295868    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmqn2\" (UniqueName: \"kubernetes.io/projected/faf8bee3-858c-45e6-bbc8-9ffcd5538b1b-kube-api-access-wmqn2\") pod \"payload-processing-e2e-ipp-a-018406-75754c7976-84hjb\" (UID: \"faf8bee3-858c-45e6-bbc8-9ffcd5538b1b\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-018406-75754c7976-84hjb"
Jul 27 20:16:21.403945 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:21.403899    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-018406-5fc4cbcbd8-5z95h"
Jul 27 20:16:21.429531 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:21.429494    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-018406-75754c7976-84hjb"
Jul 27 20:16:21.572377 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:21.572326    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-018406-5fc4cbcbd8-5z95h"]
Jul 27 20:16:21.576766 ip-10-0-129-235 kubenswrapper[2558]: W0727 20:16:21.576732    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf80492ac_8e54_4cd9_b0ab_8733548af2cc.slice/crio-5c81a099653f27fe2afdca43e8f78ee418c6c645045bade0f91e6672cfeca446 WatchSource:0}: Error finding container 5c81a099653f27fe2afdca43e8f78ee418c6c645045bade0f91e6672cfeca446: Status 404 returned error can't find the container with id 5c81a099653f27fe2afdca43e8f78ee418c6c645045bade0f91e6672cfeca446
Jul 27 20:16:21.603734 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:21.603701    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-018406-75754c7976-84hjb"]
Jul 27 20:16:21.613667 ip-10-0-129-235 kubenswrapper[2558]: W0727 20:16:21.613637    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfaf8bee3_858c_45e6_bbc8_9ffcd5538b1b.slice/crio-6051f94a80ef1240fbf86f4a8fa716991a4e2ae1a7529b3078a8208b9a40e867 WatchSource:0}: Error finding container 6051f94a80ef1240fbf86f4a8fa716991a4e2ae1a7529b3078a8208b9a40e867: Status 404 returned error can't find the container with id 6051f94a80ef1240fbf86f4a8fa716991a4e2ae1a7529b3078a8208b9a40e867
Jul 27 20:16:22.556452 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:22.556400    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-018406-5fc4cbcbd8-5z95h" event={"ID":"f80492ac-8e54-4cd9-b0ab-8733548af2cc","Type":"ContainerStarted","Data":"c9f14a16d134e39055ec6aac4f4819da32a482240cbe29a222dd1e98c4d358ba"}
Jul 27 20:16:22.556942 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:22.556566    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-018406-5fc4cbcbd8-5z95h" event={"ID":"f80492ac-8e54-4cd9-b0ab-8733548af2cc","Type":"ContainerStarted","Data":"5c81a099653f27fe2afdca43e8f78ee418c6c645045bade0f91e6672cfeca446"}
Jul 27 20:16:22.556942 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:22.556622    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-018406-5fc4cbcbd8-5z95h"
Jul 27 20:16:22.558104 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:22.558078    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-018406-75754c7976-84hjb" event={"ID":"faf8bee3-858c-45e6-bbc8-9ffcd5538b1b","Type":"ContainerStarted","Data":"5b54b1a60848c8015b212df70bcadb824f68acb33251a080a1e23b40407d06d2"}
Jul 27 20:16:22.558237 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:22.558128    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-018406-75754c7976-84hjb" event={"ID":"faf8bee3-858c-45e6-bbc8-9ffcd5538b1b","Type":"ContainerStarted","Data":"6051f94a80ef1240fbf86f4a8fa716991a4e2ae1a7529b3078a8208b9a40e867"}
Jul 27 20:16:22.558237 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:22.558158    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-018406-75754c7976-84hjb"
Jul 27 20:16:22.572993 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:22.572931    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-018406-5fc4cbcbd8-5z95h" podStartSLOduration=1.572915161 podStartE2EDuration="1.572915161s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:16:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:16:22.571672601 +0000 UTC m=+2995.443864576" watchObservedRunningTime="2026-07-27 20:16:22.572915161 +0000 UTC m=+2995.445107134"
Jul 27 20:16:22.589169 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:22.589104    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-018406-75754c7976-84hjb" podStartSLOduration=1.5890853969999998 podStartE2EDuration="1.589085397s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:16:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:16:22.58697123 +0000 UTC m=+2995.459163205" watchObservedRunningTime="2026-07-27 20:16:22.589085397 +0000 UTC m=+2995.461277637"
Jul 27 20:16:24.698167 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:24.698131    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:16:27.843392 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:27.843365    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5dfc2_103bab23-bca3-472a-974b-7731f83c5bfb/ovn-acl-logging/0.log"
Jul 27 20:16:27.853867 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:27.853841    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5dfc2_103bab23-bca3-472a-974b-7731f83c5bfb/ovn-acl-logging/0.log"
Jul 27 20:16:33.565632 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:33.565590    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-018406-5fc4cbcbd8-5z95h"
Jul 27 20:16:33.566060 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:33.565772    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-018406-75754c7976-84hjb"
Jul 27 20:16:35.579578 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:35.579543    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:16:35.860274 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:35.860185    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:16:36.045579 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:36.045529    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:16:38.588263 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:38.588225    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:16:38.688790 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:38.688751    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:16:38.850199 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:38.850073    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:16:39.142161 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:39.142118    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:16:39.405194 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:39.405057    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:16:39.510390 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:39.510355    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:16:39.515757 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:39.515724    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-933254-78765ccbd4-m6259"]
Jul 27 20:16:39.519759 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:39.519729    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-933254-78765ccbd4-m6259"
Jul 27 20:16:39.522142 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:39.522109    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-933254\""
Jul 27 20:16:39.522320 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:39.522109    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-933254-dockercfg-5hc5k\""
Jul 27 20:16:39.530090 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:39.530061    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-933254-78765ccbd4-m6259"]
Jul 27 20:16:39.543691 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:39.543653    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-933254-c4b5858d4-8qtk2"]
Jul 27 20:16:39.547319 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:39.547281    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-933254-c4b5858d4-8qtk2"
Jul 27 20:16:39.556523 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:39.556493    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-933254-c4b5858d4-8qtk2"]
Jul 27 20:16:39.659027 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:39.658892    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:16:39.669352 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:39.669310    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/448763ca-737d-4aab-8d3a-d26d0f309f52-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-933254-c4b5858d4-8qtk2\" (UID: \"448763ca-737d-4aab-8d3a-d26d0f309f52\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-933254-c4b5858d4-8qtk2"
Jul 27 20:16:39.669584 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:39.669371    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd278b5b-82dd-4048-88f2-1487bb9759e1-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-933254-78765ccbd4-m6259\" (UID: \"dd278b5b-82dd-4048-88f2-1487bb9759e1\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-933254-78765ccbd4-m6259"
Jul 27 20:16:39.669584 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:39.669507    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvrjf\" (UniqueName: \"kubernetes.io/projected/448763ca-737d-4aab-8d3a-d26d0f309f52-kube-api-access-kvrjf\") pod \"payload-processing-e2e-ipp-b-933254-c4b5858d4-8qtk2\" (UID: \"448763ca-737d-4aab-8d3a-d26d0f309f52\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-933254-c4b5858d4-8qtk2"
Jul 27 20:16:39.669686 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:39.669611    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vpg8\" (UniqueName: \"kubernetes.io/projected/dd278b5b-82dd-4048-88f2-1487bb9759e1-kube-api-access-7vpg8\") pod \"payload-pre-processing-e2e-ipp-b-933254-78765ccbd4-m6259\" (UID: \"dd278b5b-82dd-4048-88f2-1487bb9759e1\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-933254-78765ccbd4-m6259"
Jul 27 20:16:39.720014 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:39.719976    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:16:39.771045 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:39.770989    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvrjf\" (UniqueName: \"kubernetes.io/projected/448763ca-737d-4aab-8d3a-d26d0f309f52-kube-api-access-kvrjf\") pod \"payload-processing-e2e-ipp-b-933254-c4b5858d4-8qtk2\" (UID: \"448763ca-737d-4aab-8d3a-d26d0f309f52\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-933254-c4b5858d4-8qtk2"
Jul 27 20:16:39.771265 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:39.771102    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vpg8\" (UniqueName: \"kubernetes.io/projected/dd278b5b-82dd-4048-88f2-1487bb9759e1-kube-api-access-7vpg8\") pod \"payload-pre-processing-e2e-ipp-b-933254-78765ccbd4-m6259\" (UID: \"dd278b5b-82dd-4048-88f2-1487bb9759e1\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-933254-78765ccbd4-m6259"
Jul 27 20:16:39.771265 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:39.771143    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/448763ca-737d-4aab-8d3a-d26d0f309f52-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-933254-c4b5858d4-8qtk2\" (UID: \"448763ca-737d-4aab-8d3a-d26d0f309f52\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-933254-c4b5858d4-8qtk2"
Jul 27 20:16:39.771265 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:39.771173    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd278b5b-82dd-4048-88f2-1487bb9759e1-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-933254-78765ccbd4-m6259\" (UID: \"dd278b5b-82dd-4048-88f2-1487bb9759e1\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-933254-78765ccbd4-m6259"
Jul 27 20:16:39.771903 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:39.771885    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd278b5b-82dd-4048-88f2-1487bb9759e1-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-933254-78765ccbd4-m6259\" (UID: \"dd278b5b-82dd-4048-88f2-1487bb9759e1\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-933254-78765ccbd4-m6259"
Jul 27 20:16:39.771954 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:39.771933    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/448763ca-737d-4aab-8d3a-d26d0f309f52-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-933254-c4b5858d4-8qtk2\" (UID: \"448763ca-737d-4aab-8d3a-d26d0f309f52\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-933254-c4b5858d4-8qtk2"
Jul 27 20:16:39.779630 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:39.779595    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvrjf\" (UniqueName: \"kubernetes.io/projected/448763ca-737d-4aab-8d3a-d26d0f309f52-kube-api-access-kvrjf\") pod \"payload-processing-e2e-ipp-b-933254-c4b5858d4-8qtk2\" (UID: \"448763ca-737d-4aab-8d3a-d26d0f309f52\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-933254-c4b5858d4-8qtk2"
Jul 27 20:16:39.779785 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:39.779611    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vpg8\" (UniqueName: \"kubernetes.io/projected/dd278b5b-82dd-4048-88f2-1487bb9759e1-kube-api-access-7vpg8\") pod \"payload-pre-processing-e2e-ipp-b-933254-78765ccbd4-m6259\" (UID: \"dd278b5b-82dd-4048-88f2-1487bb9759e1\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-933254-78765ccbd4-m6259"
Jul 27 20:16:39.832182 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:39.832148    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-933254-78765ccbd4-m6259"
Jul 27 20:16:39.860629 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:39.860599    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-933254-c4b5858d4-8qtk2"
Jul 27 20:16:39.995834 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:39.995794    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-933254-78765ccbd4-m6259"]
Jul 27 20:16:39.997857 ip-10-0-129-235 kubenswrapper[2558]: W0727 20:16:39.997824    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd278b5b_82dd_4048_88f2_1487bb9759e1.slice/crio-e572d98b185cb650342eb82c6e87f03164edfc2894456d28efaea1931bdfc326 WatchSource:0}: Error finding container e572d98b185cb650342eb82c6e87f03164edfc2894456d28efaea1931bdfc326: Status 404 returned error can't find the container with id e572d98b185cb650342eb82c6e87f03164edfc2894456d28efaea1931bdfc326
Jul 27 20:16:40.030854 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:40.030808    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-933254-c4b5858d4-8qtk2"]
Jul 27 20:16:40.041774 ip-10-0-129-235 kubenswrapper[2558]: W0727 20:16:40.041745    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod448763ca_737d_4aab_8d3a_d26d0f309f52.slice/crio-a4c98566ce4ec03197c064e5765c88f246026a9deca4f14ed2a6690438bc3ccb WatchSource:0}: Error finding container a4c98566ce4ec03197c064e5765c88f246026a9deca4f14ed2a6690438bc3ccb: Status 404 returned error can't find the container with id a4c98566ce4ec03197c064e5765c88f246026a9deca4f14ed2a6690438bc3ccb
Jul 27 20:16:40.669663 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:40.669631    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-933254-c4b5858d4-8qtk2" event={"ID":"448763ca-737d-4aab-8d3a-d26d0f309f52","Type":"ContainerStarted","Data":"d46a44c6fba3411ed0c2dc911dcabbac570a27cdec5e8dfcfb35d628c968de9f"}
Jul 27 20:16:40.670118 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:40.669678    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-933254-c4b5858d4-8qtk2" event={"ID":"448763ca-737d-4aab-8d3a-d26d0f309f52","Type":"ContainerStarted","Data":"a4c98566ce4ec03197c064e5765c88f246026a9deca4f14ed2a6690438bc3ccb"}
Jul 27 20:16:40.670118 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:40.669719    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-933254-c4b5858d4-8qtk2"
Jul 27 20:16:40.671230 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:40.671205    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-933254-78765ccbd4-m6259" event={"ID":"dd278b5b-82dd-4048-88f2-1487bb9759e1","Type":"ContainerStarted","Data":"d36d389f9edabfc27b052e344decb46594aa08f55dad87f41713ac00b4c13004"}
Jul 27 20:16:40.671349 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:40.671245    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-933254-78765ccbd4-m6259" event={"ID":"dd278b5b-82dd-4048-88f2-1487bb9759e1","Type":"ContainerStarted","Data":"e572d98b185cb650342eb82c6e87f03164edfc2894456d28efaea1931bdfc326"}
Jul 27 20:16:40.671349 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:40.671294    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-933254-78765ccbd4-m6259"
Jul 27 20:16:40.686074 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:40.686024    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-933254-c4b5858d4-8qtk2" podStartSLOduration=1.686006349 podStartE2EDuration="1.686006349s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:16:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:16:40.684929613 +0000 UTC m=+3013.557121587" watchObservedRunningTime="2026-07-27 20:16:40.686006349 +0000 UTC m=+3013.558198359"
Jul 27 20:16:40.701546 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:40.701479    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-933254-78765ccbd4-m6259" podStartSLOduration=1.701462524 podStartE2EDuration="1.701462524s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:16:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:16:40.699321215 +0000 UTC m=+3013.571513190" watchObservedRunningTime="2026-07-27 20:16:40.701462524 +0000 UTC m=+3013.573654498"
Jul 27 20:16:43.073449 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:43.073398    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:16:51.679204 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:51.679170    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-933254-c4b5858d4-8qtk2"
Jul 27 20:16:51.679734 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:16:51.679286    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-933254-78765ccbd4-m6259"
Jul 27 20:17:00.209797 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:00.209761    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-933254-c4b5858d4-8qtk2_448763ca-737d-4aab-8d3a-d26d0f309f52/payload-processing/0.log"
Jul 27 20:17:02.723992 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:02.723954    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-018406/ipp-route-018406-kserve-5cb6d69f4c-n2whm"]
Jul 27 20:17:02.728915 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:02.728898    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-018406/ipp-route-018406-kserve-5cb6d69f4c-n2whm"
Jul 27 20:17:02.732092 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:02.732071    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-018406\"/\"ipp-route-018406-kserve-self-signed-certs\""
Jul 27 20:17:02.732234 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:02.732139    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-018406\"/\"openshift-service-ca.crt\""
Jul 27 20:17:02.732234 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:02.732180    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-018406\"/\"default-dockercfg-lvmkc\""
Jul 27 20:17:02.732234 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:02.732203    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-018406\"/\"kube-root-ca.crt\""
Jul 27 20:17:02.739602 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:02.739577    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-018406/ipp-route-018406-kserve-5cb6d69f4c-n2whm"]
Jul 27 20:17:02.758970 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:02.758939    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:17:02.789840 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:02.789800    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qsqq\" (UniqueName: \"kubernetes.io/projected/449325e2-981e-4358-adfa-75608a671d84-kube-api-access-6qsqq\") pod \"ipp-route-018406-kserve-5cb6d69f4c-n2whm\" (UID: \"449325e2-981e-4358-adfa-75608a671d84\") " pod="ai-tenant-e2e-ipp-a-018406/ipp-route-018406-kserve-5cb6d69f4c-n2whm"
Jul 27 20:17:02.789965 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:02.789845    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/449325e2-981e-4358-adfa-75608a671d84-tmp-dir\") pod \"ipp-route-018406-kserve-5cb6d69f4c-n2whm\" (UID: \"449325e2-981e-4358-adfa-75608a671d84\") " pod="ai-tenant-e2e-ipp-a-018406/ipp-route-018406-kserve-5cb6d69f4c-n2whm"
Jul 27 20:17:02.789965 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:02.789865    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/449325e2-981e-4358-adfa-75608a671d84-home\") pod \"ipp-route-018406-kserve-5cb6d69f4c-n2whm\" (UID: \"449325e2-981e-4358-adfa-75608a671d84\") " pod="ai-tenant-e2e-ipp-a-018406/ipp-route-018406-kserve-5cb6d69f4c-n2whm"
Jul 27 20:17:02.790042 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:02.789980    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/449325e2-981e-4358-adfa-75608a671d84-model-cache\") pod \"ipp-route-018406-kserve-5cb6d69f4c-n2whm\" (UID: \"449325e2-981e-4358-adfa-75608a671d84\") " pod="ai-tenant-e2e-ipp-a-018406/ipp-route-018406-kserve-5cb6d69f4c-n2whm"
Jul 27 20:17:02.790082 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:02.790052    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/449325e2-981e-4358-adfa-75608a671d84-dshm\") pod \"ipp-route-018406-kserve-5cb6d69f4c-n2whm\" (UID: \"449325e2-981e-4358-adfa-75608a671d84\") " pod="ai-tenant-e2e-ipp-a-018406/ipp-route-018406-kserve-5cb6d69f4c-n2whm"
Jul 27 20:17:02.790082 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:02.790077    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/449325e2-981e-4358-adfa-75608a671d84-tls-certs\") pod \"ipp-route-018406-kserve-5cb6d69f4c-n2whm\" (UID: \"449325e2-981e-4358-adfa-75608a671d84\") " pod="ai-tenant-e2e-ipp-a-018406/ipp-route-018406-kserve-5cb6d69f4c-n2whm"
Jul 27 20:17:02.891345 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:02.891314    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/449325e2-981e-4358-adfa-75608a671d84-model-cache\") pod \"ipp-route-018406-kserve-5cb6d69f4c-n2whm\" (UID: \"449325e2-981e-4358-adfa-75608a671d84\") " pod="ai-tenant-e2e-ipp-a-018406/ipp-route-018406-kserve-5cb6d69f4c-n2whm"
Jul 27 20:17:02.891579 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:02.891376    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/449325e2-981e-4358-adfa-75608a671d84-dshm\") pod \"ipp-route-018406-kserve-5cb6d69f4c-n2whm\" (UID: \"449325e2-981e-4358-adfa-75608a671d84\") " pod="ai-tenant-e2e-ipp-a-018406/ipp-route-018406-kserve-5cb6d69f4c-n2whm"
Jul 27 20:17:02.891579 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:02.891396    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/449325e2-981e-4358-adfa-75608a671d84-tls-certs\") pod \"ipp-route-018406-kserve-5cb6d69f4c-n2whm\" (UID: \"449325e2-981e-4358-adfa-75608a671d84\") " pod="ai-tenant-e2e-ipp-a-018406/ipp-route-018406-kserve-5cb6d69f4c-n2whm"
Jul 27 20:17:02.891579 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:02.891451    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qsqq\" (UniqueName: \"kubernetes.io/projected/449325e2-981e-4358-adfa-75608a671d84-kube-api-access-6qsqq\") pod \"ipp-route-018406-kserve-5cb6d69f4c-n2whm\" (UID: \"449325e2-981e-4358-adfa-75608a671d84\") " pod="ai-tenant-e2e-ipp-a-018406/ipp-route-018406-kserve-5cb6d69f4c-n2whm"
Jul 27 20:17:02.891579 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:02.891468    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/449325e2-981e-4358-adfa-75608a671d84-tmp-dir\") pod \"ipp-route-018406-kserve-5cb6d69f4c-n2whm\" (UID: \"449325e2-981e-4358-adfa-75608a671d84\") " pod="ai-tenant-e2e-ipp-a-018406/ipp-route-018406-kserve-5cb6d69f4c-n2whm"
Jul 27 20:17:02.891579 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:02.891483    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/449325e2-981e-4358-adfa-75608a671d84-home\") pod \"ipp-route-018406-kserve-5cb6d69f4c-n2whm\" (UID: \"449325e2-981e-4358-adfa-75608a671d84\") " pod="ai-tenant-e2e-ipp-a-018406/ipp-route-018406-kserve-5cb6d69f4c-n2whm"
Jul 27 20:17:02.891840 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:02.891819    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/449325e2-981e-4358-adfa-75608a671d84-home\") pod \"ipp-route-018406-kserve-5cb6d69f4c-n2whm\" (UID: \"449325e2-981e-4358-adfa-75608a671d84\") " pod="ai-tenant-e2e-ipp-a-018406/ipp-route-018406-kserve-5cb6d69f4c-n2whm"
Jul 27 20:17:02.891906 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:02.891884    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/449325e2-981e-4358-adfa-75608a671d84-model-cache\") pod \"ipp-route-018406-kserve-5cb6d69f4c-n2whm\" (UID: \"449325e2-981e-4358-adfa-75608a671d84\") " pod="ai-tenant-e2e-ipp-a-018406/ipp-route-018406-kserve-5cb6d69f4c-n2whm"
Jul 27 20:17:02.891976 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:02.891953    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/449325e2-981e-4358-adfa-75608a671d84-tmp-dir\") pod \"ipp-route-018406-kserve-5cb6d69f4c-n2whm\" (UID: \"449325e2-981e-4358-adfa-75608a671d84\") " pod="ai-tenant-e2e-ipp-a-018406/ipp-route-018406-kserve-5cb6d69f4c-n2whm"
Jul 27 20:17:02.893815 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:02.893797    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/449325e2-981e-4358-adfa-75608a671d84-dshm\") pod \"ipp-route-018406-kserve-5cb6d69f4c-n2whm\" (UID: \"449325e2-981e-4358-adfa-75608a671d84\") " pod="ai-tenant-e2e-ipp-a-018406/ipp-route-018406-kserve-5cb6d69f4c-n2whm"
Jul 27 20:17:02.894074 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:02.894058    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/449325e2-981e-4358-adfa-75608a671d84-tls-certs\") pod \"ipp-route-018406-kserve-5cb6d69f4c-n2whm\" (UID: \"449325e2-981e-4358-adfa-75608a671d84\") " pod="ai-tenant-e2e-ipp-a-018406/ipp-route-018406-kserve-5cb6d69f4c-n2whm"
Jul 27 20:17:02.898834 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:02.898812    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qsqq\" (UniqueName: \"kubernetes.io/projected/449325e2-981e-4358-adfa-75608a671d84-kube-api-access-6qsqq\") pod \"ipp-route-018406-kserve-5cb6d69f4c-n2whm\" (UID: \"449325e2-981e-4358-adfa-75608a671d84\") " pod="ai-tenant-e2e-ipp-a-018406/ipp-route-018406-kserve-5cb6d69f4c-n2whm"
Jul 27 20:17:03.040931 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:03.040848    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-018406/ipp-route-018406-kserve-5cb6d69f4c-n2whm"
Jul 27 20:17:03.177348 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:03.177322    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-018406/ipp-route-018406-kserve-5cb6d69f4c-n2whm"]
Jul 27 20:17:03.178791 ip-10-0-129-235 kubenswrapper[2558]: W0727 20:17:03.178764    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod449325e2_981e_4358_adfa_75608a671d84.slice/crio-d5b8d69c177ed54d6573f4c322aba8290d63e54002322220912388bde8b40281 WatchSource:0}: Error finding container d5b8d69c177ed54d6573f4c322aba8290d63e54002322220912388bde8b40281: Status 404 returned error can't find the container with id d5b8d69c177ed54d6573f4c322aba8290d63e54002322220912388bde8b40281
Jul 27 20:17:03.807206 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:03.807172    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-018406/ipp-route-018406-kserve-5cb6d69f4c-n2whm" event={"ID":"449325e2-981e-4358-adfa-75608a671d84","Type":"ContainerStarted","Data":"2b2bd4d03096ed8a89ff90f640335fcc4d50d09bf33c08a5bbf2d0ce173f5dc3"}
Jul 27 20:17:03.807660 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:03.807219    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-018406/ipp-route-018406-kserve-5cb6d69f4c-n2whm" event={"ID":"449325e2-981e-4358-adfa-75608a671d84","Type":"ContainerStarted","Data":"d5b8d69c177ed54d6573f4c322aba8290d63e54002322220912388bde8b40281"}
Jul 27 20:17:03.822978 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:03.822914    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-018406/ipp-route-018406-kserve-5cb6d69f4c-n2whm" podStartSLOduration=1.822894318 podStartE2EDuration="1.822894318s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:17:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:17:03.821901867 +0000 UTC m=+3036.694093841" watchObservedRunningTime="2026-07-27 20:17:03.822894318 +0000 UTC m=+3036.695086293"
Jul 27 20:17:13.041323 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:13.041284    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-018406/ipp-route-018406-kserve-5cb6d69f4c-n2whm"
Jul 27 20:17:13.041759 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:13.041357    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-018406/ipp-route-018406-kserve-5cb6d69f4c-n2whm"
Jul 27 20:17:13.053921 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:13.053898    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-018406/ipp-route-018406-kserve-5cb6d69f4c-n2whm"
Jul 27 20:17:13.875755 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:13.875716    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-018406/ipp-route-018406-kserve-5cb6d69f4c-n2whm"
Jul 27 20:17:20.563612 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:20.563579    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:17:20.665022 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:20.664990    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:17:49.933834 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:49.933801    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-018406-75754c7976-84hjb_faf8bee3-858c-45e6-bbc8-9ffcd5538b1b/payload-processing/0.log"
Jul 27 20:17:50.165707 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:50.165671    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-933254-c4b5858d4-8qtk2_448763ca-737d-4aab-8d3a-d26d0f309f52/payload-processing/0.log"
Jul 27 20:17:50.449029 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:50.448993    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:17:50.722896 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:50.722804    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:17:50.819351 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:50.819317    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:17:53.672687 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:53.672654    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:17:53.771622 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:53.771586    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:17:53.959353 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:53.959272    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:17:54.562800 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:54.562773    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:17:54.665610 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:54.665576    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:17:54.674659 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:54.674632    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-394c6a-fdb5656bd-pkhdj"]
Jul 27 20:17:54.678224 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:54.678206    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-394c6a-fdb5656bd-pkhdj"
Jul 27 20:17:54.680814 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:54.680792    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-394c6a\""
Jul 27 20:17:54.680924 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:54.680854    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-394c6a-dockercfg-kcbfw\""
Jul 27 20:17:54.687751 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:54.687727    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-394c6a-fdb5656bd-pkhdj"]
Jul 27 20:17:54.705155 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:54.705124    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-394c6a-548646bc76-vpwf6"]
Jul 27 20:17:54.708616 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:54.708594    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-394c6a-548646bc76-vpwf6"
Jul 27 20:17:54.720740 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:54.720712    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-394c6a-548646bc76-vpwf6"]
Jul 27 20:17:54.800264 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:54.800229    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68006c41-c6cd-40ef-9766-5f590683565b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-394c6a-fdb5656bd-pkhdj\" (UID: \"68006c41-c6cd-40ef-9766-5f590683565b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-394c6a-fdb5656bd-pkhdj"
Jul 27 20:17:54.800478 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:54.800281    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1421df22-eb16-4ba7-846e-ca6770f226ee-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-394c6a-548646bc76-vpwf6\" (UID: \"1421df22-eb16-4ba7-846e-ca6770f226ee\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-394c6a-548646bc76-vpwf6"
Jul 27 20:17:54.800478 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:54.800349    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hr4c5\" (UniqueName: \"kubernetes.io/projected/68006c41-c6cd-40ef-9766-5f590683565b-kube-api-access-hr4c5\") pod \"payload-pre-processing-e2e-ipp-cleanup-394c6a-fdb5656bd-pkhdj\" (UID: \"68006c41-c6cd-40ef-9766-5f590683565b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-394c6a-fdb5656bd-pkhdj"
Jul 27 20:17:54.800478 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:54.800397    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txn4l\" (UniqueName: \"kubernetes.io/projected/1421df22-eb16-4ba7-846e-ca6770f226ee-kube-api-access-txn4l\") pod \"payload-processing-e2e-ipp-cleanup-394c6a-548646bc76-vpwf6\" (UID: \"1421df22-eb16-4ba7-846e-ca6770f226ee\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-394c6a-548646bc76-vpwf6"
Jul 27 20:17:54.901303 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:54.901267    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txn4l\" (UniqueName: \"kubernetes.io/projected/1421df22-eb16-4ba7-846e-ca6770f226ee-kube-api-access-txn4l\") pod \"payload-processing-e2e-ipp-cleanup-394c6a-548646bc76-vpwf6\" (UID: \"1421df22-eb16-4ba7-846e-ca6770f226ee\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-394c6a-548646bc76-vpwf6"
Jul 27 20:17:54.901512 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:54.901329    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68006c41-c6cd-40ef-9766-5f590683565b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-394c6a-fdb5656bd-pkhdj\" (UID: \"68006c41-c6cd-40ef-9766-5f590683565b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-394c6a-fdb5656bd-pkhdj"
Jul 27 20:17:54.901512 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:54.901363    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1421df22-eb16-4ba7-846e-ca6770f226ee-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-394c6a-548646bc76-vpwf6\" (UID: \"1421df22-eb16-4ba7-846e-ca6770f226ee\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-394c6a-548646bc76-vpwf6"
Jul 27 20:17:54.901512 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:54.901437    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hr4c5\" (UniqueName: \"kubernetes.io/projected/68006c41-c6cd-40ef-9766-5f590683565b-kube-api-access-hr4c5\") pod \"payload-pre-processing-e2e-ipp-cleanup-394c6a-fdb5656bd-pkhdj\" (UID: \"68006c41-c6cd-40ef-9766-5f590683565b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-394c6a-fdb5656bd-pkhdj"
Jul 27 20:17:54.902060 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:54.902041    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1421df22-eb16-4ba7-846e-ca6770f226ee-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-394c6a-548646bc76-vpwf6\" (UID: \"1421df22-eb16-4ba7-846e-ca6770f226ee\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-394c6a-548646bc76-vpwf6"
Jul 27 20:17:54.902101 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:54.902083    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68006c41-c6cd-40ef-9766-5f590683565b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-394c6a-fdb5656bd-pkhdj\" (UID: \"68006c41-c6cd-40ef-9766-5f590683565b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-394c6a-fdb5656bd-pkhdj"
Jul 27 20:17:54.909051 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:54.909029    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hr4c5\" (UniqueName: \"kubernetes.io/projected/68006c41-c6cd-40ef-9766-5f590683565b-kube-api-access-hr4c5\") pod \"payload-pre-processing-e2e-ipp-cleanup-394c6a-fdb5656bd-pkhdj\" (UID: \"68006c41-c6cd-40ef-9766-5f590683565b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-394c6a-fdb5656bd-pkhdj"
Jul 27 20:17:54.909244 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:54.909222    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-txn4l\" (UniqueName: \"kubernetes.io/projected/1421df22-eb16-4ba7-846e-ca6770f226ee-kube-api-access-txn4l\") pod \"payload-processing-e2e-ipp-cleanup-394c6a-548646bc76-vpwf6\" (UID: \"1421df22-eb16-4ba7-846e-ca6770f226ee\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-394c6a-548646bc76-vpwf6"
Jul 27 20:17:54.990389 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:54.990353    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-394c6a-fdb5656bd-pkhdj"
Jul 27 20:17:55.020758 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:55.020728    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-394c6a-548646bc76-vpwf6"
Jul 27 20:17:55.150176 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:55.150138    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-394c6a-fdb5656bd-pkhdj"]
Jul 27 20:17:55.150739 ip-10-0-129-235 kubenswrapper[2558]: W0727 20:17:55.150710    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68006c41_c6cd_40ef_9766_5f590683565b.slice/crio-a72995c7feabc19a7b4a3222ab08cb2728ae6416a2342cbec7eb6441377fd2b4 WatchSource:0}: Error finding container a72995c7feabc19a7b4a3222ab08cb2728ae6416a2342cbec7eb6441377fd2b4: Status 404 returned error can't find the container with id a72995c7feabc19a7b4a3222ab08cb2728ae6416a2342cbec7eb6441377fd2b4
Jul 27 20:17:55.186400 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:55.186376    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-394c6a-548646bc76-vpwf6"]
Jul 27 20:17:55.192611 ip-10-0-129-235 kubenswrapper[2558]: W0727 20:17:55.192578    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1421df22_eb16_4ba7_846e_ca6770f226ee.slice/crio-986d7a7043a50389ef139c2cc9a1049ecf9df18211232c96af089469d0bb6e1c WatchSource:0}: Error finding container 986d7a7043a50389ef139c2cc9a1049ecf9df18211232c96af089469d0bb6e1c: Status 404 returned error can't find the container with id 986d7a7043a50389ef139c2cc9a1049ecf9df18211232c96af089469d0bb6e1c
Jul 27 20:17:56.119535 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:56.119493    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-394c6a-fdb5656bd-pkhdj" event={"ID":"68006c41-c6cd-40ef-9766-5f590683565b","Type":"ContainerStarted","Data":"28afdc4adb0827ad8a833dee0b0612849057c1e9ff42330544d819d10f6081f6"}
Jul 27 20:17:56.120021 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:56.119561    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-394c6a-fdb5656bd-pkhdj" event={"ID":"68006c41-c6cd-40ef-9766-5f590683565b","Type":"ContainerStarted","Data":"a72995c7feabc19a7b4a3222ab08cb2728ae6416a2342cbec7eb6441377fd2b4"}
Jul 27 20:17:56.120021 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:56.119629    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-394c6a-fdb5656bd-pkhdj"
Jul 27 20:17:56.121477 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:56.121446    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-394c6a-548646bc76-vpwf6" event={"ID":"1421df22-eb16-4ba7-846e-ca6770f226ee","Type":"ContainerStarted","Data":"27c140f6224851e33b3978ba3ac644c79fdf3530bccdbaf1c7f741ba8c363eff"}
Jul 27 20:17:56.121647 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:56.121500    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-394c6a-548646bc76-vpwf6" event={"ID":"1421df22-eb16-4ba7-846e-ca6770f226ee","Type":"ContainerStarted","Data":"986d7a7043a50389ef139c2cc9a1049ecf9df18211232c96af089469d0bb6e1c"}
Jul 27 20:17:56.121647 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:56.121532    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-394c6a-548646bc76-vpwf6"
Jul 27 20:17:56.138783 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:56.138730    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-394c6a-fdb5656bd-pkhdj" podStartSLOduration=2.138717321 podStartE2EDuration="2.138717321s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:17:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:17:56.134118738 +0000 UTC m=+3089.006310713" watchObservedRunningTime="2026-07-27 20:17:56.138717321 +0000 UTC m=+3089.010909295"
Jul 27 20:17:56.152916 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:56.152872    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-394c6a-548646bc76-vpwf6" podStartSLOduration=2.152858662 podStartE2EDuration="2.152858662s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:17:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:17:56.150477723 +0000 UTC m=+3089.022669699" watchObservedRunningTime="2026-07-27 20:17:56.152858662 +0000 UTC m=+3089.025050635"
Jul 27 20:17:58.172533 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:17:58.172496    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:18:07.130249 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:07.130215    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-394c6a-548646bc76-vpwf6"
Jul 27 20:18:07.130808 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:07.130299    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-394c6a-fdb5656bd-pkhdj"
Jul 27 20:18:19.465571 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:19.465537    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:18:19.496486 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:19.496456    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-394c6a-548646bc76-vpwf6"]
Jul 27 20:18:19.496731 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:19.496704    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-394c6a-548646bc76-vpwf6" podUID="1421df22-eb16-4ba7-846e-ca6770f226ee" containerName="payload-processing" containerID="cri-o://27c140f6224851e33b3978ba3ac644c79fdf3530bccdbaf1c7f741ba8c363eff" gracePeriod=30
Jul 27 20:18:19.506885 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:19.506853    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-394c6a-fdb5656bd-pkhdj"]
Jul 27 20:18:19.507293 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:19.507264    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-394c6a-fdb5656bd-pkhdj" podUID="68006c41-c6cd-40ef-9766-5f590683565b" containerName="payload-pre-processing" containerID="cri-o://28afdc4adb0827ad8a833dee0b0612849057c1e9ff42330544d819d10f6081f6" gracePeriod=30
Jul 27 20:18:19.536078 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:19.536044    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:18:19.596241 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:19.596211    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:18:19.649450 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:19.649364    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:18:19.791526 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:19.791500    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-394c6a-548646bc76-vpwf6"
Jul 27 20:18:19.794558 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:19.794540    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-394c6a-fdb5656bd-pkhdj"
Jul 27 20:18:19.832646 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:19.832620    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hr4c5\" (UniqueName: \"kubernetes.io/projected/68006c41-c6cd-40ef-9766-5f590683565b-kube-api-access-hr4c5\") pod \"68006c41-c6cd-40ef-9766-5f590683565b\" (UID: \"68006c41-c6cd-40ef-9766-5f590683565b\") "
Jul 27 20:18:19.832814 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:19.832658    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1421df22-eb16-4ba7-846e-ca6770f226ee-plugins-config-volume\") pod \"1421df22-eb16-4ba7-846e-ca6770f226ee\" (UID: \"1421df22-eb16-4ba7-846e-ca6770f226ee\") "
Jul 27 20:18:19.832814 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:19.832799    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68006c41-c6cd-40ef-9766-5f590683565b-plugins-config-volume\") pod \"68006c41-c6cd-40ef-9766-5f590683565b\" (UID: \"68006c41-c6cd-40ef-9766-5f590683565b\") "
Jul 27 20:18:19.832939 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:19.832859    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-txn4l\" (UniqueName: \"kubernetes.io/projected/1421df22-eb16-4ba7-846e-ca6770f226ee-kube-api-access-txn4l\") pod \"1421df22-eb16-4ba7-846e-ca6770f226ee\" (UID: \"1421df22-eb16-4ba7-846e-ca6770f226ee\") "
Jul 27 20:18:19.833076 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:19.833056    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1421df22-eb16-4ba7-846e-ca6770f226ee-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1421df22-eb16-4ba7-846e-ca6770f226ee" (UID: "1421df22-eb16-4ba7-846e-ca6770f226ee"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 20:18:19.833200 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:19.833167    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68006c41-c6cd-40ef-9766-5f590683565b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "68006c41-c6cd-40ef-9766-5f590683565b" (UID: "68006c41-c6cd-40ef-9766-5f590683565b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 20:18:19.833326 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:19.833238    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1421df22-eb16-4ba7-846e-ca6770f226ee-plugins-config-volume\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:18:19.835002 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:19.834982    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1421df22-eb16-4ba7-846e-ca6770f226ee-kube-api-access-txn4l" (OuterVolumeSpecName: "kube-api-access-txn4l") pod "1421df22-eb16-4ba7-846e-ca6770f226ee" (UID: "1421df22-eb16-4ba7-846e-ca6770f226ee"). InnerVolumeSpecName "kube-api-access-txn4l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:18:19.835246 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:19.835178    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68006c41-c6cd-40ef-9766-5f590683565b-kube-api-access-hr4c5" (OuterVolumeSpecName: "kube-api-access-hr4c5") pod "68006c41-c6cd-40ef-9766-5f590683565b" (UID: "68006c41-c6cd-40ef-9766-5f590683565b"). InnerVolumeSpecName "kube-api-access-hr4c5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:18:19.934510 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:19.934476    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hr4c5\" (UniqueName: \"kubernetes.io/projected/68006c41-c6cd-40ef-9766-5f590683565b-kube-api-access-hr4c5\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:18:19.934510 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:19.934504    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68006c41-c6cd-40ef-9766-5f590683565b-plugins-config-volume\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:18:19.934510 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:19.934516    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-txn4l\" (UniqueName: \"kubernetes.io/projected/1421df22-eb16-4ba7-846e-ca6770f226ee-kube-api-access-txn4l\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:18:19.991281 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:19.991248    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:18:20.270810 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:20.270775    2558 generic.go:356] "Generic (PLEG): container finished" podID="68006c41-c6cd-40ef-9766-5f590683565b" containerID="28afdc4adb0827ad8a833dee0b0612849057c1e9ff42330544d819d10f6081f6" exitCode=0
Jul 27 20:18:20.271020 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:20.270822    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-394c6a-fdb5656bd-pkhdj"
Jul 27 20:18:20.271020 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:20.270866    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-394c6a-fdb5656bd-pkhdj" event={"ID":"68006c41-c6cd-40ef-9766-5f590683565b","Type":"ContainerDied","Data":"28afdc4adb0827ad8a833dee0b0612849057c1e9ff42330544d819d10f6081f6"}
Jul 27 20:18:20.271020 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:20.270919    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-394c6a-fdb5656bd-pkhdj" event={"ID":"68006c41-c6cd-40ef-9766-5f590683565b","Type":"ContainerDied","Data":"a72995c7feabc19a7b4a3222ab08cb2728ae6416a2342cbec7eb6441377fd2b4"}
Jul 27 20:18:20.271020 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:20.270944    2558 scope.go:117] "RemoveContainer" containerID="28afdc4adb0827ad8a833dee0b0612849057c1e9ff42330544d819d10f6081f6"
Jul 27 20:18:20.272476 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:20.272435    2558 generic.go:356] "Generic (PLEG): container finished" podID="1421df22-eb16-4ba7-846e-ca6770f226ee" containerID="27c140f6224851e33b3978ba3ac644c79fdf3530bccdbaf1c7f741ba8c363eff" exitCode=0
Jul 27 20:18:20.272614 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:20.272482    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-394c6a-548646bc76-vpwf6"
Jul 27 20:18:20.272614 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:20.272508    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-394c6a-548646bc76-vpwf6" event={"ID":"1421df22-eb16-4ba7-846e-ca6770f226ee","Type":"ContainerDied","Data":"27c140f6224851e33b3978ba3ac644c79fdf3530bccdbaf1c7f741ba8c363eff"}
Jul 27 20:18:20.272614 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:20.272550    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-394c6a-548646bc76-vpwf6" event={"ID":"1421df22-eb16-4ba7-846e-ca6770f226ee","Type":"ContainerDied","Data":"986d7a7043a50389ef139c2cc9a1049ecf9df18211232c96af089469d0bb6e1c"}
Jul 27 20:18:20.281943 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:20.281929    2558 scope.go:117] "RemoveContainer" containerID="28afdc4adb0827ad8a833dee0b0612849057c1e9ff42330544d819d10f6081f6"
Jul 27 20:18:20.282224 ip-10-0-129-235 kubenswrapper[2558]: E0727 20:18:20.282202    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28afdc4adb0827ad8a833dee0b0612849057c1e9ff42330544d819d10f6081f6\": container with ID starting with 28afdc4adb0827ad8a833dee0b0612849057c1e9ff42330544d819d10f6081f6 not found: ID does not exist" containerID="28afdc4adb0827ad8a833dee0b0612849057c1e9ff42330544d819d10f6081f6"
Jul 27 20:18:20.282352 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:20.282234    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28afdc4adb0827ad8a833dee0b0612849057c1e9ff42330544d819d10f6081f6"} err="failed to get container status \"28afdc4adb0827ad8a833dee0b0612849057c1e9ff42330544d819d10f6081f6\": rpc error: code = NotFound desc = could not find container \"28afdc4adb0827ad8a833dee0b0612849057c1e9ff42330544d819d10f6081f6\": container with ID starting with 28afdc4adb0827ad8a833dee0b0612849057c1e9ff42330544d819d10f6081f6 not found: ID does not exist"
Jul 27 20:18:20.282352 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:20.282256    2558 scope.go:117] "RemoveContainer" containerID="27c140f6224851e33b3978ba3ac644c79fdf3530bccdbaf1c7f741ba8c363eff"
Jul 27 20:18:20.290855 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:20.290836    2558 scope.go:117] "RemoveContainer" containerID="27c140f6224851e33b3978ba3ac644c79fdf3530bccdbaf1c7f741ba8c363eff"
Jul 27 20:18:20.291112 ip-10-0-129-235 kubenswrapper[2558]: E0727 20:18:20.291096    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27c140f6224851e33b3978ba3ac644c79fdf3530bccdbaf1c7f741ba8c363eff\": container with ID starting with 27c140f6224851e33b3978ba3ac644c79fdf3530bccdbaf1c7f741ba8c363eff not found: ID does not exist" containerID="27c140f6224851e33b3978ba3ac644c79fdf3530bccdbaf1c7f741ba8c363eff"
Jul 27 20:18:20.291164 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:20.291118    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27c140f6224851e33b3978ba3ac644c79fdf3530bccdbaf1c7f741ba8c363eff"} err="failed to get container status \"27c140f6224851e33b3978ba3ac644c79fdf3530bccdbaf1c7f741ba8c363eff\": rpc error: code = NotFound desc = could not find container \"27c140f6224851e33b3978ba3ac644c79fdf3530bccdbaf1c7f741ba8c363eff\": container with ID starting with 27c140f6224851e33b3978ba3ac644c79fdf3530bccdbaf1c7f741ba8c363eff not found: ID does not exist"
Jul 27 20:18:20.297210 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:20.297190    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-394c6a-fdb5656bd-pkhdj"]
Jul 27 20:18:20.301194 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:20.301174    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-394c6a-fdb5656bd-pkhdj"]
Jul 27 20:18:20.310513 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:20.310494    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-394c6a-548646bc76-vpwf6"]
Jul 27 20:18:20.315895 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:20.315875    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-394c6a-548646bc76-vpwf6"]
Jul 27 20:18:21.681358 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:21.681321    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1421df22-eb16-4ba7-846e-ca6770f226ee" path="/var/lib/kubelet/pods/1421df22-eb16-4ba7-846e-ca6770f226ee/volumes"
Jul 27 20:18:21.681772 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:21.681718    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68006c41-c6cd-40ef-9766-5f590683565b" path="/var/lib/kubelet/pods/68006c41-c6cd-40ef-9766-5f590683565b/volumes"
Jul 27 20:18:25.998536 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:25.998486    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:18:26.060047 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:26.060006    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:18:26.252090 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:26.252014    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:18:26.457902 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:26.457867    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:18:45.161089 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:45.161055    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:18:45.181467 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:45.181432    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-018406-75754c7976-84hjb"]
Jul 27 20:18:45.181745 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:45.181714    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-018406-75754c7976-84hjb" podUID="faf8bee3-858c-45e6-bbc8-9ffcd5538b1b" containerName="payload-processing" containerID="cri-o://5b54b1a60848c8015b212df70bcadb824f68acb33251a080a1e23b40407d06d2" gracePeriod=30
Jul 27 20:18:45.191459 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:45.191181    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-018406-5fc4cbcbd8-5z95h"]
Jul 27 20:18:45.191459 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:45.191394    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-018406-5fc4cbcbd8-5z95h" podUID="f80492ac-8e54-4cd9-b0ab-8733548af2cc" containerName="payload-pre-processing" containerID="cri-o://c9f14a16d134e39055ec6aac4f4819da32a482240cbe29a222dd1e98c4d358ba" gracePeriod=30
Jul 27 20:18:45.223441 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:45.223383    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:18:45.294908 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:45.294882    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:18:45.395843 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:45.395798    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:18:45.422629 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:45.422543    2558 generic.go:356] "Generic (PLEG): container finished" podID="f80492ac-8e54-4cd9-b0ab-8733548af2cc" containerID="c9f14a16d134e39055ec6aac4f4819da32a482240cbe29a222dd1e98c4d358ba" exitCode=0
Jul 27 20:18:45.422629 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:45.422613    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-018406-5fc4cbcbd8-5z95h" event={"ID":"f80492ac-8e54-4cd9-b0ab-8733548af2cc","Type":"ContainerDied","Data":"c9f14a16d134e39055ec6aac4f4819da32a482240cbe29a222dd1e98c4d358ba"}
Jul 27 20:18:45.424470 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:45.424448    2558 generic.go:356] "Generic (PLEG): container finished" podID="faf8bee3-858c-45e6-bbc8-9ffcd5538b1b" containerID="5b54b1a60848c8015b212df70bcadb824f68acb33251a080a1e23b40407d06d2" exitCode=0
Jul 27 20:18:45.424614 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:45.424522    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-018406-75754c7976-84hjb" event={"ID":"faf8bee3-858c-45e6-bbc8-9ffcd5538b1b","Type":"ContainerDied","Data":"5b54b1a60848c8015b212df70bcadb824f68acb33251a080a1e23b40407d06d2"}
Jul 27 20:18:45.467473 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:45.467449    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-018406-75754c7976-84hjb"
Jul 27 20:18:45.470681 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:45.470662    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-018406-5fc4cbcbd8-5z95h"
Jul 27 20:18:45.575590 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:45.575554    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pz242\" (UniqueName: \"kubernetes.io/projected/f80492ac-8e54-4cd9-b0ab-8733548af2cc-kube-api-access-pz242\") pod \"f80492ac-8e54-4cd9-b0ab-8733548af2cc\" (UID: \"f80492ac-8e54-4cd9-b0ab-8733548af2cc\") "
Jul 27 20:18:45.575767 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:45.575674    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f80492ac-8e54-4cd9-b0ab-8733548af2cc-plugins-config-volume\") pod \"f80492ac-8e54-4cd9-b0ab-8733548af2cc\" (UID: \"f80492ac-8e54-4cd9-b0ab-8733548af2cc\") "
Jul 27 20:18:45.575767 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:45.575701    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wmqn2\" (UniqueName: \"kubernetes.io/projected/faf8bee3-858c-45e6-bbc8-9ffcd5538b1b-kube-api-access-wmqn2\") pod \"faf8bee3-858c-45e6-bbc8-9ffcd5538b1b\" (UID: \"faf8bee3-858c-45e6-bbc8-9ffcd5538b1b\") "
Jul 27 20:18:45.575767 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:45.575742    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/faf8bee3-858c-45e6-bbc8-9ffcd5538b1b-plugins-config-volume\") pod \"faf8bee3-858c-45e6-bbc8-9ffcd5538b1b\" (UID: \"faf8bee3-858c-45e6-bbc8-9ffcd5538b1b\") "
Jul 27 20:18:45.576093 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:45.576070    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f80492ac-8e54-4cd9-b0ab-8733548af2cc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f80492ac-8e54-4cd9-b0ab-8733548af2cc" (UID: "f80492ac-8e54-4cd9-b0ab-8733548af2cc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 20:18:45.576159 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:45.576098    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/faf8bee3-858c-45e6-bbc8-9ffcd5538b1b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "faf8bee3-858c-45e6-bbc8-9ffcd5538b1b" (UID: "faf8bee3-858c-45e6-bbc8-9ffcd5538b1b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 20:18:45.577892 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:45.577870    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f80492ac-8e54-4cd9-b0ab-8733548af2cc-kube-api-access-pz242" (OuterVolumeSpecName: "kube-api-access-pz242") pod "f80492ac-8e54-4cd9-b0ab-8733548af2cc" (UID: "f80492ac-8e54-4cd9-b0ab-8733548af2cc"). InnerVolumeSpecName "kube-api-access-pz242". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:18:45.577966 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:45.577954    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/faf8bee3-858c-45e6-bbc8-9ffcd5538b1b-kube-api-access-wmqn2" (OuterVolumeSpecName: "kube-api-access-wmqn2") pod "faf8bee3-858c-45e6-bbc8-9ffcd5538b1b" (UID: "faf8bee3-858c-45e6-bbc8-9ffcd5538b1b"). InnerVolumeSpecName "kube-api-access-wmqn2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:18:45.671260 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:45.671226    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:18:45.676748 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:45.676679    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f80492ac-8e54-4cd9-b0ab-8733548af2cc-plugins-config-volume\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:18:45.676748 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:45.676709    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wmqn2\" (UniqueName: \"kubernetes.io/projected/faf8bee3-858c-45e6-bbc8-9ffcd5538b1b-kube-api-access-wmqn2\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:18:45.676748 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:45.676725    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/faf8bee3-858c-45e6-bbc8-9ffcd5538b1b-plugins-config-volume\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:18:45.676748 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:45.676739    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pz242\" (UniqueName: \"kubernetes.io/projected/f80492ac-8e54-4cd9-b0ab-8733548af2cc-kube-api-access-pz242\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:18:46.432552 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:46.432517    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-018406-5fc4cbcbd8-5z95h" event={"ID":"f80492ac-8e54-4cd9-b0ab-8733548af2cc","Type":"ContainerDied","Data":"5c81a099653f27fe2afdca43e8f78ee418c6c645045bade0f91e6672cfeca446"}
Jul 27 20:18:46.432552 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:46.432542    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-018406-5fc4cbcbd8-5z95h"
Jul 27 20:18:46.433127 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:46.432604    2558 scope.go:117] "RemoveContainer" containerID="c9f14a16d134e39055ec6aac4f4819da32a482240cbe29a222dd1e98c4d358ba"
Jul 27 20:18:46.434316 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:46.434133    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-018406-75754c7976-84hjb"
Jul 27 20:18:46.434316 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:46.434167    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-018406-75754c7976-84hjb" event={"ID":"faf8bee3-858c-45e6-bbc8-9ffcd5538b1b","Type":"ContainerDied","Data":"6051f94a80ef1240fbf86f4a8fa716991a4e2ae1a7529b3078a8208b9a40e867"}
Jul 27 20:18:46.442459 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:46.442440    2558 scope.go:117] "RemoveContainer" containerID="5b54b1a60848c8015b212df70bcadb824f68acb33251a080a1e23b40407d06d2"
Jul 27 20:18:46.453026 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:46.452996    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-018406-75754c7976-84hjb"]
Jul 27 20:18:46.455595 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:46.455575    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-018406-75754c7976-84hjb"]
Jul 27 20:18:46.464348 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:46.464327    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-018406-5fc4cbcbd8-5z95h"]
Jul 27 20:18:46.466573 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:46.466554    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-018406-5fc4cbcbd8-5z95h"]
Jul 27 20:18:47.674451 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:47.674367    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f80492ac-8e54-4cd9-b0ab-8733548af2cc" path="/var/lib/kubelet/pods/f80492ac-8e54-4cd9-b0ab-8733548af2cc/volumes"
Jul 27 20:18:47.674816 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:47.674744    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="faf8bee3-858c-45e6-bbc8-9ffcd5538b1b" path="/var/lib/kubelet/pods/faf8bee3-858c-45e6-bbc8-9ffcd5538b1b/volumes"
Jul 27 20:18:50.671054 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:50.671019    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-018406/ipp-route-018406-kserve-5cb6d69f4c-n2whm"]
Jul 27 20:18:50.770668 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:18:50.770634    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:19:05.310071 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:05.310031    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:19:05.376865 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:05.376815    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:19:05.691014 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:05.690976    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-018406/ipp-route-018406-kserve-5cb6d69f4c-n2whm" podUID="449325e2-981e-4358-adfa-75608a671d84" containerName="main" containerID="cri-o://2b2bd4d03096ed8a89ff90f640335fcc4d50d09bf33c08a5bbf2d0ce173f5dc3" gracePeriod=45
Jul 27 20:19:05.946028 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:05.945969    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-018406/ipp-route-018406-kserve-5cb6d69f4c-n2whm"
Jul 27 20:19:05.960727 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:05.960703    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/449325e2-981e-4358-adfa-75608a671d84-tmp-dir\") pod \"449325e2-981e-4358-adfa-75608a671d84\" (UID: \"449325e2-981e-4358-adfa-75608a671d84\") "
Jul 27 20:19:05.960852 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:05.960786    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/449325e2-981e-4358-adfa-75608a671d84-dshm\") pod \"449325e2-981e-4358-adfa-75608a671d84\" (UID: \"449325e2-981e-4358-adfa-75608a671d84\") "
Jul 27 20:19:05.960852 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:05.960814    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6qsqq\" (UniqueName: \"kubernetes.io/projected/449325e2-981e-4358-adfa-75608a671d84-kube-api-access-6qsqq\") pod \"449325e2-981e-4358-adfa-75608a671d84\" (UID: \"449325e2-981e-4358-adfa-75608a671d84\") "
Jul 27 20:19:05.960852 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:05.960845    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/449325e2-981e-4358-adfa-75608a671d84-tls-certs\") pod \"449325e2-981e-4358-adfa-75608a671d84\" (UID: \"449325e2-981e-4358-adfa-75608a671d84\") "
Jul 27 20:19:05.961014 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:05.960868    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/449325e2-981e-4358-adfa-75608a671d84-home\") pod \"449325e2-981e-4358-adfa-75608a671d84\" (UID: \"449325e2-981e-4358-adfa-75608a671d84\") "
Jul 27 20:19:05.961014 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:05.960893    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/449325e2-981e-4358-adfa-75608a671d84-model-cache\") pod \"449325e2-981e-4358-adfa-75608a671d84\" (UID: \"449325e2-981e-4358-adfa-75608a671d84\") "
Jul 27 20:19:05.961014 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:05.960940    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/449325e2-981e-4358-adfa-75608a671d84-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "449325e2-981e-4358-adfa-75608a671d84" (UID: "449325e2-981e-4358-adfa-75608a671d84"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:19:05.961380 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:05.961156    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/449325e2-981e-4358-adfa-75608a671d84-model-cache" (OuterVolumeSpecName: "model-cache") pod "449325e2-981e-4358-adfa-75608a671d84" (UID: "449325e2-981e-4358-adfa-75608a671d84"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:19:05.961380 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:05.961167    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/449325e2-981e-4358-adfa-75608a671d84-tmp-dir\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:19:05.961380 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:05.961168    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/449325e2-981e-4358-adfa-75608a671d84-home" (OuterVolumeSpecName: "home") pod "449325e2-981e-4358-adfa-75608a671d84" (UID: "449325e2-981e-4358-adfa-75608a671d84"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:19:05.963757 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:05.963720    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/449325e2-981e-4358-adfa-75608a671d84-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "449325e2-981e-4358-adfa-75608a671d84" (UID: "449325e2-981e-4358-adfa-75608a671d84"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 20:19:05.965639 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:05.963996    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/449325e2-981e-4358-adfa-75608a671d84-dshm" (OuterVolumeSpecName: "dshm") pod "449325e2-981e-4358-adfa-75608a671d84" (UID: "449325e2-981e-4358-adfa-75608a671d84"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:19:05.965639 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:05.965250    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/449325e2-981e-4358-adfa-75608a671d84-kube-api-access-6qsqq" (OuterVolumeSpecName: "kube-api-access-6qsqq") pod "449325e2-981e-4358-adfa-75608a671d84" (UID: "449325e2-981e-4358-adfa-75608a671d84"). InnerVolumeSpecName "kube-api-access-6qsqq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:19:06.062098 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:06.062063    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/449325e2-981e-4358-adfa-75608a671d84-dshm\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:19:06.062098 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:06.062094    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6qsqq\" (UniqueName: \"kubernetes.io/projected/449325e2-981e-4358-adfa-75608a671d84-kube-api-access-6qsqq\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:19:06.062098 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:06.062105    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/449325e2-981e-4358-adfa-75608a671d84-tls-certs\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:19:06.062316 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:06.062113    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/449325e2-981e-4358-adfa-75608a671d84-home\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:19:06.062316 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:06.062121    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/449325e2-981e-4358-adfa-75608a671d84-model-cache\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:19:06.553014 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:06.552982    2558 generic.go:356] "Generic (PLEG): container finished" podID="449325e2-981e-4358-adfa-75608a671d84" containerID="2b2bd4d03096ed8a89ff90f640335fcc4d50d09bf33c08a5bbf2d0ce173f5dc3" exitCode=0
Jul 27 20:19:06.553479 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:06.553026    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-018406/ipp-route-018406-kserve-5cb6d69f4c-n2whm"
Jul 27 20:19:06.553479 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:06.553070    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-018406/ipp-route-018406-kserve-5cb6d69f4c-n2whm" event={"ID":"449325e2-981e-4358-adfa-75608a671d84","Type":"ContainerDied","Data":"2b2bd4d03096ed8a89ff90f640335fcc4d50d09bf33c08a5bbf2d0ce173f5dc3"}
Jul 27 20:19:06.553479 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:06.553113    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-018406/ipp-route-018406-kserve-5cb6d69f4c-n2whm" event={"ID":"449325e2-981e-4358-adfa-75608a671d84","Type":"ContainerDied","Data":"d5b8d69c177ed54d6573f4c322aba8290d63e54002322220912388bde8b40281"}
Jul 27 20:19:06.553479 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:06.553130    2558 scope.go:117] "RemoveContainer" containerID="2b2bd4d03096ed8a89ff90f640335fcc4d50d09bf33c08a5bbf2d0ce173f5dc3"
Jul 27 20:19:06.565397 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:06.565380    2558 scope.go:117] "RemoveContainer" containerID="2b2bd4d03096ed8a89ff90f640335fcc4d50d09bf33c08a5bbf2d0ce173f5dc3"
Jul 27 20:19:06.565693 ip-10-0-129-235 kubenswrapper[2558]: E0727 20:19:06.565674    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b2bd4d03096ed8a89ff90f640335fcc4d50d09bf33c08a5bbf2d0ce173f5dc3\": container with ID starting with 2b2bd4d03096ed8a89ff90f640335fcc4d50d09bf33c08a5bbf2d0ce173f5dc3 not found: ID does not exist" containerID="2b2bd4d03096ed8a89ff90f640335fcc4d50d09bf33c08a5bbf2d0ce173f5dc3"
Jul 27 20:19:06.565741 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:06.565701    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b2bd4d03096ed8a89ff90f640335fcc4d50d09bf33c08a5bbf2d0ce173f5dc3"} err="failed to get container status \"2b2bd4d03096ed8a89ff90f640335fcc4d50d09bf33c08a5bbf2d0ce173f5dc3\": rpc error: code = NotFound desc = could not find container \"2b2bd4d03096ed8a89ff90f640335fcc4d50d09bf33c08a5bbf2d0ce173f5dc3\": container with ID starting with 2b2bd4d03096ed8a89ff90f640335fcc4d50d09bf33c08a5bbf2d0ce173f5dc3 not found: ID does not exist"
Jul 27 20:19:06.577311 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:06.577289    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-018406/ipp-route-018406-kserve-5cb6d69f4c-n2whm"]
Jul 27 20:19:06.580930 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:06.580907    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-018406/ipp-route-018406-kserve-5cb6d69f4c-n2whm"]
Jul 27 20:19:07.677636 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:07.675236    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="449325e2-981e-4358-adfa-75608a671d84" path="/var/lib/kubelet/pods/449325e2-981e-4358-adfa-75608a671d84/volumes"
Jul 27 20:19:27.986972 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:27.986938    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:19:28.177085 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:28.177054    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:19:33.995607 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:33.995580    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:19:39.908985 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:39.908947    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:19:39.974588 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:39.974537    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:19:40.193449 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:40.193342    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:19:40.456134 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:40.456017    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:19:50.975074 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:50.975039    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:19:51.002069 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:51.002034    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-933254-c4b5858d4-8qtk2"]
Jul 27 20:19:51.002274 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:51.002245    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-933254-c4b5858d4-8qtk2" podUID="448763ca-737d-4aab-8d3a-d26d0f309f52" containerName="payload-processing" containerID="cri-o://d46a44c6fba3411ed0c2dc911dcabbac570a27cdec5e8dfcfb35d628c968de9f" gracePeriod=30
Jul 27 20:19:51.011885 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:51.011854    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-933254-78765ccbd4-m6259"]
Jul 27 20:19:51.012134 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:51.012087    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-933254-78765ccbd4-m6259" podUID="dd278b5b-82dd-4048-88f2-1487bb9759e1" containerName="payload-pre-processing" containerID="cri-o://d36d389f9edabfc27b052e344decb46594aa08f55dad87f41713ac00b4c13004" gracePeriod=30
Jul 27 20:19:51.057434 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:51.056194    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:19:51.131116 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:51.131096    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:19:51.229373 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:51.229291    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:19:51.284766 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:51.284721    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-933254-c4b5858d4-8qtk2"
Jul 27 20:19:51.306007 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:51.305983    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-933254-78765ccbd4-m6259"
Jul 27 20:19:51.377920 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:51.377888    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/448763ca-737d-4aab-8d3a-d26d0f309f52-plugins-config-volume\") pod \"448763ca-737d-4aab-8d3a-d26d0f309f52\" (UID: \"448763ca-737d-4aab-8d3a-d26d0f309f52\") "
Jul 27 20:19:51.377920 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:51.377924    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd278b5b-82dd-4048-88f2-1487bb9759e1-plugins-config-volume\") pod \"dd278b5b-82dd-4048-88f2-1487bb9759e1\" (UID: \"dd278b5b-82dd-4048-88f2-1487bb9759e1\") "
Jul 27 20:19:51.378141 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:51.377948    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7vpg8\" (UniqueName: \"kubernetes.io/projected/dd278b5b-82dd-4048-88f2-1487bb9759e1-kube-api-access-7vpg8\") pod \"dd278b5b-82dd-4048-88f2-1487bb9759e1\" (UID: \"dd278b5b-82dd-4048-88f2-1487bb9759e1\") "
Jul 27 20:19:51.378141 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:51.378025    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kvrjf\" (UniqueName: \"kubernetes.io/projected/448763ca-737d-4aab-8d3a-d26d0f309f52-kube-api-access-kvrjf\") pod \"448763ca-737d-4aab-8d3a-d26d0f309f52\" (UID: \"448763ca-737d-4aab-8d3a-d26d0f309f52\") "
Jul 27 20:19:51.378341 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:51.378319    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd278b5b-82dd-4048-88f2-1487bb9759e1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "dd278b5b-82dd-4048-88f2-1487bb9759e1" (UID: "dd278b5b-82dd-4048-88f2-1487bb9759e1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 20:19:51.378377 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:51.378328    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/448763ca-737d-4aab-8d3a-d26d0f309f52-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "448763ca-737d-4aab-8d3a-d26d0f309f52" (UID: "448763ca-737d-4aab-8d3a-d26d0f309f52"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 20:19:51.378440 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:51.378398    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/448763ca-737d-4aab-8d3a-d26d0f309f52-plugins-config-volume\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:19:51.378483 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:51.378444    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd278b5b-82dd-4048-88f2-1487bb9759e1-plugins-config-volume\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:19:51.380328 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:51.380302    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd278b5b-82dd-4048-88f2-1487bb9759e1-kube-api-access-7vpg8" (OuterVolumeSpecName: "kube-api-access-7vpg8") pod "dd278b5b-82dd-4048-88f2-1487bb9759e1" (UID: "dd278b5b-82dd-4048-88f2-1487bb9759e1"). InnerVolumeSpecName "kube-api-access-7vpg8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:19:51.380469 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:51.380348    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/448763ca-737d-4aab-8d3a-d26d0f309f52-kube-api-access-kvrjf" (OuterVolumeSpecName: "kube-api-access-kvrjf") pod "448763ca-737d-4aab-8d3a-d26d0f309f52" (UID: "448763ca-737d-4aab-8d3a-d26d0f309f52"). InnerVolumeSpecName "kube-api-access-kvrjf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:19:51.479366 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:51.479335    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7vpg8\" (UniqueName: \"kubernetes.io/projected/dd278b5b-82dd-4048-88f2-1487bb9759e1-kube-api-access-7vpg8\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:19:51.479366 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:51.479363    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kvrjf\" (UniqueName: \"kubernetes.io/projected/448763ca-737d-4aab-8d3a-d26d0f309f52-kube-api-access-kvrjf\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:19:51.527042 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:51.526956    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:19:51.810859 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:51.810761    2558 generic.go:356] "Generic (PLEG): container finished" podID="448763ca-737d-4aab-8d3a-d26d0f309f52" containerID="d46a44c6fba3411ed0c2dc911dcabbac570a27cdec5e8dfcfb35d628c968de9f" exitCode=0
Jul 27 20:19:51.810859 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:51.810799    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-933254-c4b5858d4-8qtk2"
Jul 27 20:19:51.810859 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:51.810844    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-933254-c4b5858d4-8qtk2" event={"ID":"448763ca-737d-4aab-8d3a-d26d0f309f52","Type":"ContainerDied","Data":"d46a44c6fba3411ed0c2dc911dcabbac570a27cdec5e8dfcfb35d628c968de9f"}
Jul 27 20:19:51.811150 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:51.810894    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-933254-c4b5858d4-8qtk2" event={"ID":"448763ca-737d-4aab-8d3a-d26d0f309f52","Type":"ContainerDied","Data":"a4c98566ce4ec03197c064e5765c88f246026a9deca4f14ed2a6690438bc3ccb"}
Jul 27 20:19:51.811150 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:51.810925    2558 scope.go:117] "RemoveContainer" containerID="d46a44c6fba3411ed0c2dc911dcabbac570a27cdec5e8dfcfb35d628c968de9f"
Jul 27 20:19:51.812230 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:51.812208    2558 generic.go:356] "Generic (PLEG): container finished" podID="dd278b5b-82dd-4048-88f2-1487bb9759e1" containerID="d36d389f9edabfc27b052e344decb46594aa08f55dad87f41713ac00b4c13004" exitCode=0
Jul 27 20:19:51.812359 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:51.812249    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-933254-78765ccbd4-m6259"
Jul 27 20:19:51.812359 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:51.812295    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-933254-78765ccbd4-m6259" event={"ID":"dd278b5b-82dd-4048-88f2-1487bb9759e1","Type":"ContainerDied","Data":"d36d389f9edabfc27b052e344decb46594aa08f55dad87f41713ac00b4c13004"}
Jul 27 20:19:51.812359 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:51.812351    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-933254-78765ccbd4-m6259" event={"ID":"dd278b5b-82dd-4048-88f2-1487bb9759e1","Type":"ContainerDied","Data":"e572d98b185cb650342eb82c6e87f03164edfc2894456d28efaea1931bdfc326"}
Jul 27 20:19:51.821132 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:51.821116    2558 scope.go:117] "RemoveContainer" containerID="d46a44c6fba3411ed0c2dc911dcabbac570a27cdec5e8dfcfb35d628c968de9f"
Jul 27 20:19:51.821373 ip-10-0-129-235 kubenswrapper[2558]: E0727 20:19:51.821352    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d46a44c6fba3411ed0c2dc911dcabbac570a27cdec5e8dfcfb35d628c968de9f\": container with ID starting with d46a44c6fba3411ed0c2dc911dcabbac570a27cdec5e8dfcfb35d628c968de9f not found: ID does not exist" containerID="d46a44c6fba3411ed0c2dc911dcabbac570a27cdec5e8dfcfb35d628c968de9f"
Jul 27 20:19:51.821479 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:51.821387    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d46a44c6fba3411ed0c2dc911dcabbac570a27cdec5e8dfcfb35d628c968de9f"} err="failed to get container status \"d46a44c6fba3411ed0c2dc911dcabbac570a27cdec5e8dfcfb35d628c968de9f\": rpc error: code = NotFound desc = could not find container \"d46a44c6fba3411ed0c2dc911dcabbac570a27cdec5e8dfcfb35d628c968de9f\": container with ID starting with d46a44c6fba3411ed0c2dc911dcabbac570a27cdec5e8dfcfb35d628c968de9f not found: ID does not exist"
Jul 27 20:19:51.821479 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:51.821426    2558 scope.go:117] "RemoveContainer" containerID="d36d389f9edabfc27b052e344decb46594aa08f55dad87f41713ac00b4c13004"
Jul 27 20:19:51.830137 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:51.830116    2558 scope.go:117] "RemoveContainer" containerID="d36d389f9edabfc27b052e344decb46594aa08f55dad87f41713ac00b4c13004"
Jul 27 20:19:51.830543 ip-10-0-129-235 kubenswrapper[2558]: E0727 20:19:51.830523    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d36d389f9edabfc27b052e344decb46594aa08f55dad87f41713ac00b4c13004\": container with ID starting with d36d389f9edabfc27b052e344decb46594aa08f55dad87f41713ac00b4c13004 not found: ID does not exist" containerID="d36d389f9edabfc27b052e344decb46594aa08f55dad87f41713ac00b4c13004"
Jul 27 20:19:51.830622 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:51.830551    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d36d389f9edabfc27b052e344decb46594aa08f55dad87f41713ac00b4c13004"} err="failed to get container status \"d36d389f9edabfc27b052e344decb46594aa08f55dad87f41713ac00b4c13004\": rpc error: code = NotFound desc = could not find container \"d36d389f9edabfc27b052e344decb46594aa08f55dad87f41713ac00b4c13004\": container with ID starting with d36d389f9edabfc27b052e344decb46594aa08f55dad87f41713ac00b4c13004 not found: ID does not exist"
Jul 27 20:19:51.830685 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:51.830673    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-933254-c4b5858d4-8qtk2"]
Jul 27 20:19:51.834112 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:51.834088    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-933254-c4b5858d4-8qtk2"]
Jul 27 20:19:51.842993 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:51.842959    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-933254-78765ccbd4-m6259"]
Jul 27 20:19:51.846563 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:51.846544    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-933254-78765ccbd4-m6259"]
Jul 27 20:19:53.675286 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:53.675250    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="448763ca-737d-4aab-8d3a-d26d0f309f52" path="/var/lib/kubelet/pods/448763ca-737d-4aab-8d3a-d26d0f309f52/volumes"
Jul 27 20:19:53.675687 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:53.675648    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd278b5b-82dd-4048-88f2-1487bb9759e1" path="/var/lib/kubelet/pods/dd278b5b-82dd-4048-88f2-1487bb9759e1/volumes"
Jul 27 20:19:57.749493 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:57.749458    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:19:57.807928 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:57.807889    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:19:58.012046 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:58.011966    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:19:58.222957 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:19:58.222917    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:20:08.795774 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:08.795742    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:20:08.822509 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:08.822468    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-d3c546-6b6f876dfd-tg88d"]
Jul 27 20:20:08.822735 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:08.822708    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-d3c546-6b6f876dfd-tg88d" podUID="548a6292-1d21-4a62-abb1-67bbc6b93ccf" containerName="payload-processing" containerID="cri-o://2fce1cbf562354d9dfce70dd1db4d43dcc8926b1239000978f72d08ce000049d" gracePeriod=30
Jul 27 20:20:08.825857 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:08.825819    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-d3c546-68dbffb748-g5tfw"]
Jul 27 20:20:08.826078 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:08.826052    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d3c546-68dbffb748-g5tfw" podUID="cb09fff4-29c5-4ee5-909d-376451de8715" containerName="payload-pre-processing" containerID="cri-o://3a4c0c433767548e29f9a1fec0546f8cd87a6113c34a9343e15262b3445d9009" gracePeriod=30
Jul 27 20:20:08.862980 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:08.862931    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:20:08.914746 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:08.914691    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:20:08.978671 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:08.978587    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:20:09.104822 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:09.104789    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-d3c546-6b6f876dfd-tg88d"
Jul 27 20:20:09.107710 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:09.107692    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d3c546-68dbffb748-g5tfw"
Jul 27 20:20:09.253782 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:09.253736    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-242vk\" (UniqueName: \"kubernetes.io/projected/548a6292-1d21-4a62-abb1-67bbc6b93ccf-kube-api-access-242vk\") pod \"548a6292-1d21-4a62-abb1-67bbc6b93ccf\" (UID: \"548a6292-1d21-4a62-abb1-67bbc6b93ccf\") "
Jul 27 20:20:09.253976 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:09.253805    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rxbct\" (UniqueName: \"kubernetes.io/projected/cb09fff4-29c5-4ee5-909d-376451de8715-kube-api-access-rxbct\") pod \"cb09fff4-29c5-4ee5-909d-376451de8715\" (UID: \"cb09fff4-29c5-4ee5-909d-376451de8715\") "
Jul 27 20:20:09.253976 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:09.253855    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/548a6292-1d21-4a62-abb1-67bbc6b93ccf-plugins-config-volume\") pod \"548a6292-1d21-4a62-abb1-67bbc6b93ccf\" (UID: \"548a6292-1d21-4a62-abb1-67bbc6b93ccf\") "
Jul 27 20:20:09.253976 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:09.253904    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb09fff4-29c5-4ee5-909d-376451de8715-plugins-config-volume\") pod \"cb09fff4-29c5-4ee5-909d-376451de8715\" (UID: \"cb09fff4-29c5-4ee5-909d-376451de8715\") "
Jul 27 20:20:09.254258 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:09.254231    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/548a6292-1d21-4a62-abb1-67bbc6b93ccf-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "548a6292-1d21-4a62-abb1-67bbc6b93ccf" (UID: "548a6292-1d21-4a62-abb1-67bbc6b93ccf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 20:20:09.254333 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:09.254274    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb09fff4-29c5-4ee5-909d-376451de8715-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "cb09fff4-29c5-4ee5-909d-376451de8715" (UID: "cb09fff4-29c5-4ee5-909d-376451de8715"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 20:20:09.256005 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:09.255979    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/548a6292-1d21-4a62-abb1-67bbc6b93ccf-kube-api-access-242vk" (OuterVolumeSpecName: "kube-api-access-242vk") pod "548a6292-1d21-4a62-abb1-67bbc6b93ccf" (UID: "548a6292-1d21-4a62-abb1-67bbc6b93ccf"). InnerVolumeSpecName "kube-api-access-242vk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:20:09.256105 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:09.256024    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb09fff4-29c5-4ee5-909d-376451de8715-kube-api-access-rxbct" (OuterVolumeSpecName: "kube-api-access-rxbct") pod "cb09fff4-29c5-4ee5-909d-376451de8715" (UID: "cb09fff4-29c5-4ee5-909d-376451de8715"). InnerVolumeSpecName "kube-api-access-rxbct". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:20:09.320761 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:09.320723    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:20:09.354883 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:09.354815    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-242vk\" (UniqueName: \"kubernetes.io/projected/548a6292-1d21-4a62-abb1-67bbc6b93ccf-kube-api-access-242vk\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:20:09.354883 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:09.354837    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rxbct\" (UniqueName: \"kubernetes.io/projected/cb09fff4-29c5-4ee5-909d-376451de8715-kube-api-access-rxbct\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:20:09.354883 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:09.354847    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/548a6292-1d21-4a62-abb1-67bbc6b93ccf-plugins-config-volume\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:20:09.354883 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:09.354857    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb09fff4-29c5-4ee5-909d-376451de8715-plugins-config-volume\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:20:09.925592 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:09.925560    2558 generic.go:356] "Generic (PLEG): container finished" podID="548a6292-1d21-4a62-abb1-67bbc6b93ccf" containerID="2fce1cbf562354d9dfce70dd1db4d43dcc8926b1239000978f72d08ce000049d" exitCode=0
Jul 27 20:20:09.926019 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:09.925605    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-d3c546-6b6f876dfd-tg88d"
Jul 27 20:20:09.926019 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:09.925638    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-d3c546-6b6f876dfd-tg88d" event={"ID":"548a6292-1d21-4a62-abb1-67bbc6b93ccf","Type":"ContainerDied","Data":"2fce1cbf562354d9dfce70dd1db4d43dcc8926b1239000978f72d08ce000049d"}
Jul 27 20:20:09.926019 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:09.925693    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-d3c546-6b6f876dfd-tg88d" event={"ID":"548a6292-1d21-4a62-abb1-67bbc6b93ccf","Type":"ContainerDied","Data":"fe026e5f6d6c15180eb30c098602708fa8cda1fd1d06437716bc31908db39608"}
Jul 27 20:20:09.926019 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:09.925720    2558 scope.go:117] "RemoveContainer" containerID="2fce1cbf562354d9dfce70dd1db4d43dcc8926b1239000978f72d08ce000049d"
Jul 27 20:20:09.927118 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:09.927069    2558 generic.go:356] "Generic (PLEG): container finished" podID="cb09fff4-29c5-4ee5-909d-376451de8715" containerID="3a4c0c433767548e29f9a1fec0546f8cd87a6113c34a9343e15262b3445d9009" exitCode=0
Jul 27 20:20:09.927118 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:09.927100    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d3c546-68dbffb748-g5tfw" event={"ID":"cb09fff4-29c5-4ee5-909d-376451de8715","Type":"ContainerDied","Data":"3a4c0c433767548e29f9a1fec0546f8cd87a6113c34a9343e15262b3445d9009"}
Jul 27 20:20:09.927235 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:09.927115    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d3c546-68dbffb748-g5tfw"
Jul 27 20:20:09.927235 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:09.927151    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d3c546-68dbffb748-g5tfw" event={"ID":"cb09fff4-29c5-4ee5-909d-376451de8715","Type":"ContainerDied","Data":"ef41533d741a596b1549d0a05b37e481f91cabe07d4578ffa0a7568a3054d512"}
Jul 27 20:20:09.935645 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:09.935558    2558 scope.go:117] "RemoveContainer" containerID="2fce1cbf562354d9dfce70dd1db4d43dcc8926b1239000978f72d08ce000049d"
Jul 27 20:20:09.935828 ip-10-0-129-235 kubenswrapper[2558]: E0727 20:20:09.935811    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2fce1cbf562354d9dfce70dd1db4d43dcc8926b1239000978f72d08ce000049d\": container with ID starting with 2fce1cbf562354d9dfce70dd1db4d43dcc8926b1239000978f72d08ce000049d not found: ID does not exist" containerID="2fce1cbf562354d9dfce70dd1db4d43dcc8926b1239000978f72d08ce000049d"
Jul 27 20:20:09.935878 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:09.935836    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2fce1cbf562354d9dfce70dd1db4d43dcc8926b1239000978f72d08ce000049d"} err="failed to get container status \"2fce1cbf562354d9dfce70dd1db4d43dcc8926b1239000978f72d08ce000049d\": rpc error: code = NotFound desc = could not find container \"2fce1cbf562354d9dfce70dd1db4d43dcc8926b1239000978f72d08ce000049d\": container with ID starting with 2fce1cbf562354d9dfce70dd1db4d43dcc8926b1239000978f72d08ce000049d not found: ID does not exist"
Jul 27 20:20:09.935878 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:09.935853    2558 scope.go:117] "RemoveContainer" containerID="3a4c0c433767548e29f9a1fec0546f8cd87a6113c34a9343e15262b3445d9009"
Jul 27 20:20:09.944194 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:09.944176    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-d3c546-6b6f876dfd-tg88d"]
Jul 27 20:20:09.944494 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:09.944477    2558 scope.go:117] "RemoveContainer" containerID="3a4c0c433767548e29f9a1fec0546f8cd87a6113c34a9343e15262b3445d9009"
Jul 27 20:20:09.944754 ip-10-0-129-235 kubenswrapper[2558]: E0727 20:20:09.944738    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a4c0c433767548e29f9a1fec0546f8cd87a6113c34a9343e15262b3445d9009\": container with ID starting with 3a4c0c433767548e29f9a1fec0546f8cd87a6113c34a9343e15262b3445d9009 not found: ID does not exist" containerID="3a4c0c433767548e29f9a1fec0546f8cd87a6113c34a9343e15262b3445d9009"
Jul 27 20:20:09.944810 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:09.944761    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a4c0c433767548e29f9a1fec0546f8cd87a6113c34a9343e15262b3445d9009"} err="failed to get container status \"3a4c0c433767548e29f9a1fec0546f8cd87a6113c34a9343e15262b3445d9009\": rpc error: code = NotFound desc = could not find container \"3a4c0c433767548e29f9a1fec0546f8cd87a6113c34a9343e15262b3445d9009\": container with ID starting with 3a4c0c433767548e29f9a1fec0546f8cd87a6113c34a9343e15262b3445d9009 not found: ID does not exist"
Jul 27 20:20:09.954348 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:09.954321    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-d3c546-6b6f876dfd-tg88d"]
Jul 27 20:20:09.962894 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:09.962865    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-d3c546-68dbffb748-g5tfw"]
Jul 27 20:20:09.967692 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:09.967671    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-d3c546-68dbffb748-g5tfw"]
Jul 27 20:20:11.675334 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:11.675295    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="548a6292-1d21-4a62-abb1-67bbc6b93ccf" path="/var/lib/kubelet/pods/548a6292-1d21-4a62-abb1-67bbc6b93ccf/volumes"
Jul 27 20:20:11.675725 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:11.675685    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb09fff4-29c5-4ee5-909d-376451de8715" path="/var/lib/kubelet/pods/cb09fff4-29c5-4ee5-909d-376451de8715/volumes"
Jul 27 20:20:15.492372 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:15.492344    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:20:15.592626 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:15.592591    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:20:15.767013 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:15.766928    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:20:15.997267 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:15.997234    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:20:26.496625 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:26.496535    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:20:26.518302 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:26.518272    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-73ce16-5979bbc9bb-lcrpb"]
Jul 27 20:20:26.518548 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:26.518520    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-73ce16-5979bbc9bb-lcrpb" podUID="551bb311-0b2a-429f-8a4d-40d44d3685b8" containerName="payload-processing" containerID="cri-o://3a6387a7b524438fe5cadfc3afcd2483af0467ea7a691be29225d3fc7224aa35" gracePeriod=30
Jul 27 20:20:26.526289 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:26.526260    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-73ce16-975d66488-vs24j"]
Jul 27 20:20:26.526690 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:26.526641    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-73ce16-975d66488-vs24j" podUID="166f3512-54e0-47ef-af08-ef653572c50c" containerName="payload-pre-processing" containerID="cri-o://4d01d169120929633ce52f725026715fbded86f331ed577129827260448a459c" gracePeriod=30
Jul 27 20:20:26.562155 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:26.562127    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:20:26.624537 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:26.624511    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:20:26.685546 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:26.685496    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:20:26.800260 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:26.800239    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-73ce16-5979bbc9bb-lcrpb"
Jul 27 20:20:26.803479 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:26.803461    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-73ce16-975d66488-vs24j"
Jul 27 20:20:26.901943 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:26.901910    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5rp2b\" (UniqueName: \"kubernetes.io/projected/551bb311-0b2a-429f-8a4d-40d44d3685b8-kube-api-access-5rp2b\") pod \"551bb311-0b2a-429f-8a4d-40d44d3685b8\" (UID: \"551bb311-0b2a-429f-8a4d-40d44d3685b8\") "
Jul 27 20:20:26.901943 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:26.901946    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l2qkr\" (UniqueName: \"kubernetes.io/projected/166f3512-54e0-47ef-af08-ef653572c50c-kube-api-access-l2qkr\") pod \"166f3512-54e0-47ef-af08-ef653572c50c\" (UID: \"166f3512-54e0-47ef-af08-ef653572c50c\") "
Jul 27 20:20:26.902156 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:26.902005    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/166f3512-54e0-47ef-af08-ef653572c50c-plugins-config-volume\") pod \"166f3512-54e0-47ef-af08-ef653572c50c\" (UID: \"166f3512-54e0-47ef-af08-ef653572c50c\") "
Jul 27 20:20:26.902205 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:26.902165    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/551bb311-0b2a-429f-8a4d-40d44d3685b8-plugins-config-volume\") pod \"551bb311-0b2a-429f-8a4d-40d44d3685b8\" (UID: \"551bb311-0b2a-429f-8a4d-40d44d3685b8\") "
Jul 27 20:20:26.902450 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:26.902399    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/166f3512-54e0-47ef-af08-ef653572c50c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "166f3512-54e0-47ef-af08-ef653572c50c" (UID: "166f3512-54e0-47ef-af08-ef653572c50c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 20:20:26.902577 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:26.902556    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/551bb311-0b2a-429f-8a4d-40d44d3685b8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "551bb311-0b2a-429f-8a4d-40d44d3685b8" (UID: "551bb311-0b2a-429f-8a4d-40d44d3685b8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 20:20:26.904217 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:26.904194    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/166f3512-54e0-47ef-af08-ef653572c50c-kube-api-access-l2qkr" (OuterVolumeSpecName: "kube-api-access-l2qkr") pod "166f3512-54e0-47ef-af08-ef653572c50c" (UID: "166f3512-54e0-47ef-af08-ef653572c50c"). InnerVolumeSpecName "kube-api-access-l2qkr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:20:26.904324 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:26.904223    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/551bb311-0b2a-429f-8a4d-40d44d3685b8-kube-api-access-5rp2b" (OuterVolumeSpecName: "kube-api-access-5rp2b") pod "551bb311-0b2a-429f-8a4d-40d44d3685b8" (UID: "551bb311-0b2a-429f-8a4d-40d44d3685b8"). InnerVolumeSpecName "kube-api-access-5rp2b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:20:27.003146 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:27.003112    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/166f3512-54e0-47ef-af08-ef653572c50c-plugins-config-volume\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:20:27.003146 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:27.003140    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/551bb311-0b2a-429f-8a4d-40d44d3685b8-plugins-config-volume\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:20:27.003146 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:27.003152    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5rp2b\" (UniqueName: \"kubernetes.io/projected/551bb311-0b2a-429f-8a4d-40d44d3685b8-kube-api-access-5rp2b\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:20:27.003379 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:27.003163    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l2qkr\" (UniqueName: \"kubernetes.io/projected/166f3512-54e0-47ef-af08-ef653572c50c-kube-api-access-l2qkr\") on node \"ip-10-0-129-235.ec2.internal\" DevicePath \"\""
Jul 27 20:20:27.024296 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:27.024262    2558 generic.go:356] "Generic (PLEG): container finished" podID="166f3512-54e0-47ef-af08-ef653572c50c" containerID="4d01d169120929633ce52f725026715fbded86f331ed577129827260448a459c" exitCode=0
Jul 27 20:20:27.024498 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:27.024304    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-73ce16-975d66488-vs24j"
Jul 27 20:20:27.024498 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:27.024341    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-73ce16-975d66488-vs24j" event={"ID":"166f3512-54e0-47ef-af08-ef653572c50c","Type":"ContainerDied","Data":"4d01d169120929633ce52f725026715fbded86f331ed577129827260448a459c"}
Jul 27 20:20:27.024498 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:27.024400    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-73ce16-975d66488-vs24j" event={"ID":"166f3512-54e0-47ef-af08-ef653572c50c","Type":"ContainerDied","Data":"d4a493bd43a19b01f781f3063d8b4802b19ac9c114adb5f0fdc6dc6e09244e00"}
Jul 27 20:20:27.024498 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:27.024462    2558 scope.go:117] "RemoveContainer" containerID="4d01d169120929633ce52f725026715fbded86f331ed577129827260448a459c"
Jul 27 20:20:27.030467 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:27.026251    2558 generic.go:356] "Generic (PLEG): container finished" podID="551bb311-0b2a-429f-8a4d-40d44d3685b8" containerID="3a6387a7b524438fe5cadfc3afcd2483af0467ea7a691be29225d3fc7224aa35" exitCode=0
Jul 27 20:20:27.030467 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:27.026302    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-73ce16-5979bbc9bb-lcrpb" event={"ID":"551bb311-0b2a-429f-8a4d-40d44d3685b8","Type":"ContainerDied","Data":"3a6387a7b524438fe5cadfc3afcd2483af0467ea7a691be29225d3fc7224aa35"}
Jul 27 20:20:27.030467 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:27.026341    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-73ce16-5979bbc9bb-lcrpb" event={"ID":"551bb311-0b2a-429f-8a4d-40d44d3685b8","Type":"ContainerDied","Data":"1b30e23343223755cbe03f3f744821acfbd20b13c2e6fcdfce01e8bd5369d266"}
Jul 27 20:20:27.030467 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:27.026398    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-73ce16-5979bbc9bb-lcrpb"
Jul 27 20:20:27.032710 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:27.032684    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:20:27.037285 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:27.037268    2558 scope.go:117] "RemoveContainer" containerID="4d01d169120929633ce52f725026715fbded86f331ed577129827260448a459c"
Jul 27 20:20:27.037624 ip-10-0-129-235 kubenswrapper[2558]: E0727 20:20:27.037602    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d01d169120929633ce52f725026715fbded86f331ed577129827260448a459c\": container with ID starting with 4d01d169120929633ce52f725026715fbded86f331ed577129827260448a459c not found: ID does not exist" containerID="4d01d169120929633ce52f725026715fbded86f331ed577129827260448a459c"
Jul 27 20:20:27.037747 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:27.037634    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d01d169120929633ce52f725026715fbded86f331ed577129827260448a459c"} err="failed to get container status \"4d01d169120929633ce52f725026715fbded86f331ed577129827260448a459c\": rpc error: code = NotFound desc = could not find container \"4d01d169120929633ce52f725026715fbded86f331ed577129827260448a459c\": container with ID starting with 4d01d169120929633ce52f725026715fbded86f331ed577129827260448a459c not found: ID does not exist"
Jul 27 20:20:27.037747 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:27.037657    2558 scope.go:117] "RemoveContainer" containerID="3a6387a7b524438fe5cadfc3afcd2483af0467ea7a691be29225d3fc7224aa35"
Jul 27 20:20:27.046279 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:27.046262    2558 scope.go:117] "RemoveContainer" containerID="3a6387a7b524438fe5cadfc3afcd2483af0467ea7a691be29225d3fc7224aa35"
Jul 27 20:20:27.046645 ip-10-0-129-235 kubenswrapper[2558]: E0727 20:20:27.046616    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a6387a7b524438fe5cadfc3afcd2483af0467ea7a691be29225d3fc7224aa35\": container with ID starting with 3a6387a7b524438fe5cadfc3afcd2483af0467ea7a691be29225d3fc7224aa35 not found: ID does not exist" containerID="3a6387a7b524438fe5cadfc3afcd2483af0467ea7a691be29225d3fc7224aa35"
Jul 27 20:20:27.046746 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:27.046655    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a6387a7b524438fe5cadfc3afcd2483af0467ea7a691be29225d3fc7224aa35"} err="failed to get container status \"3a6387a7b524438fe5cadfc3afcd2483af0467ea7a691be29225d3fc7224aa35\": rpc error: code = NotFound desc = could not find container \"3a6387a7b524438fe5cadfc3afcd2483af0467ea7a691be29225d3fc7224aa35\": container with ID starting with 3a6387a7b524438fe5cadfc3afcd2483af0467ea7a691be29225d3fc7224aa35 not found: ID does not exist"
Jul 27 20:20:27.049466 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:27.049445    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-73ce16-975d66488-vs24j"]
Jul 27 20:20:27.054894 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:27.054875    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-73ce16-975d66488-vs24j"]
Jul 27 20:20:27.063970 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:27.063948    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-73ce16-5979bbc9bb-lcrpb"]
Jul 27 20:20:27.069784 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:27.069761    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-73ce16-5979bbc9bb-lcrpb"]
Jul 27 20:20:27.675518 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:27.675487    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="166f3512-54e0-47ef-af08-ef653572c50c" path="/var/lib/kubelet/pods/166f3512-54e0-47ef-af08-ef653572c50c/volumes"
Jul 27 20:20:27.675865 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:27.675831    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="551bb311-0b2a-429f-8a4d-40d44d3685b8" path="/var/lib/kubelet/pods/551bb311-0b2a-429f-8a4d-40d44d3685b8/volumes"
Jul 27 20:20:33.112352 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:33.112315    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:20:33.212150 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:33.212090    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:20:33.390515 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:33.390435    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:20:33.598817 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:33.598764    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:20:34.031584 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:34.031556    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-d99cd47d4-bgbtm_c552df59-560f-4daa-8a45-9b2135dbe53e/authorino/0.log"
Jul 27 20:20:40.138962 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:40.138930    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6c9f4fc9b4-zbrwt_6bd0e839-62c0-48cc-a815-d1b944088d2e/copy-manifests/0.log"
Jul 27 20:20:40.145495 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:40.145467    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6c9f4fc9b4-zbrwt_6bd0e839-62c0-48cc-a815-d1b944088d2e/manager/0.log"
Jul 27 20:20:41.683233 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:41.683199    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kvgqm_641fc784-7dcc-4cfc-9e4b-6f1b50f224dd/util/0.log"
Jul 27 20:20:41.689329 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:41.689313    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kvgqm_641fc784-7dcc-4cfc-9e4b-6f1b50f224dd/pull/0.log"
Jul 27 20:20:41.694631 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:41.694614    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kvgqm_641fc784-7dcc-4cfc-9e4b-6f1b50f224dd/extract/0.log"
Jul 27 20:20:41.807757 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:41.807727    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cvkh7_4a03b37a-e996-4c5f-bb3a-3201a0481e74/util/0.log"
Jul 27 20:20:41.813709 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:41.813669    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cvkh7_4a03b37a-e996-4c5f-bb3a-3201a0481e74/pull/0.log"
Jul 27 20:20:41.819072 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:41.819057    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cvkh7_4a03b37a-e996-4c5f-bb3a-3201a0481e74/extract/0.log"
Jul 27 20:20:41.925145 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:41.925119    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732gpq7_5fa21c1c-c090-438b-a17a-eb72ab2c48a4/util/0.log"
Jul 27 20:20:41.930869 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:41.930848    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732gpq7_5fa21c1c-c090-438b-a17a-eb72ab2c48a4/pull/0.log"
Jul 27 20:20:41.936905 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:41.936852    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732gpq7_5fa21c1c-c090-438b-a17a-eb72ab2c48a4/extract/0.log"
Jul 27 20:20:42.044827 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:42.044799    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qrh56_2cf47ec3-ce9f-4a23-9a8a-b45415b64482/util/0.log"
Jul 27 20:20:42.050701 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:42.050682    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qrh56_2cf47ec3-ce9f-4a23-9a8a-b45415b64482/pull/0.log"
Jul 27 20:20:42.057172 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:42.057154    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qrh56_2cf47ec3-ce9f-4a23-9a8a-b45415b64482/extract/0.log"
Jul 27 20:20:42.169983 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:42.169958    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-d99cd47d4-bgbtm_c552df59-560f-4daa-8a45-9b2135dbe53e/authorino/0.log"
Jul 27 20:20:42.408051 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:42.408024    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-5ql7p_bbda13de-368c-468d-ab0d-bee187a9f1e6/manager/0.log"
Jul 27 20:20:42.515593 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:42.515561    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-h55r8_b036279b-e3e1-48fb-ab19-c6d96d41a423/kuadrant-console-plugin/0.log"
Jul 27 20:20:42.749993 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:42.749908    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-dbbw8_d794dacf-50aa-4a2c-91aa-d1fae69da1ae/manager/0.log"
Jul 27 20:20:43.635627 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:43.635600    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b7cdcf968-xsw47_b5cb52b2-b8b8-463c-b3cc-96e8e46f26cd/kube-auth-proxy/0.log"
Jul 27 20:20:44.745273 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:44.745244    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmmh5r_8bf5e0d0-50bf-421d-9032-bef3ae29a6ba/main/0.log"
Jul 27 20:20:44.857816 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:44.857787    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-4wgfz_a498b8bd-3065-48e6-bf8a-34ab47aaae24/main/0.log"
Jul 27 20:20:49.884159 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:49.884126    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-gzr5q/must-gather-5vdqp"]
Jul 27 20:20:49.884622 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:49.884498    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="548a6292-1d21-4a62-abb1-67bbc6b93ccf" containerName="payload-processing"
Jul 27 20:20:49.884622 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:49.884511    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="548a6292-1d21-4a62-abb1-67bbc6b93ccf" containerName="payload-processing"
Jul 27 20:20:49.884622 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:49.884527    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f80492ac-8e54-4cd9-b0ab-8733548af2cc" containerName="payload-pre-processing"
Jul 27 20:20:49.884622 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:49.884535    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="f80492ac-8e54-4cd9-b0ab-8733548af2cc" containerName="payload-pre-processing"
Jul 27 20:20:49.884622 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:49.884562    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="166f3512-54e0-47ef-af08-ef653572c50c" containerName="payload-pre-processing"
Jul 27 20:20:49.884622 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:49.884571    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="166f3512-54e0-47ef-af08-ef653572c50c" containerName="payload-pre-processing"
Jul 27 20:20:49.884622 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:49.884578    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd278b5b-82dd-4048-88f2-1487bb9759e1" containerName="payload-pre-processing"
Jul 27 20:20:49.884622 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:49.884583    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd278b5b-82dd-4048-88f2-1487bb9759e1" containerName="payload-pre-processing"
Jul 27 20:20:49.884622 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:49.884590    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb09fff4-29c5-4ee5-909d-376451de8715" containerName="payload-pre-processing"
Jul 27 20:20:49.884622 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:49.884595    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb09fff4-29c5-4ee5-909d-376451de8715" containerName="payload-pre-processing"
Jul 27 20:20:49.884622 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:49.884602    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="448763ca-737d-4aab-8d3a-d26d0f309f52" containerName="payload-processing"
Jul 27 20:20:49.884622 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:49.884607    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="448763ca-737d-4aab-8d3a-d26d0f309f52" containerName="payload-processing"
Jul 27 20:20:49.884622 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:49.884614    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68006c41-c6cd-40ef-9766-5f590683565b" containerName="payload-pre-processing"
Jul 27 20:20:49.884622 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:49.884619    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="68006c41-c6cd-40ef-9766-5f590683565b" containerName="payload-pre-processing"
Jul 27 20:20:49.884622 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:49.884627    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="449325e2-981e-4358-adfa-75608a671d84" containerName="main"
Jul 27 20:20:49.885384 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:49.884636    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="449325e2-981e-4358-adfa-75608a671d84" containerName="main"
Jul 27 20:20:49.885384 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:49.884651    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="551bb311-0b2a-429f-8a4d-40d44d3685b8" containerName="payload-processing"
Jul 27 20:20:49.885384 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:49.884660    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="551bb311-0b2a-429f-8a4d-40d44d3685b8" containerName="payload-processing"
Jul 27 20:20:49.885384 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:49.884675    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="faf8bee3-858c-45e6-bbc8-9ffcd5538b1b" containerName="payload-processing"
Jul 27 20:20:49.885384 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:49.884684    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="faf8bee3-858c-45e6-bbc8-9ffcd5538b1b" containerName="payload-processing"
Jul 27 20:20:49.885384 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:49.884694    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1421df22-eb16-4ba7-846e-ca6770f226ee" containerName="payload-processing"
Jul 27 20:20:49.885384 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:49.884702    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="1421df22-eb16-4ba7-846e-ca6770f226ee" containerName="payload-processing"
Jul 27 20:20:49.885384 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:49.884854    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd278b5b-82dd-4048-88f2-1487bb9759e1" containerName="payload-pre-processing"
Jul 27 20:20:49.885384 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:49.884871    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="551bb311-0b2a-429f-8a4d-40d44d3685b8" containerName="payload-processing"
Jul 27 20:20:49.885384 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:49.884882    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb09fff4-29c5-4ee5-909d-376451de8715" containerName="payload-pre-processing"
Jul 27 20:20:49.885384 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:49.884893    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="548a6292-1d21-4a62-abb1-67bbc6b93ccf" containerName="payload-processing"
Jul 27 20:20:49.885384 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:49.884906    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="449325e2-981e-4358-adfa-75608a671d84" containerName="main"
Jul 27 20:20:49.885384 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:49.884916    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="68006c41-c6cd-40ef-9766-5f590683565b" containerName="payload-pre-processing"
Jul 27 20:20:49.885384 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:49.884927    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="448763ca-737d-4aab-8d3a-d26d0f309f52" containerName="payload-processing"
Jul 27 20:20:49.885384 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:49.884962    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="1421df22-eb16-4ba7-846e-ca6770f226ee" containerName="payload-processing"
Jul 27 20:20:49.885384 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:49.884977    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="166f3512-54e0-47ef-af08-ef653572c50c" containerName="payload-pre-processing"
Jul 27 20:20:49.885384 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:49.884988    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="f80492ac-8e54-4cd9-b0ab-8733548af2cc" containerName="payload-pre-processing"
Jul 27 20:20:49.885384 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:49.884996    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="faf8bee3-858c-45e6-bbc8-9ffcd5538b1b" containerName="payload-processing"
Jul 27 20:20:49.888579 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:49.888558    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gzr5q/must-gather-5vdqp"
Jul 27 20:20:49.890987 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:49.890962    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:20:49.891096 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:49.890969    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gzr5q\"/\"openshift-service-ca.crt\""
Jul 27 20:20:49.891296 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:49.891279    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gzr5q\"/\"kube-root-ca.crt\""
Jul 27 20:20:49.897897 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:49.897873    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gzr5q/must-gather-5vdqp"]
Jul 27 20:20:49.991371 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:49.991334    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log" limit=1024
Jul 27 20:20:50.010761 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:50.010735    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/554d6b89-f435-4360-8350-990cb824ce34-must-gather-output\") pod \"must-gather-5vdqp\" (UID: \"554d6b89-f435-4360-8350-990cb824ce34\") " pod="openshift-must-gather-gzr5q/must-gather-5vdqp"
Jul 27 20:20:50.010845 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:50.010789    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rjp4\" (UniqueName: \"kubernetes.io/projected/554d6b89-f435-4360-8350-990cb824ce34-kube-api-access-4rjp4\") pod \"must-gather-5vdqp\" (UID: \"554d6b89-f435-4360-8350-990cb824ce34\") " pod="openshift-must-gather-gzr5q/must-gather-5vdqp"
Jul 27 20:20:50.111594 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:50.111550    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rjp4\" (UniqueName: \"kubernetes.io/projected/554d6b89-f435-4360-8350-990cb824ce34-kube-api-access-4rjp4\") pod \"must-gather-5vdqp\" (UID: \"554d6b89-f435-4360-8350-990cb824ce34\") " pod="openshift-must-gather-gzr5q/must-gather-5vdqp"
Jul 27 20:20:50.111775 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:50.111641    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/554d6b89-f435-4360-8350-990cb824ce34-must-gather-output\") pod \"must-gather-5vdqp\" (UID: \"554d6b89-f435-4360-8350-990cb824ce34\") " pod="openshift-must-gather-gzr5q/must-gather-5vdqp"
Jul 27 20:20:50.111965 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:50.111948    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/554d6b89-f435-4360-8350-990cb824ce34-must-gather-output\") pod \"must-gather-5vdqp\" (UID: \"554d6b89-f435-4360-8350-990cb824ce34\") " pod="openshift-must-gather-gzr5q/must-gather-5vdqp"
Jul 27 20:20:50.119484 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:50.119456    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rjp4\" (UniqueName: \"kubernetes.io/projected/554d6b89-f435-4360-8350-990cb824ce34-kube-api-access-4rjp4\") pod \"must-gather-5vdqp\" (UID: \"554d6b89-f435-4360-8350-990cb824ce34\") " pod="openshift-must-gather-gzr5q/must-gather-5vdqp"
Jul 27 20:20:50.199104 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:50.199019    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gzr5q/must-gather-5vdqp"
Jul 27 20:20:50.339195 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:50.339169    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gzr5q/must-gather-5vdqp"]
Jul 27 20:20:50.341076 ip-10-0-129-235 kubenswrapper[2558]: W0727 20:20:50.341040    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod554d6b89_f435_4360_8350_990cb824ce34.slice/crio-c8b26149593635ed8af38cc3e755fddc6aa461ff7ef33b80b10220b2c4bd1636 WatchSource:0}: Error finding container c8b26149593635ed8af38cc3e755fddc6aa461ff7ef33b80b10220b2c4bd1636: Status 404 returned error can't find the container with id c8b26149593635ed8af38cc3e755fddc6aa461ff7ef33b80b10220b2c4bd1636
Jul 27 20:20:51.158030 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:51.158002    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gzr5q/must-gather-5vdqp" event={"ID":"554d6b89-f435-4360-8350-990cb824ce34","Type":"ContainerStarted","Data":"c8b26149593635ed8af38cc3e755fddc6aa461ff7ef33b80b10220b2c4bd1636"}
Jul 27 20:20:52.169222 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:52.169180    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gzr5q/must-gather-5vdqp" event={"ID":"554d6b89-f435-4360-8350-990cb824ce34","Type":"ContainerStarted","Data":"2219cf3149247fe6ff3f479a30391993d03df5c70a841fb68b2e42f7c3096451"}
Jul 27 20:20:52.169751 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:52.169235    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gzr5q/must-gather-5vdqp" event={"ID":"554d6b89-f435-4360-8350-990cb824ce34","Type":"ContainerStarted","Data":"ce945911315788f0f96793c477b5856ed98ac89f026a0ce1d861db1e24a5e30d"}
Jul 27 20:20:52.187360 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:52.187297    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-gzr5q/must-gather-5vdqp" podStartSLOduration=2.430316104 podStartE2EDuration="3.187275081s" totalImagesPullingTime="756.958977ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:20:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:20:52.185668773 +0000 UTC m=+3265.057860750" watchObservedRunningTime="2026-07-27 20:20:52.187275081 +0000 UTC m=+3265.059467057"
Jul 27 20:20:52.616917 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:52.616873    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-2g8q7_3590f4c7-efa9-4c98-ae82-99afc906db15/global-pull-secret-syncer/0.log"
Jul 27 20:20:52.740032 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:52.739999    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-5zsvw_7719045e-d600-437b-aa76-7f03b29fecd9/konnectivity-agent/0.log"
Jul 27 20:20:52.841530 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:52.841498    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-235.ec2.internal_eaacc6d8a8d0349d7d1ac6d641f2511f/haproxy/0.log"
Jul 27 20:20:56.600592 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:56.600559    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kvgqm_641fc784-7dcc-4cfc-9e4b-6f1b50f224dd/extract/0.log"
Jul 27 20:20:56.620873 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:56.620842    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kvgqm_641fc784-7dcc-4cfc-9e4b-6f1b50f224dd/util/0.log"
Jul 27 20:20:56.640059 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:56.639946    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kvgqm_641fc784-7dcc-4cfc-9e4b-6f1b50f224dd/pull/0.log"
Jul 27 20:20:56.667659 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:56.667515    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cvkh7_4a03b37a-e996-4c5f-bb3a-3201a0481e74/extract/0.log"
Jul 27 20:20:56.689452 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:56.689338    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cvkh7_4a03b37a-e996-4c5f-bb3a-3201a0481e74/util/0.log"
Jul 27 20:20:56.713976 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:56.713950    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cvkh7_4a03b37a-e996-4c5f-bb3a-3201a0481e74/pull/0.log"
Jul 27 20:20:56.749122 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:56.749046    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732gpq7_5fa21c1c-c090-438b-a17a-eb72ab2c48a4/extract/0.log"
Jul 27 20:20:56.777866 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:56.777841    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732gpq7_5fa21c1c-c090-438b-a17a-eb72ab2c48a4/util/0.log"
Jul 27 20:20:56.797933 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:56.797896    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732gpq7_5fa21c1c-c090-438b-a17a-eb72ab2c48a4/pull/0.log"
Jul 27 20:20:56.826256 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:56.826227    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qrh56_2cf47ec3-ce9f-4a23-9a8a-b45415b64482/extract/0.log"
Jul 27 20:20:56.847600 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:56.847562    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qrh56_2cf47ec3-ce9f-4a23-9a8a-b45415b64482/util/0.log"
Jul 27 20:20:56.868611 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:56.868579    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qrh56_2cf47ec3-ce9f-4a23-9a8a-b45415b64482/pull/0.log"
Jul 27 20:20:57.405444 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:57.405274    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-d99cd47d4-bgbtm_c552df59-560f-4daa-8a45-9b2135dbe53e/authorino/0.log"
Jul 27 20:20:57.503910 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:57.503878    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-5ql7p_bbda13de-368c-468d-ab0d-bee187a9f1e6/manager/0.log"
Jul 27 20:20:57.539478 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:57.538082    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-h55r8_b036279b-e3e1-48fb-ab19-c6d96d41a423/kuadrant-console-plugin/0.log"
Jul 27 20:20:57.769489 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:57.768797    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-dbbw8_d794dacf-50aa-4a2c-91aa-d1fae69da1ae/manager/0.log"
Jul 27 20:20:59.500214 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:59.500161    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a36a8d24-7a5f-4435-b2e3-5f019285639b/alertmanager/0.log"
Jul 27 20:20:59.522337 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:59.522266    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a36a8d24-7a5f-4435-b2e3-5f019285639b/config-reloader/0.log"
Jul 27 20:20:59.544824 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:59.544797    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a36a8d24-7a5f-4435-b2e3-5f019285639b/kube-rbac-proxy-web/0.log"
Jul 27 20:20:59.566935 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:59.566903    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a36a8d24-7a5f-4435-b2e3-5f019285639b/kube-rbac-proxy/0.log"
Jul 27 20:20:59.592051 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:59.592024    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a36a8d24-7a5f-4435-b2e3-5f019285639b/kube-rbac-proxy-metric/0.log"
Jul 27 20:20:59.614092 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:59.613697    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a36a8d24-7a5f-4435-b2e3-5f019285639b/prom-label-proxy/0.log"
Jul 27 20:20:59.636698 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:59.636674    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a36a8d24-7a5f-4435-b2e3-5f019285639b/init-config-reloader/0.log"
Jul 27 20:20:59.704451 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:59.704400    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-pwsmc_9b70adeb-3b2e-41f2-a157-6b2dfe797669/kube-state-metrics/0.log"
Jul 27 20:20:59.727097 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:59.727062    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-pwsmc_9b70adeb-3b2e-41f2-a157-6b2dfe797669/kube-rbac-proxy-main/0.log"
Jul 27 20:20:59.748598 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:59.748571    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-pwsmc_9b70adeb-3b2e-41f2-a157-6b2dfe797669/kube-rbac-proxy-self/0.log"
Jul 27 20:20:59.776539 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:59.776452    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-55cdff9587-pc94t_94c83f10-a0af-4c99-b91f-14bac2e99cfd/metrics-server/0.log"
Jul 27 20:20:59.904204 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:59.904168    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b65wp_4aa68886-81b2-47c7-9aef-c6d73404122f/node-exporter/0.log"
Jul 27 20:20:59.924924 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:59.924891    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b65wp_4aa68886-81b2-47c7-9aef-c6d73404122f/kube-rbac-proxy/0.log"
Jul 27 20:20:59.950442 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:20:59.949136    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b65wp_4aa68886-81b2-47c7-9aef-c6d73404122f/init-textfile/0.log"
Jul 27 20:21:00.060680 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:21:00.060565    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-p4c8p_3a5c72a0-8769-4caf-80e4-cfc8db418c38/kube-rbac-proxy-main/0.log"
Jul 27 20:21:00.082405 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:21:00.082345    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-p4c8p_3a5c72a0-8769-4caf-80e4-cfc8db418c38/kube-rbac-proxy-self/0.log"
Jul 27 20:21:00.108644 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:21:00.108581    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-p4c8p_3a5c72a0-8769-4caf-80e4-cfc8db418c38/openshift-state-metrics/0.log"
Jul 27 20:21:00.561693 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:21:00.561666    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-65b64784b7-vnbrq_489d2925-8a91-49b6-8a13-0280ea8b8bc2/telemeter-client/0.log"
Jul 27 20:21:00.584084 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:21:00.583979    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-65b64784b7-vnbrq_489d2925-8a91-49b6-8a13-0280ea8b8bc2/reload/0.log"
Jul 27 20:21:00.608547 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:21:00.608451    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-65b64784b7-vnbrq_489d2925-8a91-49b6-8a13-0280ea8b8bc2/kube-rbac-proxy/0.log"
Jul 27 20:21:03.408293 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:21:03.408238    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-gzr5q/perf-node-gather-daemonset-r524l"]
Jul 27 20:21:03.415116 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:21:03.415089    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gzr5q/perf-node-gather-daemonset-r524l"
Jul 27 20:21:03.418088 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:21:03.417578    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-gzr5q\"/\"default-dockercfg-x6f4j\""
Jul 27 20:21:03.422561 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:21:03.422533    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gzr5q/perf-node-gather-daemonset-r524l"]
Jul 27 20:21:03.483995 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:21:03.483969    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7bfd7b9bdd-h69g8_8fb60a14-bcf1-491a-a882-ab2bbfbc02dc/console/0.log"
Jul 27 20:21:03.524377 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:21:03.524328    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-7c97dc4555-qm44q_212c42c2-b2a1-4082-8ea8-2a7f867bd45a/download-server/0.log"
Jul 27 20:21:03.578586 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:21:03.578542    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfjws\" (UniqueName: \"kubernetes.io/projected/f4b7f94f-b46e-41c0-a4de-ca4239c6355e-kube-api-access-pfjws\") pod \"perf-node-gather-daemonset-r524l\" (UID: \"f4b7f94f-b46e-41c0-a4de-ca4239c6355e\") " pod="openshift-must-gather-gzr5q/perf-node-gather-daemonset-r524l"
Jul 27 20:21:03.578973 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:21:03.578849    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f4b7f94f-b46e-41c0-a4de-ca4239c6355e-sys\") pod \"perf-node-gather-daemonset-r524l\" (UID: \"f4b7f94f-b46e-41c0-a4de-ca4239c6355e\") " pod="openshift-must-gather-gzr5q/perf-node-gather-daemonset-r524l"
Jul 27 20:21:03.578973 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:21:03.578892    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f4b7f94f-b46e-41c0-a4de-ca4239c6355e-lib-modules\") pod \"perf-node-gather-daemonset-r524l\" (UID: \"f4b7f94f-b46e-41c0-a4de-ca4239c6355e\") " pod="openshift-must-gather-gzr5q/perf-node-gather-daemonset-r524l"
Jul 27 20:21:03.578973 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:21:03.578943    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f4b7f94f-b46e-41c0-a4de-ca4239c6355e-podres\") pod \"perf-node-gather-daemonset-r524l\" (UID: \"f4b7f94f-b46e-41c0-a4de-ca4239c6355e\") " pod="openshift-must-gather-gzr5q/perf-node-gather-daemonset-r524l"
Jul 27 20:21:03.579196 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:21:03.578997    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f4b7f94f-b46e-41c0-a4de-ca4239c6355e-proc\") pod \"perf-node-gather-daemonset-r524l\" (UID: \"f4b7f94f-b46e-41c0-a4de-ca4239c6355e\") " pod="openshift-must-gather-gzr5q/perf-node-gather-daemonset-r524l"
Jul 27 20:21:03.680081 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:21:03.679996    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfjws\" (UniqueName: \"kubernetes.io/projected/f4b7f94f-b46e-41c0-a4de-ca4239c6355e-kube-api-access-pfjws\") pod \"perf-node-gather-daemonset-r524l\" (UID: \"f4b7f94f-b46e-41c0-a4de-ca4239c6355e\") " pod="openshift-must-gather-gzr5q/perf-node-gather-daemonset-r524l"
Jul 27 20:21:03.680081 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:21:03.680032    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f4b7f94f-b46e-41c0-a4de-ca4239c6355e-sys\") pod \"perf-node-gather-daemonset-r524l\" (UID: \"f4b7f94f-b46e-41c0-a4de-ca4239c6355e\") " pod="openshift-must-gather-gzr5q/perf-node-gather-daemonset-r524l"
Jul 27 20:21:03.680081 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:21:03.680048    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f4b7f94f-b46e-41c0-a4de-ca4239c6355e-lib-modules\") pod \"perf-node-gather-daemonset-r524l\" (UID: \"f4b7f94f-b46e-41c0-a4de-ca4239c6355e\") " pod="openshift-must-gather-gzr5q/perf-node-gather-daemonset-r524l"
Jul 27 20:21:03.680334 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:21:03.680087    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f4b7f94f-b46e-41c0-a4de-ca4239c6355e-podres\") pod \"perf-node-gather-daemonset-r524l\" (UID: \"f4b7f94f-b46e-41c0-a4de-ca4239c6355e\") " pod="openshift-must-gather-gzr5q/perf-node-gather-daemonset-r524l"
Jul 27 20:21:03.680334 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:21:03.680117    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f4b7f94f-b46e-41c0-a4de-ca4239c6355e-proc\") pod \"perf-node-gather-daemonset-r524l\" (UID: \"f4b7f94f-b46e-41c0-a4de-ca4239c6355e\") " pod="openshift-must-gather-gzr5q/perf-node-gather-daemonset-r524l"
Jul 27 20:21:03.680334 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:21:03.680142    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f4b7f94f-b46e-41c0-a4de-ca4239c6355e-sys\") pod \"perf-node-gather-daemonset-r524l\" (UID: \"f4b7f94f-b46e-41c0-a4de-ca4239c6355e\") " pod="openshift-must-gather-gzr5q/perf-node-gather-daemonset-r524l"
Jul 27 20:21:03.680334 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:21:03.680199    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f4b7f94f-b46e-41c0-a4de-ca4239c6355e-proc\") pod \"perf-node-gather-daemonset-r524l\" (UID: \"f4b7f94f-b46e-41c0-a4de-ca4239c6355e\") " pod="openshift-must-gather-gzr5q/perf-node-gather-daemonset-r524l"
Jul 27 20:21:03.680334 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:21:03.680303    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f4b7f94f-b46e-41c0-a4de-ca4239c6355e-lib-modules\") pod \"perf-node-gather-daemonset-r524l\" (UID: \"f4b7f94f-b46e-41c0-a4de-ca4239c6355e\") " pod="openshift-must-gather-gzr5q/perf-node-gather-daemonset-r524l"
Jul 27 20:21:03.680334 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:21:03.680307    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f4b7f94f-b46e-41c0-a4de-ca4239c6355e-podres\") pod \"perf-node-gather-daemonset-r524l\" (UID: \"f4b7f94f-b46e-41c0-a4de-ca4239c6355e\") " pod="openshift-must-gather-gzr5q/perf-node-gather-daemonset-r524l"
Jul 27 20:21:03.690970 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:21:03.690934    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfjws\" (UniqueName: \"kubernetes.io/projected/f4b7f94f-b46e-41c0-a4de-ca4239c6355e-kube-api-access-pfjws\") pod \"perf-node-gather-daemonset-r524l\" (UID: \"f4b7f94f-b46e-41c0-a4de-ca4239c6355e\") " pod="openshift-must-gather-gzr5q/perf-node-gather-daemonset-r524l"
Jul 27 20:21:03.734212 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:21:03.734182    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gzr5q/perf-node-gather-daemonset-r524l"
Jul 27 20:21:03.922089 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:21:03.922060    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gzr5q/perf-node-gather-daemonset-r524l"]
Jul 27 20:21:04.280975 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:21:04.280879    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gzr5q/perf-node-gather-daemonset-r524l" event={"ID":"f4b7f94f-b46e-41c0-a4de-ca4239c6355e","Type":"ContainerStarted","Data":"6b31f8a49dbe89b972bf356223183a09c1163494d66835ee8b59b42f47fbbcee"}
Jul 27 20:21:04.280975 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:21:04.280945    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gzr5q/perf-node-gather-daemonset-r524l" event={"ID":"f4b7f94f-b46e-41c0-a4de-ca4239c6355e","Type":"ContainerStarted","Data":"3a7414786093e86b46e987a8ee48a5ec747565d35976a2731e30f51bfc07f0ae"}
Jul 27 20:21:04.281218 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:21:04.281030    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-gzr5q/perf-node-gather-daemonset-r524l"
Jul 27 20:21:04.303239 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:21:04.303176    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-gzr5q/perf-node-gather-daemonset-r524l" podStartSLOduration=1.303156434 podStartE2EDuration="1.303156434s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 20:21:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:21:04.299368 +0000 UTC m=+3277.171559987" watchObservedRunningTime="2026-07-27 20:21:04.303156434 +0000 UTC m=+3277.175348408"
Jul 27 20:21:05.199859 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:21:05.199812    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-4rxwd_219afe94-ca1b-4a80-a421-e6cb1bb321b1/dns/0.log"
Jul 27 20:21:05.222468 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:21:05.222429    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-4rxwd_219afe94-ca1b-4a80-a421-e6cb1bb321b1/kube-rbac-proxy/0.log"
Jul 27 20:21:05.381546 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:21:05.381517    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-bnrsg_d48f4d00-ad6d-4b8e-9a57-de8ed6729c98/dns-node-resolver/0.log"
Jul 27 20:21:05.988238 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:21:05.988210    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qljqz_e0267f96-3a9d-45d1-b943-92899defe086/node-ca/0.log"
Jul 27 20:21:07.097256 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:21:07.097232    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b7cdcf968-xsw47_b5cb52b2-b8b8-463c-b3cc-96e8e46f26cd/kube-auth-proxy/0.log"
Jul 27 20:21:08.022060 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:21:08.022031    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-9r7sw_6e0976f8-4dd4-4fb2-8458-b426a96a105c/serve-healthcheck-canary/0.log"
Jul 27 20:21:08.683248 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:21:08.683204    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jscrz_8bb804bd-b63d-43b7-8159-80cd1abfab55/kube-rbac-proxy/0.log"
Jul 27 20:21:08.704872 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:21:08.704833    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jscrz_8bb804bd-b63d-43b7-8159-80cd1abfab55/exporter/0.log"
Jul 27 20:21:08.725974 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:21:08.725940    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jscrz_8bb804bd-b63d-43b7-8159-80cd1abfab55/extractor/0.log"
Jul 27 20:21:10.299286 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:21:10.299258    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-gzr5q/perf-node-gather-daemonset-r524l"
Jul 27 20:21:11.436775 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:21:11.436744    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6c9f4fc9b4-zbrwt_6bd0e839-62c0-48cc-a815-d1b944088d2e/manager/0.log"
Jul 27 20:21:11.460709 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:21:11.460680    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6c9f4fc9b4-zbrwt_6bd0e839-62c0-48cc-a815-d1b944088d2e/copy-manifests/0.log"
Jul 27 20:21:12.867287 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:21:12.867254    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-86d698cc8c-g5xw9_15e0f92b-511e-482d-a5ec-69408cf73193/manager/0.log"
Jul 27 20:21:13.766426 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:21:13.766377    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b7cdcf968-xsw47_b5cb52b2-b8b8-463c-b3cc-96e8e46f26cd/kube-auth-proxy/0.log"
Jul 27 20:21:15.507739 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:21:15.507707    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b7cdcf968-xsw47_b5cb52b2-b8b8-463c-b3cc-96e8e46f26cd/kube-auth-proxy/0.log"
Jul 27 20:21:19.659640 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:21:19.659607    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-2fqrl_438069a4-ca90-4995-ba78-32af922abb36/kube-multus/0.log"
Jul 27 20:21:19.714249 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:21:19.714219    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2xk7d_e9a0f4ae-3761-4764-baa1-8d8c005113a0/kube-multus-additional-cni-plugins/0.log"
Jul 27 20:21:19.735425 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:21:19.735385    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2xk7d_e9a0f4ae-3761-4764-baa1-8d8c005113a0/egress-router-binary-copy/0.log"
Jul 27 20:21:19.756089 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:21:19.756058    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2xk7d_e9a0f4ae-3761-4764-baa1-8d8c005113a0/cni-plugins/0.log"
Jul 27 20:21:19.776475 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:21:19.776448    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2xk7d_e9a0f4ae-3761-4764-baa1-8d8c005113a0/bond-cni-plugin/0.log"
Jul 27 20:21:19.795919 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:21:19.795892    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2xk7d_e9a0f4ae-3761-4764-baa1-8d8c005113a0/routeoverride-cni/0.log"
Jul 27 20:21:19.815887 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:21:19.815865    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2xk7d_e9a0f4ae-3761-4764-baa1-8d8c005113a0/whereabouts-cni-bincopy/0.log"
Jul 27 20:21:19.836568 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:21:19.836530    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2xk7d_e9a0f4ae-3761-4764-baa1-8d8c005113a0/whereabouts-cni/0.log"
Jul 27 20:21:20.280161 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:21:20.280110    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-zbjtr_229036e0-7551-4987-98d3-0ccef9c511e5/network-metrics-daemon/0.log"
Jul 27 20:21:20.297382 ip-10-0-129-235 kubenswrapper[2558]: I0727 20:21:20.297357    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-zbjtr_229036e0-7551-4987-98d3-0ccef9c511e5/kube-rbac-proxy/0.log"